Second-placed Atletico were never in the contest and Radamel Falcao's last-gasp strike came as little consolation to coach Diego Simeone, who saw his troops slip 12 points behind La Liga leaders Barcelona.
Atletico's inability defend crosses was their downfall, while by contrast Rayo defended bravely and attacked impressively to register their fifth victory in their last six matches at the Estadio Teresa Rivero.
Paco Jemez's hosts took three minutes to open the scoring when Piti's cross was not cut out by Daniel Cata Diaz, whose meek attempt to clear the danger left Bangoura an easy tap-in from inside the six-yard box for his third goal in six games.
Rayo forced a good save from Thibaut Courtois in the 23rd minute as Alejandro Dominguez's left-footed effort was parried to safety by the Belgian.
Just after the half-hour the hosts did have their second, Bangoura's cross finding Baptistao who beat Cata Diaz to the punch to turn the ball in at the near post.
Atletico were struggling to stop Rayo from out wide and after 40 minutes Anaitz Arbilla was played in from the right although his shot from just outside the box sailed over the bar.
In a bid to arrest their slide, Simeone replaced Adrian Lopez with Arda Turan at the break, and the visitors created the first decent chance of the second half when Juanfran crossed for Falcao but the Colombian could not get any power in his header as Martinez Ruben gathered.
Ruben was called into action again in the 63rd minute as Filipe Luis beat Alejandro Galvezon down the left and cut in before shooting at the keeper.
Atletico went on the attack again with 10 minutes left with Filipe Luis' mazy run taking him well into the box before a Rayo defender could clear the danger.
Jemez was sent to the stands after arguing with the fourth official before Turan almost reduced the arrears with a right-footed effort from inside the box which flew inches wide of Ruben's left-hand post.
Oliver Torres had a late shot, before an unmarked Falcao finished neatly from close range with the last kick of the game.