Top scorer Piti gave Paco Jemez's side a first-half lead which was cancelled out by Robert Acquafresca but then quickly restored by Franco Vazquez.
Piti struck again, this time from the penalty spot, to increase Rayo's advantage but Vicente Iborra set up an exciting finish to the game by scoring with four minutes remaining.
Levante, now without a win in their last seven games, remain 13th in the table.
Despite being forced to field a depleted side owing to injuries to Jose Manuel Casado, Tito, Lass Bangoura, Leo Baptistao and Andrija Delibasic, Rayo were comfortably the better team in the opening 45 minutes, dominating possession and creating many more chances.
Chori Dominguez, starting up front, had the first attempt on target with a softly struck effort which was easy for Levante goalkeeper Keylor Navas to gather, and then created the danger which lead to the goal.
The Argentinian sent a dangerous free-kick towards the Levante goal, Navas could only parry the ball back out into the goal mouth and it eventually fell to Piti, who lashed it into the back of the net to score his 16th goal of the season.
Rayo goalkeeper Ruben Martinez did not have to make a single save in the first half but was grateful to his defenders Alejandro Galvez and Jordi Figueras, who both made decisive blocks to keep out shots from Acquafresca and Michel respectively.
However, the hosts made a much stronger start to the second half. Substitute Ruben Garcia was denied by Martinez five minutes after the re-start but Levante stayed on the attack and equalised immediately after.
No-one picked up Garcia on the left-hand side of the box, calling for an offside on Iborra, but Garcia played on and crossed to Acquafresca, who slotted in from a few yards out.
However, no sooner had Levante got back on level terms, they were behind again.
Rayo launched a quick counter-attack, with left-back Nacho racing forward before playing in Piti. The forward shot first time and Navas could not hold on to the ball, allowing Vazquez to tap home the rebound.
Levante could have equalised just a minute later, when Iborra shot across the goal but it went beyond the post and was a touch too fast for Garcia to get on the end of.
Pedro Rios later came close to equalising with a brilliant effort, bringing the ball down with his chest then volleying just inches wide of the right hand post.
Rayo appeared to have wrapped the game up when Chori was tripped in the box by Garcia, and Piti sent Navas the wrong way to double his tally for the afternoon.
However, Iborra headed a Garcia cross home with four minutes to go to force an exciting finish to the game.
And Acquafresca was presented with a great chance to equalise when he found himself through on goal, but shot too early from outside the area and missed the target.