A smart finish by captain Piti gave the Madrid side the lead early in the first half and substitute Andrija Delibasic made sure of the points late on.
The visitors could have been ahead on five minutes when Leo Baptistao got through on goal but he hit his shot straight at Celta goalkeeper Javi Varas.
Celta also had an early chance to take the lead, but Park Chu Young failed to connect with a cross from Augusto Fernandez when just a few yards from goal.
Rayo did take the lead on 14 minutes thanks to smart work from winger Lass and forward Piti. The Guinean entered the box from the left-hand side and rounded a defender before putting in a low cross, which Piti back-heeled into the goal.
Abel Resino's side lacked firepower up front, due in no small part to the absence of suspended Iago Aspas, their top scorer and focal point in attack. They had many attempts on goal but could only manage two shots on target over the 90 minutes, resorting to trying their luck from long distance as they failed to penetrate the Rayo defence.
Park missed a great chance to equalise midway through the first half but could not hit the target despite being under no pressure, before Baptistao squandered his second one-on-one chance of the game, shooting wide with just Varas to beat.
The Brazilian then hit the post early in the second half, while Rayo goalkeeper Ruben Martinez pulled off the save of the game to tip away a venomous shot from Augusto Fernandez.
Baptistao was injured and was replaced by Delibasic with 11 minutes to go, and the Montenegro international wasted no time in making his mark on the game.
Jose Manuel Casado got down the left wing and delivered an accurate cross for the forward to tap in from six yards out to seal the three points for Paco Jemez's side.
The result means Rayo temporarily move up to eighth place in the La Liga table while Celta slip below their Galician rivals Deportivo and down to 19th in the table.