Madrid had looked set to run riot in Jose Mourinho's 100th league game in charge after scoring twice in the opening 12 minutes through Alvaro Morata and Ramos while going close on a number of other occasions.
However, they had their wings clipped when Ramos was sent off in the 18th minute after picking up two soft bookings in quick succession.
That dismissal allowed Rayo to regroup following their early troubles and, although they could not make anything of their numerical advantage, they at least avoided a humiliating defeat which looked likely following Madrid's barnstorming start.
The result sees Madrid remain four points behind second-placed neighbours Atletico and 16 adrift of pacesetters Barcelona in the Primera Division, while Rayo slip to seventh spot.
Mourinho made four changes to the side that started Wednesday's Champions League draw with Manchester United, and one of the newcomers Morata needed just three minutes to make his mark.
The 20-year-old, making his first league start after being preferred up front in place of Karim Benzema, slotted home at the far post following a low ball from Mesut Ozil across the six-yard box.
Another new face, Kaka, came close to doubling the lead in the sixth minute but he shot wide before Rayo raced down the other end to create their first chance, which saw Alejandro Dominguez flash a shot inches wide.
Ozil was again involved as Madrid did make it 2-0 in the 12th minute, the Germany international sending in a free-kick from the right which Ramos headed home after holding off his marker.
Michael Essien almost increased the home side's lead four minutes later with a dipping curling effort from the edge of the box which Rayo goalkeeper Ruben Martinez did well to tip over.
It was looking like being a long night for Rayo at that stage, but they received an unexpected boost when Ramos was sent off.
The Spain international picked up his first yellow card after brushing Roberto Trashorras off the ball, and then moments later he was handed his marching orders after a Jose Manuel Casado cross hit his hand at close range.
It was tough on Ramos to be sent off for those two incidents, although he may not have helped his cause by pretending the cross had struck his face and not his hand.
Madrid were further incensed three minutes later when the referee refused to show any card after Rayo captain Piti dived in the area in a bid to win a penalty, while the hosts were livid in the second half when Rayo forward Alhassane Bangoura escaped a second booking for handling the ball while controlling a pass.
Mourinho's men need not have worried though as Rayo could not find a way back into the match, Trashorras firing one of the visitors' better chances over in the first half while Rayo also hit the post in the second period, although an offside flag would have seen any goal ruled out anyway.