Alberto Bueno scored the only goal of the game on 29 minutes in what was an even game of few clear goalscoring opportunities for either side.
Rayo remain 19th in the Primera Division table, level on points with Valladolid and two behind Osasuna, while Getafe remain 10th and have now lost four consecutive league games.
Getafe made the brighter start, with Pablo Sarabia and Sergio Escudero having early half-chances, while Rayo struggled to get out of their own half.
The visitors looked to have been awarded a penalty after 10 minutes for an apparent handball, but referee Carlos Clos Gomez decided to consult with his linesman, who gave a corner instead.
The visitors' next opportunity was an overhead kick from Bueno, which did not trouble Miguel Angel Moya in goal.
Paco Jemez's side were getting more and more comfortable and took the lead just before the half hour mark, Bueno turning in a low cross from Alejandro Galvez from six yards out.
Rayo stopper Ruben Martinez had to be alert to keep out a header following a free-kick, while his opposite number Moya almost got himself in a messy situation when coming out to claim a through ball and failing to gather, but his defence did enough to prevent Nery Castillo from taking advantage.
Borja Fernandez tried his luck from 30 yards early in the second half, while Angel Lafita thought he should have been given a penalty when tugged in the box, and Ciprian Marica was able to shoot on goal but unable to make Martinez work.
As in the first half, Rayo took their time to get into the game after the break, but then created a flurry of chances.
Substitute Adrian Embarba shot just wide of the far post from a difficult angle, while Adrian Gonzalez squandered a better opportunity later on, pondering for too long on the ball when through on goal and failing to beat Moya.
Getafe peppered the Rayo box in the dying minutes and the visitors were grateful to Raul Baena and then Nacho for blocking the ball on the goalline, and Jemez's side hung on to claim only their fifth victory of their season, their third away from home.