It was a sweet moment for Kenny Dalglish's side after their win over City in the Carling Cup semi-finals in midweek and against a background of heightened tensions between the two bitter rivals following Luis Suarez's eight-match ban for racially abusing United defender Patrice Evra.
Daniel Agger opened the scoring with a 21st-minute header for Liverpool but Ji-sung Park equalised for United six minutes before the break and a replay looked on the cards until Kuyt grabbed the winner two minutes from time.
The match between QPR and Chelsea was also very much overshadowed by off-the-field issues in the build-up, with the teams meeting for the first time since John Terry was charged with racially abusing Anton Ferdinand during October's Premier League game between the two, a charge he denies.
The Football Association agreed the teams could forgo the traditional pre-match handshake in an effort to reduce tensions, which were ratcheted up yesterday when the Metropolitan Police announced they were investigating a "malicious communication" sent to QPR and addressed to Ferdinand.
Events on the pitch did little to take the spotlight off Terry and Ferdinand, with Chelsea claiming a 1-0 victory courtesy of Juan Mata's 62nd-minute penalty for a debatable foul by Clint Hill on Daniel Sturridge.
In the 3pm kick-offs, once again it was Crawley who provided the shock of the day, the League Two side defeating Championship Hull 1-0 thanks to Matt Tubbs' 57th-minute goal to reach the fifth round for the second successive year.
League One Sheffield Wednesday looked to have followed up their fourth-round win over West Ham by knocking out another Championship side as Clinton Morrison lobbed them ahead at Blackpool but Kevin Phillips equalised in injury time via a dubious penalty.
Last year's runners-up Stoke made a great start, taking the lead in the fifth minute through Cameron Jerome, and Robert Huth added a second nine minutes from time for a 2-0 win at local rivals Derby.
Bolton, the team Stoke beat 5-0 in the semi-finals in 2011, are also through thanks to Chris Eagles' 56th-minute winner against Swansea - visiting keeper Gerhard Tremmel culpable.
Luke Moore had put the Swans ahead but it was quickly cancelled out just before half-time by Darren Pratley, scoring for the first time since a goal for Swansea in last season's play-off semi-final.
A goal from Simeon Jackson five minutes from time earned Norwich a 2-1 win at West Brom in another all Premier League clash after Marc Antoine Fortune had cancelled out Grant Holt's first-half opener.
League Two Swindon were the heroes of the third round after knocking out top-flight Wigan but they went down 2-0 to Leicester as Jermaine Beckford made it five goals in the competition this season.
Birmingham were comfortable 4-0 winners against Sheffield United at Bramall Lane thanks to goals from 17-year-old Nathan Redmond, Wade Elliott and a brace from Adam Rooney.
Rickie Lambert's 31st-minute goal looked like it would be enough for Championship high-fliers Southampton to beat Millwall but Darius Henderson struck four minutes from time to secure a 1-1 draw.
In the all League One clash, Damion Stewart's own goal (12) put Stevenage into the fifth round for the first time in their history as they beat Notts County 1-0.
The biggest shock of the day came in the final match as Brighton dumped out Premier League Newcastle with a 1-0 victory at the Amex Stadium.
It was a deja vu moment for Magpies defender Mike Williamson, who deflected in Will Buckley's close-range effort for the only goal 14 minutes from time.
Williamson also scored an own goal last season when Newcastle lost to then League Two side Stevenage in the third round of the competition.
FA Cup Fourth Round Results
Liverpool 2-1 Manchester United
QPR 0-1 Chelsea
Brighton 1-0 Newcastle
Bolton 2-1 Swansea
West Brom 1-2 Norwich
Blackpool 1-1 Sheffield Wednesday
Millwall 1-1 Southampton
Derby County 0-2 Stoke
Leicester 2-0 Swindon
Sheffield United 0-4 Birmingham
Hull City 0-1 Crawley Town
Stevenage 1-0 Notts County
Report | Reaction