Magilton, who took over the helm at Loftus Road in the summer, is already under fire with Rangers having failed to win a game in the Championship this season - and chairman Flavio Briatore is not renowned for his patience.
But after being given an almighty scare by their League Two opponents, Ephraim pounced for the winner in the 68th minute and Routledge made sure in the last minute to send the 1967 winners into the hat for the third-round draw on Saturday.
Michael Symes was a handful for QPR all night and he deservedly got a consolation goal for Stanley in stoppage time.
After QPR striker Patrick Agyemang blasted a shot over the crossbar, Accrington, who knocked Nottingham Forest out of the Carling Cup in 2006, nearly took a shock lead.
Symes unleashed a 25-yard effort which was saved by QPR goalkeeper Tom Heaton, on loan from Manchester United.
Heaton was called into action again in the 12th minute to deny Symes for a second time before Gavin Mahon's 18-yard shot was comfortably dealt with by Stanley stopper Alan Martin.
Symes was in the thick of the action again 12 minutes later. After he was up-ended by Mikele Leigertwood, James Ryan produced a decent attempt from 30 yards which Heaton parried.
QPR were struggling to handle Symes and he was on hand to fire another attempt at goal but could not find the target from 18 yards.
The home side responded and Rowan Vine's effort was parried away by Martin.
Magilton made a double substitution at the break as he targeted an improvement in the second half and within 65 seconds Akos Buzsaky blasted a shot wide before Damion Stewart put his effort over the bar from six yards.
QPR were starting to dominate and after Adel Taarabt was off-target, Buzsaky forced Martin into a smart save to deny him from 20 yards.
Stanley were coming under increasing pressure and Martin was called into action again to keep out Gavin Mahon's strike from range.
The home side finally broke Accrington's resistance in the 68th minute through Ephraim
Peter Ramage delivered the cross and Ephraim brought the ball down before producing a neat lob from 18 yards.
They nearly added to their lead moments later only for Martin to come out on top yet again to thwart Vine.
Stanley pressed for an equaliser and Billy Kee and Dean Winnard fired efforts off-target either side of John Miles being denied by Heaton before Routledge thumped home a second.
There was still time for Accrington to deservedly get a goal through the impressive Symes.