Dale were trailing 1-0 for 80 minutes of the clash, Ivan Toney having headed the Cobblers in front just five minutes in.
But Reuben Noble-Lazarus breathed life into Dale’s Cup dreams with an 85th-minute leveller and Oliver Lancashire headed the winner three minutes into time added on.
The Cobblers had set off on the right foot, taking the lead when Darren Carter drove a free-kick from the right towards the near post and Toney rose unchallenged, glancing a firm header past Conrad Logan from six yards.
That early blow had Rochdale on the ropes and with the home side at sixes and sevens, Northampton twice went close to doubling their advantage.
Firstly, a quick passing move opened up the Dale defence, John-Joe O’Toole squaring for Emile Sinclair 10 yards from goal only for the pacy wideman to make a hash of his finish.
There was another let off for Dale in the 15th minute when Marc Richards was on target with his fierce drive but was parried by Logan.
Dale levelled in the 85th minute when Reuben Noble-Lazarus, making his home debut since arriving on loan from Barnsley, slammed a smart finish into the top corner.
And deep into time added on Lancashire met Rose’s cross to secure Dale’s passage into the next round.