The 25-year-old's last-gasp effort secured a 3-2 victory against the npower Championship promotion-chasers at St Mary's and sets up a home tie with Bolton.
Millwall started the match strongly and broke the deadlock after 17 minutes when Liam Trotter struck home from close range. Adam Lallana pegged the visitors back with a superb solo goal and after the break Rickie Lambert fired Southampton into the lead.
For all of the home side's attacking intent, goalkeeper Bartosz Bialkowski was being called into action regularly and was beaten by former Saints loanee Dany N'Guessan with 11 minutes remaining.
The match looked to be heading for extra time until Feeney cut in from the left to curl in a stunning winner in stoppage time.
In the night's other fourth-round replay, Blackpool earned a trip to Everton after a scintillating 3-0 demolition of Sheffield Wednesday at Hillsborough.
The game was effectively won in the opening 15 minutes as Matt Phillips continued his red-hot scoring streak with a precise finish and Lomana LuaLua curled in a superb effort soon after.
The Seasiders produced an outstanding attacking display and they made it three shortly after the break with another goal of sheer quality, Ludovic Sylvestre bending in from the edge of the area.