The midfielder's 49th-minute strike settled a desperately poor fifth-round tie that was watched by a crowd of just 16,777 at the Stadium of Light, with Southampton's England striker Rickie Lambert guilty of an astonishing miss in the closing stages.
Victory was no more than the Capital One Cup finalists deserved despite Poyet making nine changes, as opposite number Mauricio Pochettino paid the price for not fielding his strongest team.
Gardner's decisive strike was a rare moment of quality in a game which otherwise failed to capture the imagination, although that will matter little to the Black Cats faithful as they dream of a Wembley double.
Southampton, who were unbeaten in seven matches in all competitions before kick-off, headed away on the long journey south wondering what might have been after ending a run of four clashes with the Wearsiders this season without claiming a single victory.
Both Barclays Premier League fixtures ended in draws and Sunderland edged past the visitors in the Capital One Cup fourth round on their way to the final.
But if familiarity breeds contempt, it was commitments elsewhere which shaped Poyet's team selection as he made wholesale changes to the side which lost 2-0 at home to Hull last weekend and handed Argentinian striker Ignacio Scocco a debut.
The Uruguayan manager had admitted in advance that the last thing he wanted amid a cluttered fixture schedule was a replay, but neither his men nor the Saints did much to avoid that situation in a drab opening 45 minutes.
Sunderland had Lee Cattermole making his 100th start for the club, and he proved impressive in the holding midfield role, while Emanuele Giaccherini and Fabio Borini caused problems out wide. The hosts took the game to the visitors for long periods, but created few chances of any note.
They were appealing in vain for a penalty after Luke Shaw and Borini collided just inside the box with 12 minutes gone, but referee Mike Dean waved play on, much to the annoyance of the locals.
Giaccherini threatened to make the most of full-back Nathaniel Clyne's 21st-minute slip as he carved his way into the penalty area, but he and team-mate Gardner collided as both shaped to shoot and the chance went begging.
Borini was aggrieved to be given offside, and with some justification, after being played in over the top by central defender Santiago Vergini. Southampton goalkeeper Kelvin Davis was delighted to see the flag go up as the Italian's shot squirmed through his defences.
Southampton gradually worked their way into the game as Adam Lallana started to make his presence felt, and he was the source of their only real chance when he floated a cross to Lambert, who headed down for the on-rushing James Ward-Prowse.
However, the midfielder scuffed his effort into the ground and goalkeeper Oscar Ustari was able to save comfortably.
But the game finally came to life four minutes after the restart when Gardner capitalised on dithering among the Southampton defenders to lash an unstoppable shot past helpless former Sunderland goalkeeper Davis and in off the underside of the crossbar.
Southampton's response was limp to say the least, with the game deteriorating into an untidy midfield scramble, although Steven Davis did whistle a right-footed shot from long distance past Ustari's left post with 58 minutes gone.
Pochettino made his move with 28 minutes remaining when he replaced Ward-Prowse and Guly Do Prado with Morgan Schneiderlin and Jay Rodriguez, with his side finally showing signs of life.
Schneiderlin in particular made a difference as the visitors started to compete in the middle of the park.
Sunderland might have increased their lead with Sebastian Larsson firing straight at goalkeeper Davis with 15 minutes remaining, while Lallana passed up a gilt-edged chance to level four minutes later when he failed to convert Clyne's inviting near-post cross.
Lambert really should have equalised eight minutes from time when Clyne picked him out in front of goal, but the striker contrived to lift the ball over the bar from just three yards.
Substitute Connor Wickham and Borini both went close in a tense conclusion, but Gardner's contribution was ultimately enough.