Stevenage, who knocked out Championship side Reading in the last round, went ahead in the 12th minute when Stewart deflected Luke Freeman's strike into his own net.
Stewart almost made amends at the death but Chris Day clawed his effort off the line before Lawrie Wilson wasted a glorious chance to seal Stevenage's progress.
The home side dominated the opening exchanges and Darius Charles blazed over the crossbar after just seven minutes but Freeman's industry led to Boro's opener five minutes later.
The 19-year-old has continued to impressive since signing permanently from Arsenal earlier this month and after drawing goalkeeper Stuart Nelson wide he fired in a shot that Stewart could only deflect in.
Notts County woke up after the break and Alan Judge forced Chris Day into a fine save before the goalkeeper blocked Lee Hughes' weak effort.
Stevenage should have doubled their advantage when Charles spurned a good chance after 64 minutes before Day superbly saved Stewart's stoppage-time effort and Wilson wasted a three-man counter-attack.