After securing just five points in 2013, the Gunners were set for a second successive home draw until 12 minutes from time.
Substitute Lukas Podolski got the goal, although it was Geoff Cameron who did the damage by deflecting the German's free-kick past his own goalkeeper.
Arsenal remain sixth in the standings after the 1-0 win, but closed the gap on fourth-place Tottenham to a single point - much to Wenger's relief.
"I said already before the West Ham and Liverpool games, that we are in a position where we cannot drop points," the Gunners boss said.
"We dropped two points against Liverpool but we couldn't do it again today [Saturday].
"What is more important is the quality of performances is there. The spirit is there.
"The target to be in the top four and to focus on the two cups we are still in is very important now as we have strengthened our belief and our confidence."
Saturday's win was only Arsenal's second in the league since the turn of the year but one Wenger believes his side deserved.
"They made it difficult for us," he said. "They were organised, defended very deep.
"We didn't find the space but we kept going and, while we are a bit lucky with the goal, we deserved this win.
"We had chances, their 'keeper made a lot of saves. We played in their half a lot and overall I think we deserved to win the game."
Like so many previous clashes between the clubs, there were several unsavoury flash points.
Michael Owen aimed a punch at Mikel Arteta after reacting badly to a challenge from the Spaniard towards the end, which came after a strong Ryan Shawcross challenge on Laurent Koscielny.
The latter resulted in a booking for the Stoke captain, who was booed throughout due to the tackle that broke Aaron Ramsey's leg three years ago.
"It looked to me [like it should have been a red card] but I have to look at it again as I am maybe not completely objective when it comes to Ryan Shawcross," Wenger said.
"That is why I will be a bit cautious with my statement but I have not seen the Michael Owen incident."
As the former England striker was not booked by referee Chris Foy, he could now face retrospective action from the Football Association - something Stoke manager Tony Pulis was not keen to dwell on.
"I think Arteta's challenge was a poor challenge, but Michael should not do that," he said.
"That should be left to the referee and I think if you look at Glenn Whelan's challenge on Santi Cazorla, it is a really good challenge and plays the ball.
"But not only has Chris given a free-kick but he has also booked Glenn, which was disappointing.
"Arsenal are a great side with great players - they brought Cazorla and Podolski on. They are a fabulous team with fabulous players."
Pushed further on the Owen incident, Pulis added: "We'll see what they say. Obviously I am going to be biased as it is my football team.
"I have seen the challenge, I think it poor and should be a free-kick.
"If Michael has done what he has done and they look at it, then they will deal with it."
Pulis cut a frustrated figure in the post-match press conference after seeing a point slip out of his side's grasp late on.
"Obviously we're disappointed," the Welshman said.
"We've leaked a lot of goals in the last couple of games and although we've played Man City twice, Chelsea and now Arsenal it has been a difficult run.
"But that was the same at the beginning of the season and we only have Manchester United and Tottenham at our place now.
"We've got most of the big ones out of the way now."