With Chelsea and Tottenham not in action this weekend because of the FA Cup semi-finals, the Gunners had the chance to put Champions League qualification back in their own hands.
However, the Canaries - who still need a couple of wins to secure top-flight status - produced a resolute battling display and took the lead on 56 minutes through Michael Turner's close-range header.
With the clock ticking, Arsenal were handed a way back into the match with just five minutes left when the assistant referee spotted a shirt pull on Olivier Giroud by Kei Kamara.
Mikel Arteta kept his cool from the spot to level, before Giroud's close-range effort turned the match around and substitute Lukas Podolski wrapped things 3-1 up in stoppage time.
"We left it late, and it was of course unexpected to be 1-0 down at that stage of the game because we were on top and missed chances," said Wenger.
"Norwich we knew were strong on set pieces and they punished us for that.
"We needed spirit and also nerves, we have shown both and in the end got the win we deserved.
"But credit to Norwich, they have played very well and if they play like that, then they will not go down. They defended well and gave us problems.
"We missed chances and from then on it was important to show absolute resilience and we did it."
Norwich bitterly protested the penalty decision, which came following a corner and was not given initially by referee Mike Jones until he consulted with Richard West on the far side.
Wenger, though, felt justice had been served.
He said: "Honestly, I have not seen it but people have told me it was a penalty. Television confirmed the penalty.
"Maybe it was not a corner, but they tell me as well it was not a free-kick (for Norwich's goal)."
Arsenal host Everton on Tuesday night and then head to Fulham before Spurs play against Manchester City next weekend, and so could open up a significant gap on their north-London rivals.
Wenger, though, stressed: "At the moment, the position is not important, has no real meaning, what is important for us is to keep our run going and the consistency of our wins.
"It is really tight, and the team who is the most consistent will get there.
"At the moment, we have that consistency, so we want to focus on that.
"We have another big game on Tuesday night and have a good opportunity that we want to take."
England midfielder Jack Wilshere returned from an ankle injury, before being substituted in the second half.
Wenger said: "He worked very hard. He has been in full training since Wednesday. Walcott has only been in full training since yesterday, we thought he had enough in the locker to play an hour.
"I maybe would not have put him straight away in if Rosicky was fit.
"I did not want to destructure the team who have been successful and he was the best for this position."
Wenger rejected reports Podolski was frustrated at his lack of game time.
"Look how he played when he came on. As soon as you have a player who does not play for three games, the papers get a story like that," the Arsenal boss said.
"He looks very happy to be here."
Chris Hughton felt Norwich had not been given the rub of the green by the officials.
He said: "My first thought is that it was not a corner in the first place, but what I am more aggrieved about is there is contact with Kei, but he swivels and plays the ball.
"You have a linesman who makes a decision from 40 or 50 yards away and the referee who is eight yards away and has the perfect view.
"Their third goal was also offside.
"I have had a private conversation with the referee, but I am still not convinced."
Hughton must now regroup his squad for the visit of Reading and games with Stoke and Aston Villa, which will ultimately define the end to their campaign.
"Our season was not going to be about what happened here or at Old Trafford," said the Norwich boss.
"We have five games now, three at home, so we need to get enough points to keep us where we want to be.
"Generally we have been good at home, so there are opportunities for us, but we have to take those."