With the game goalless early in the second half, Aaron Ramsey raced on to Olivier Giroud's through ball with Swansea right-back Dwight Tiendalli in pursuit.
The Welshman went to ground but referee Howard Webb waved away the Arsenal appeals, and replays appeared to suggest Ramsey had clipped his own heels.
But Wenger firmly believed his side should have been awarded a spot-kick.
"It was a penalty and the referee saw it," he said.
"Do you know why he did not give it? Because he thought the guy had not done it on purpose, and I also think it was accidental.
"But he (Ramsey) did not trip himself, he was caught clearly by the leg of the Swansea player. The referee hesitated and he did not give it."
Swansea manager Michael Laudrup admitted he had believed at the time it was a penalty, although he had not had a chance to see the incident again.
"On the bench my initial reaction was it was a penalty, but I have not seen it again after the game and sometimes you can change your mind when you see things again," he said.
The tie had meandered along with little incident of note during a tepid opening half, but it sprung to life in the second 45 minutes.
Michu, who settled last month's league meeting between the sides, had looked to again be Arsenal's downfall as he gave Swansea the lead just 73 seconds after coming on as a second-half substitute.
But two goals in three minutes from Lukas Podolski and Kieran Gibbs put the Gunners on course to advance, only for Danny Graham, who scored a late leveller against Aston Villa in midweek, to rifle home with three minutes left.
Wenger felt his side should have seen the game out after leading late on.
He said: "We should have won this game, the first half was a bit locked and one-paced but the second was much more open.
"We created many chances and found ourselves 1-0 down, but we showed character and quality.
"Unfortunately I felt on the corner just before their goal, when we conceded it we lacked some calmness and focus.
"It is frustrating to concede a goal like this when we were first to the ball but didn't clear it, but we showed heart and it is good to repeat that."
The result adds another fixture to the two clubs' already crowded schedules, with the replay to decide who will travel to Brighton in the fourth round set to take place on Wednesday, January 16.
And both managers admitted it is an extra game they could have done without.
Wenger added: "I am frustrated as I did not want a replay, but if it is the choice between staying in or going out then I would rather take the replay at the Emirates, and they have a lot of games too."
Laudrup, who started with top scorer Michu on the bench and rested defender Ashley Williams, felt a draw was a fair result, and again praised the character shown by his side.
He said: "I think the overall feeling is that it was a fair result.
"In the second half we made a couple of changes that paid off with Michu's goal, but to be fair to Arsenal they put us under pressure, they kept the ball, took over possession and in those 15 minutes we lost the ball too much.
"Then they scored two very late goals, but again we came back and we again showed the character of our team to score the equaliser."
Graham's strike was his third in three games and highlighted his value to Swansea at a time when speculation linking him with a January move is increasing.
Laudrup plans to speak to Graham about his future and praised the striker's recent performances.
He said: "In the last couple of games we have seen the Danny we saw at the end of pre-season.
"Not just his scoring, but he is making the right runs when we have the ball, and he looks fit and confident and that is positive for us."
He added: "I am talking to the players who have not played regularly, but it does not mean I am telling them they can leave."