Arsene Wenger's side were 2-0 down at Emirates Stadium on Sunday heading into at half-time, with another shot at a long-overdue trophy looking to have passed them by.
However, a rousing second-half performance turned the tie around with a brace of penalties from captain Robin van Persie sandwiching a fortunate effort from the England winger as Arsenal secured a place in the last 16 against either Sunderland or Middlesbrough next month.
Walcott - signed from Southampton in 2006 - is one of a string of senior players yet to lift silverware with the Gunners. However, the 22-year-old is out to put that record straight this season.
He said: "I would like to look back on my career at the trophies I have won. There are none yet. This is a competition that we can win.
"A couple of big teams have gone out and, if we can come from 2-0 down, hopefully we can beat anybody now."
Walcott revealed a rousing address from Van Persie during the interval helped lift Arsenal for the second half.
He said: "Robin is the captain and showed again how much he believes in this team and how much he wants to win this competition."
After a disappointing opening half, Walcott came to life in the second, his penetrating runs eventually rewarded on 56 minutes when Alan Hutton's clearance from his goalline cross bounced back off the England man and into the Villa net.
Walcott maintains he does not pay any attention to his critics.
He said: "I just want to play my game and show people what I can do. I am the best judge of my performance, not anyone else.
"I was not happy with my last few games but, hopefully, now I can try to get back to a level I should be at. Everyone should realise I am a good judge of my character."
Arsenal were boosted by the return to action of full-back Bacary Sagna for the closing minutes on Sunday, the France defender having been out since breaking his ankle at Tottenham in October.
"It has been quite difficult all these months - four is a long time. I just needed to get back as quick as possible and now I finally am," Sagna told Arsenal Player.
"I just need to work even harder to get back because I have only had two training sessions and it is not easy, but I will do it.
"I am just biding my time to start a game now but I am not rushing.
"I just want to give the maximum to get myself into the best position."
Villa manager Alex McLeish, meanwhile, has called on the Football Association to take a look at a stray elbow from Van Persie, which caught defender Carlos Cuellar as they jumped for a high ball in the 64th minute.
Should it be established referee Mike Jones witnessed the incident at the time, then no retrospective action can be taken by the disciplinary authorities. However, if the official did not, then the FA could investigate further, with possible sanctions against the Arsenal captain.
McLeish felt Villa - who were without injured duo Marc Albrighton and James Collins - paid a heavy price for some poor judgements, with both goalscorers conceding fouls which were punished by penalties.
"We made a few key errors which cost us in the Arsenal flurry, but I cannot complain about the players' effort because they went right to the very end," McLeish said.
"It would have been nice to see them shut the game out and show their maturity, that they can be winners in this team."
Republic of Ireland defender Dunne, 32, insists the side will take on board the lessons of defeat.
"We just have to learn from it," he said.
"We have pulled ourselves into a situation where we can win the game and get through to the next round and we have not seen it through.
"Hopefully now we will learn from that experience and we know, next time we are in that situation against a team like this, how to see it through."