Guzan played in every round in the lead-up to the 2010 final only for the then number one Brad Friedel to be preferred against Manchester United, who lifted the trophy after a 2-1 win.
Now Guzan, who is the current first choice ahead of Shay Given, is hoping to help Villa take another step towards this season's final with victory at League One side Swindon.
The United States international said: "Being left out for the final was one of biggest disappointments of career.
"It was quite frustrating and a difficult pill to swallow.
"But that's behind me and I think that has made me a stronger goalkeeper and a stronger person.
"If we can get back there to Wembley again, that would be great."
Guzan admitted: "It would definitely be good to back there and play. But that's a long way off.
"We've got a massive game on Tuesday where we have to take care of business first with.
"For me, it is just a case of one game at a time and keep moving forward.
"We will try to get the win and into the hat for the next round and if we can do that it will be a job well done."
Villa caused one of the upsets of the previous round when they overcame Premier League champions Manchester City 4-2 at the Etihad Stadium.
Now it will be their turn to be underdogs when they travel to the County Ground.
Villa manager Paul Lambert said: "The response to a defeat against Southampton the previous weekend in beating Manchester City in the previous round at their place was brilliant.
"But that's gone now. It's another tough game we are facing on Tuesday.
"It's not going to be a foregone conclusion. I've been there a few times with Norwich."
Villa will be forced into at least one change against Swindon with left-back Joe Bennett suspended after his sending-off for two bookable offences against Norwich.
Lambert confirmed winger Charles N'Zogbia will be side-lined for "some weeks" after suffering knee ligament damage in training on Thursday.