Now, the Tottenham midfielder can taste the World Cup, and he is determined to secure himself a seat on the plane to Brazil next summer.
Townsend, 22, admits he questioned where his career was headed after several 'last-chance saloon' loan spells before finally making the breakthrough at White Hart Lane under Andre Villas-Boas.
The attack-minded midfielder enjoyed the perfect international debut, scoring against Montenegro to cap a live-wire performance at Wembley, which was continued against Poland - a game that found him embroiled at the centre of the infamous half-time space monkey 'joke' from manager Roy Hodgson.
Having done things very much the hard-way, Townsend intends to make the most of the here and now.
"Jon (Obika) and I went (to Yeovil) so young and naive, but also professional, wanting to do everything right and make our first loan successful," said Townsend, who was a second-half substitute in Friday night's 2-0 defeat by Chile at Wembley, but could be back in the starting line-up against Germany on Tuesday.
"We took an electric stove into the hotel, which we were hiding in the cupboard, and used to take turns to cook for each other - anything with pasta in - and cleaning up afterwards.
"It was a humble beginning really.
"Then to play every game and make it successful, I don't think I would be here today [Saturday] if it wasn't for that loan spell."
Indeed, Townsend was on the verge of being released by Tottenham before a change in the Academy management set-up.
"I had grown up a Spurs fan so be told you are not good enough is obviously heartbreaking," he said. "But then 24 hours later I was back in, so I was delighted and wanted to take that second chance."
Townsend feels his experiences away from White Hart Lane helped him return more streetwise for the challenges ahead.
"I thought it was last-chance saloon for me," he said.
"I had been through an unsuccessful loan spell at Watford (in January 2011) and I had three or four months to save my reputation. Thankfully, I went to Millwall (in March 2011) and I excelled.
"Then after the loan at QPR (last season), I had proved myself in the Premier League so it was either now or never to see if I am good enough for a top club.
"The new manager (Villas-Boas) said that he wanted me to be more involved this season, he said he saw my time at QPR and thought I had a future at Spurs and did not want to sell me.
"That gave me the confidence from day one to not feel any pressure to perform or be sold, to just play my normal game."
Townsend watched the last World Cup while on holiday in Cyprus and admits it is somewhat surreal to now be contemplating taking part on such a stage himself.
"I was nowhere near it (England) at that stage, and I did not think it would happen this soon," he said.
"Hopefully now I can keep playing well and secure my place on that plane."
Townsend added: "For every young kid it is a dream to play for your country and at the World Cup.
"Now I have got my foot through the door with my first two caps, I can kind of taste it, and I am not going to let that opportunity slip, to think back in 10 or 15 years what could have been.
"I just want to make the most of my opportunity and whenever I am given the chance, I just have to take it."
Townsend, however, knows there is plenty of competition for the midfield wideman roles - with the likes of Arsenal duo Theo Walcott and Alex Oxlade-Chamberlain soon set to return from injury.
"It is not just a straight shootout (with Walcott) - there are five, six or seven players all fighting for the spots," said Townsend, who withdrew from the England Under-21 squad for the Euro 2013 finals after being charged by the Football Association over breaches of betting regulations.
"It is down to not only international form, but also club form from now until the end of the season as to who will take them."