The winger came through the ranks at Selhurst Park and went on to make over 100 appearances for the Eagles.
His performances during the club's previous Premier League campaign during the 2004-05 season won him many admirers, and following relegation he left for Tottenham at the start of a nomadic spell which saw him feature for eight clubs in just six years.
But the 28-year-old now finds himself settled at Swansea, and is in the best form of his career to date, relishing the freedom afforded him by Swans boss Michael Laudrup.
Despite the number of teams he has turned out for, Routledge has never faced his first club on their own turf.
It's an experience he is looking forward to.
He said: "It is going to be special for me, knowing I get the chance to play somewhere where it all began for me.
"Hopefully I can be involved and we can get a good result.
"But for me to go back is a big occasion for me personally.
"It's just worked out that it's the first time I actually get to go and play there since I left.
"It's just the way my career path has gone and how they have done, now it's just so happened it's finally lined up so I can get to go back for the first time.
"Probably only the kit man still there from my days, but the owners have done a great job and Ian Holloway is a good manager so hopefully they can do well over the season and hopefully stay in the Premier League for a few more years."
Routledge admits it seems fitting he returns to Palace with Swansea, as they are the two clubs where he has felt the most content.
"It's a fair comment to say that. This is the most settled I've been, this is my home now and now get chance to go back to the place where I learnt everything so it's big occasion.
"Palace taught me everything and I owe them more or less everything and I don't know where I would have been without them.
"When I left it wasn't on great terms (after former owner Simon Jordan branded him a "money-grabber"), but I did it for the right reasons, it is what it is and I am where I am now. Hopefully it will be a good game."
Swansea head into the game on the back of their fantastic win over Valencia in the Europa League, and a good performance in Monday's 2-2 draw with Liverpool.
And Routledge hope the Swans can keep that run going.
He said: "The start we had was always going to be tough knowing we had to juggle the Europa League qualifiers and those first two big fixtures against Manchester United and Tottenham.
"But we got a great result against West Brom and put in a good performance against Liverpool and it was another great performance in Valencia, which shows we're growing as a squad as well.
"So if we put what we did in those three games together I'm sure we could come away with a good result on Sunday."