Casey finished 19 over par the last time the event was played in Rochester, but 10 years on the 36-year-old went into Friday's second round just two shots off the lead.
"I don't remember much of it," Casey said of 2003, when journeyman Shaun Micheel won by two shots to join Mike Weir, Jim Furyk and Ben Curtis as that year's major champions.
"That was very early in my career, probably my third US major, and I really didn't know what to expect. I was very much a rookie and maybe a little out of my depth just not knowing how to play these golf courses. It can be a scarring experience if you're not ready for it."
Casey has yet to record a top-10 finish in the event but is getting close to the form which made him world number three in 2009 before problems with form and fitness sent him tumbling down the world rankings.
The former Ryder Cup star was 169th in the world before winning the Irish Open in June, so it was perhaps no wonder that he made sure to check out his name high on the leaderboard.
"It's a while since my name has been up there. It felt good," said Casey, who broke his collarbone snowboarding at the end of 2011 and also went through a painful divorce around the same time.
"You don't realise how much you miss something until it's gone and sitting on the sofa watching the big events of the last year and a half I have had lots of extra motivation to get back.
"It added another reason for getting out there and working hard and things like Justin (Rose) winning the US Open is good fuel for the fire."
Former US Open champion Furyk and Masters champion Adam Scott shared the lead on five under, with Lee Westwood and Canada's David Hearn a shot behind.
Furyk, looking to become the second 43-year-old in succession to win a major after Phil Mickelson's Open triumph, was set to equal the course record of 64 when he reached six under, but bogeyed the last for a five-under 65.
And Scott even had dreams of shooting the first sub-63 round in major history when he stormed to the turn in 30 with five birdies in succession and reached six under after 14, only to bogey the 16th.
"There's nothing to complain about after a 65 but I was feeling it when we had the rain delay (70 minutes due to the threat of lightning) and I came back to a 233-yard par three for my first swing," Scott said.