Dufner equalled the lowest round in major championship history with a superb 63, although the moment was bittersweet after he left a birdie putt from 12 feet for an historic 62 woefully short.
It was the 26th 63 recorded in a major and came just hours after Webb Simpson had equalled the previous course record of 64 set by Ben Hogan in 1942 and matched by Curtis Strange in the 1989 US Open.
"It's tough," said Dufner, who finished nine under par to lead by two from Masters champion Adam Scott and American pair Jim Furyk (68) and Matt Kuchar, who carded a 66 after dropping his first shot in two days on the 18th.
"I showed a little bit of nerves there leaving it short. It's one you would like to gun and have a chance at history but it was probably the worst putt I hit of the day, which is a little disappointing.
"But all in all it's a 63 and name on top of the leaderboard, so that's a great position to be playing from. To join history, to shoot a 63 in a major, pretty unbelievable.
"The history of the game is something that is dear to my heart. To be part of history is a neat accomplishment, I never thought a guy from north-east Ohio would be able to do these things. Hopefully it will propel me to a great weekend."
In the final round of the 2011 US PGA, Dufner led by five shots after nearest challenger Keegan Bradley triple-bogeyed the 15th, only to find water on the same hole minutes later. After doing well to escape with a bogey there, he failed to get up and down from sand on the next and then three-putted the 17th.
Both men parred the 18th to go into a three-hole play-off which Bradley won by a shot to become the first player to capture a major with a long putter.
Simpson, who became the second the following June at the US Open, was seven under par after 15 holes and needed to play the last three in one under to shoot 62, but could only manage a bogey and two pars.
Dufner carded an eagle on the second and three birdies to be out in 31 and picked up further shots at the 11th, 13th and 16th before agonisingly missing from 20ft for birdie on the 17th.
"I hit a great putt there," added Dufner, who was playing with the last man to shoot 63 in a major, Steve Stricker doing so in the 2011 US PGA. "I thought that one was going to go in. I didn't think the one on 18 was going to go in when it came off."
Tiger Woods, seeking a 15th major title but a first since the 2008 US Open, was 10 shots off the pace after adding a 70 to his opening 71.