Manchester City goalkeeper Hart has found himself in the spotlight once again after the latest in a series of high-profile blunders which allowed Fernando Torres to fire Chelsea to a last-gasp 2-1 victory at Stamford Bridge last weekend.
However, amid speculation that manager Manuel Pellegrini is ready to drop the 26-year-old, he has received the backing of Tottenham counterpart Brad Friedel and now Clemence.
The former Liverpool and England number one told talkSPORT: "Joe Hart is the best goalkeeper England have got at this moment, but he is going through a difficult time, and I'm pretty sure he would be the first to admit that.
"It was interesting listening to Brad Friedel, and I agree with him. We all go through bad times and you need people to stand by you and have confidence in you.
"What we have to be careful of - and we are terrible for it in this country - is getting on somebody's back and destroying their confidence.
"I've seen it happen to Paul Robinson when that ball bounced over his foot in Croatia, and Scott Carson was never forgiven for the Croatia game at Wembley (where defeat for England cost them a place at Euro 2008).
"It just affects them and some don't recover from it. I don't want to see that happen to Joe. He's a wonderful lad, a great character and he has got a strong personality.
"They say as a goalkeeper you have to be daft but, at this particular time, you have to have thick skin because you are going to get more criticism than anyone else and you have to be able to rise above it."
Romania international Costel Pantilimon got the nod for Wednesday night's 2-0 extra-time victory at Newcastle in the Capital One Cup, and Clemence believes the competition he provides could prove key.
He said: "It's good to have somebody that makes you think, 'If I make a mistake here, I'm under pressure and I might not be in the team next time'. Intense competition does keep you on your toes.
"At Liverpool, they used to change the reserve goalkeeper every three years to make sure I had a new challenger so I didn't get complacent.
"It does help, without a shadow of a doubt. John Ruddy and Fraser Forster have come a long way in these last few years and are doing really well themselves and putting pressure on him (with England). Ben Foster, when he gets fit again, will put pressure on Joe as well."