The 22-year-old Caulker wanted a guarantee of regular first-team football, and when then-manager Andre Villas-Boas could not give him that promise at White Hart Lane, the writing was on the wall.
A move to newly-promoted Cardiff followed and, given the captain's armband, Caulker has started every Barclays Premier League game since.
A return to Roy Hodgson's England squad has been his reward and Sherwood, who replaced Villas-Boas in December, has revealed he would have looked to keep the talented defender at the club.
"He wanted game time and more than the previous manager would guarantee him," said Sherwood.
"The club decided they would let the player go and they think they had a good deal. It is up to Steven to build on that and prove the club were wrong to let him go.
"He wanted to go and play. The only thing I could have done is say 'you will play more football'.
"We are happy with the squad we have got and the club made the decision to let him go. It happens. I wish Gareth Bale was still here. We saw in the Champions League what he can do at that level."
Sherwood's previous developmental role at Spurs means he knows Caulker well and he has backed the centre-back to flourish.
"He is a man," Sherwood said. "I mean that in the sense that even though he has been a kid, he has always been a man in stature.
"He knows what is right and wrong. He is willing to learn and he is willing to stand up for his beliefs. I like that.
"There are too many side-parting 'Smedley' kids about. It is good to have a character, someone who might be harder to manage but is a good player.
"Steven is obviously a good player and he has been called into the England squad. He is a regular England international of the future, I have no doubt."
Despite his obvious appreciation of Caulker, Sherwood believes it will be difficult for him to force his way into Hodgson's 23-man World Cup squad - even if he features against Denmark on Wednesday night.
"It depends what options Roy has other than Steven," he said.
"It would be difficult to throw in someone who would only have two caps.
"For the manager's point of view, you can guarantee you will have an honest performance. If he makes mistakes, they are honest mistakes and that is all you want as a manager."