British Swimming chief executive David Sparkes yesterday claimed the 18-year-old, who is appearing in new reality television show Splash!, should consider waiting until the end of his competitive career before doing such programmes.
"You can rest assured the next Chinese diving superstar will not have such distractions from training," he said.
And last year Great Britain performance director Alexei Evangulov admitted his worry that Daley's media work was harming his Olympic prospects.
But Andy Banks, the Olympic bronze medallist's coach, said such criticism risked pushing him out of the sport.
Banks told the Daily Telegraph: "There's a danger here that if Tom gets too much grief all the time then he'll say, 'I'm quite enjoying my TV stuff. Diving was great but I've done it for 10 years, so see you later. I'm off'. That wouldn't do anyone any good as far as our sport is concerned.
"For the whole of the 10 years since Tom first walked through the door he has loved every minute of his diving. I'd hate for him to be pushed out of the sport because he no longer enjoys it because people are giving him too much grief.
"At the moment he is still in a position where he really enjoys what he's doing, but watch this space. Four years is a long time and a lot of water is going to flow under the bridge between now and then."