Clarke did not deny telling England's number 11 to "get ready for a ******* broken arm" as he took guard to face fast bowler Mitchell Johnson at the Gabba, a comment Anderson was visibly unimpressed by.
England captain Alastair Cook, who saw his team capitulate to a crushing 381-run defeat in Brisbane, admitted it was "war" on the pitch and took issue with Australia batsman David Warner describing England counterpart Jonathan Trott as "weak".
Clarke though felt everyone involved need to accept it was all part of the banter surrounding an Ashes series.
Put on the spot about his alleged comment to Anderson, Clarke said: "You must have great ears.
"Through my career, there has always been banter on the cricket field - and I cop as much as I give, that's for sure.
"That's part and parcel of the game ... but all the England players know we certainly respect them, and we understand we have to be at our best to beat them.
"I've heard a lot worse said on a cricket field than what the Australia players or the England players said throughout this Test match."
Cook stopped short of calling for any disciplinary measures by the International Cricket Council against Warner for his remarks about Trott, but there was no mistaking his disapproval.
"I think the comment by David Warner was pretty disrespectful to any professional cricketer really," Cook said.
"On the pitch it's pretty much a 'war' anyway, so there's always going to be a few battles and a few words.
"That's the way people want to watch cricket being played - tough, hard cricket - which, on the pitch, is fine.
"When you play each other for quite a few games in a row the niggles do increase."
Clarke claims the tough-talking merely demonstrates how much the Ashes means to those involved.
"It's because both teams want to win so badly," he added.
"We all respect the history and traditions of the game. Australia v England has always been competitive ... I think that's great for the game.
"I certainly understand and respect that there's a line - and both teams shouldn't overstep that line - and I hope that hasn't been the case through this Test.
"I can only talk from our team, but there's not one player on the England team that anyone has a personal vendetta against, or anybody disrespects as a cricketer."
He is just doing all he can to get the best out of Australia.
"It's about trying to do whatever you can to help your team have success," he said.
"Pushing the line, not overstepping it ... for someone like Warner or Mitch or Watto (Shane Watson), they really love that competitive battle.
"They love opposition talking to them or having a crack back - that is what drives them.
"I'm sure there are plenty of guys in the England team (like that) - I think Jimmy Anderson has made it clear he likes that battle as well."
While Clarke can at least reflect on an overdue Australia success, his opposite number must work out quickly how to negate the threat of Johnson.
The left-armer's nine-wicket match haul helped hustle England out for 136 and then 179 - with Trott his most hapless victim.
"We've been outplayed pretty much after the first day," said Cook.
"We've certainly lost the first battle; we now need to go away and regroup, stay strong as a unit and really be honest with ourselves.
"Certainly there won't be any hiding away.
"We hold our hands up ... we've been found short in this game."
Johnson is England's most obvious problem - along with number three Trott.
"We've got to look at the way we're going to play him," added Cook.
"You can't brush the issue under the carpet. He's hurt us, and we're going to have to show our ability in the next game."
As for Trott's struggles against the short ball, Cook said: "He's had a tough game, he knows that.
"But you have to remember the guy is class.
"He's had a little blip in these last couple of games. But he's a class player, and class players bounce back."
Trott can expect no respite, even if a slower surface is likely for the second Test in Adelaide.
"He's been thinking about the short ball, from what I've seen," said Johnson.
"There were a couple of nice ones that were zinging past his nose.
"As a fast bowler, you give a bit of a stare and have a look into the eyes.
"I don't know ... there might have been a little bit of fear there.
"I'll keep doing it. It's working."