The off-spinner, who announced his retirement from all forms of cricket over the weekend with immediate effect, was careful to name no names when he used the description, but it was widely interpreted as being about players in the England team.
With media and former players pondering who he could have been referring to, the 34-year-old replied via Twitter to one of them - Ashes-winning captain Michael Vaughan - to correct the misconception, and on Christmas Eve further clarified his views in his column in The Sun.
Swann wrote: "I wasn't talking about Kevin Pietersen or any England player when I made comments during my retirement press conference.
"Myself and Kev have had a very good working relationship and his reintegration back into the team has been spot on.
"The truth is I was talking about other players around the world who don't carry themselves in a manner I think is befitting of wearing an international shirt.
"With hindsight, I should have clarified there and then that I wasn't talking about England players, rather than leaving it hanging in the air."
England head into the Boxing Day Test in Melbourne having already surrendered the urn following three emphatic defeats, but Swann insisted the tourists' camp remained united.
"The England dressing room is a very happy place." he said. "We are disappointed with how this series has gone, but we still support each other.
"Nobody is shirking responsibility. Everyone is being honest with each other and is desperate to put things right.
"There is no way I would think any England players are up their own backsides. They handle themselves superbly."
He added: "The last thing I want is to leave on any sort of sour note.
"I went out for a meal with eight or nine of the lads on Sunday evening. We had a great night reminiscing, talking about the good times and even singing Christmas songs. It was brilliant.
"They're my best mates and I certainly wasn't having a pop at any of them. I'd never do that."