West Ham manager Allardyce criticised Cole for venting in public when he was withdrawn after just 40 minutes of his side's 3-0 home Barclays Premier League loss.
Allardyce called on the former Lille and Liverpool playmaker to keep any future dissatisfaction behind closed doors.
Former England midfielder Cole moved earlier in the week to dispel reports he wants to leave West Ham in January, in the wake of Saturday's defeat against his former club.
Allardyce said the situation was "blown out of proportion", but still urged 32-year-old Cole to show greater professionalism.
"We can't lose our confidence," he said. "There's a squad system here that has to be used to the benefit and the best of everybody.
"I totally understand nobody wants to be substituted before half-time.
"But purely from a tactical point of view, I had to change something, the way the game was at that stage, and that's my job.
"Joe was the one with Jack Collison.
"We don't hear about how disappointed Jack Collison was.
"So all players have to accept it if I bring them off, they have to accept it's my responsibility to change the way the team's playing, to get us back in the game.
"It's a difficult period but players show their true team spirit in situations like this.
"They show their true togetherness in situations like this, and that's what they've got to do, they've got to stick together.
"They can't fragment and think it's all about them, it's not just about them: it's about the whole team and everything that goes with it to achieve the right results."
Allardyce said he is just as frustrated with West Ham's lack of firepower and form as anyone else.
But ahead of Saturday's relegation six-pointer with Fulham at Upton Park, he challenged his squad to stay calm and collected to help pull away from the league's lower reaches.
Allardyce said West Ham do not need any distractions in a "defining week" where they travel to Fulham in the Premier League on Tuesday night.
He said: "By the end of the next week, we'll either be sat in the upper reaches of the bottom half of the table, or we'll still be around where we are now.
"It's a big defining week for us; the most important thing is to win points against your near rivals.
"The trouble is with you lads you blow it all out of proportion in the media, with stories coming out about 'I'm off, I'm leaving'.
"I prefer that type of reaction in terms of showing frustration, not in the public eye.
"And that's the professionalism you've got to show, however frustrated you are, however disappointed you are.
"And there's nobody more disappointed than I am at the minute with the results.
"I get to carry the burden: they get the opportunity to put it right.
"I have to sit on the line, and I try to guide them the way to put it right.
"And the only way to do that, whichever player, is to play their best.
"Their responsibility is to play their best every week, and if I consider that the player's played his best, irrespective of result, I can safely say to them 'you've given your all, I couldn't have asked for much more'."