The former Newcastle striker set two goals up for Kevin Nolan in the 2-0 Premier League win before being shown a red card by referee Howard Webb as he appeared to brush Flores with his elbow midway through the second half.
Allardyce was unconvinced Webb had had an unobstructed view of the incident, and bemoaned what he saw as play-acting from Swansea's Spanish centre-back.
He told BT Sport: "I've just had a look at it and from our point of view I can't see why Howard has decided to give him a red card.
"His arm's gone over the top of his head and while his arm's swinging round, coming from a position where both players have tangled and are coming round to correct their balance, there were players running round to obscure his (Webb's) view.
"So he can't be absolutely certain of what's happened, except that Chico Flores is over-exaggerating, which is generally the norm for him unfortunately.
"To clutch his face and roll around, I think that's had a big influence on Howard's decision, thinking that it's caught him in the face when it hasn't at all."
Allardyce then recalled the incident which so greatly amused him during West Ham's October trip to Swansea.
Flores had gone to ground clutching his face and moaning following a challenge from Carlton Cole in the goalless draw, and Allardyce was later seen laughing in the Spaniard's face because of his "squealing".
"That was him squealing and centre-halves aren't supposed to squeal are they?" Allardyce added.
Referring back to Saturday's game, he said: "If we're going to red card people for that then we might as well stop playing and I might as well retire.
"The game's all about physical contact - there's two challenges gone in and there's nothing happened of any major incident.
"Nobody's been hurt."
The Hammers boss felt Carroll's dismissal had somewhat sullied the fact his side had managed to clinch only their third home league win of the season, a result which lifted them out of the relegation places.
He added: "It's spoiled the fact that we've done brilliantly today, we've contained Swansea and found good goals from two good finishes.
"My players were brilliant today. It's a fantastic result and under the circumstances, where we had 10 men for the last 35 minutes, Swansea did not have a shot on goal that was on target."