The Dragons had made a solid start to their opening Group A fixture against the heavily-fancied visitors at Cardiff City Stadium before the West Ham defender was shown a straight red card for a reckless lunge on Guillaume Gillet after 25 minutes.
The hosts hopes crumbled from that moment as Vincent Kompany's 42nd-minute header and Jan Vertonghen's ferocious free-kick seven minutes from time gave Belgium the three points.
And Coleman was highly critical of the sending off, and Gillet's behaviour following the challenge.
He said: "I didn't think it was a red card, maybe I am biased. I think the referee sent him off because he went on the reaction of the player who was on the floor, who incidentally went on to finish the rest of the 90 minutes.
"Correct me if I am wrong, football is a contact sport. It was not a malicious tackle, I have been in football long enough to know what a leg-breaking tackle is and that was not it, so it is not a red card in my book.
"They were counter-attacking us and we cannot let players run past us. He is a defender and I have been there, sometimes you can't help yourself and you want to go and win the ball. But I have seen much more dangerous tackles than that.
"I am not going to punish James, we can't let teams like Belgium come here and stand off them, you have to get close and make contact.
"That is not a red card, it is maybe a yellow card and a warning, but not a straight red."
Coleman was also less than impressed with the decision to penalise captain Aaron Ramsey for a foul on Romelu Lukaku, which allowed Vertonghen to fire home the clincher.
But he praised the performance of a side lacking injury victims Joe Allen, Craig Bellamy, Wayne Hennessey Neil Taylor, Andrew Crofts and Jack Collison.
"We were disappointed with some of the decisions against us," he said. "The free kick against us at the end that they scored from was not a free kick, watch it as many times as you like.
"I can understand referees get it wrong sometimes as things happen in the spur of the moment but not in situations like that.
"One player is running behind the other, Lukaku goes to shoot, he does not know Aaron is behind him, Aaron is not trying to make a challenge, but he gets his legs caught between Aaron's. It ends up as a free-kick and a goal.
"We were disappointed with a lot of decisions but I am not disappointed with the players.
"They were magnificent and showed a lot of heart but at the end of the day we were beaten by a good team."
Wales will now seek to regroup before Tuesday's visit to play Serbia in Novi Sad, with Coleman hopeful that Liverpool midfielder Allen may have overcome an illness.
Belgium will move on to meet Croatia in Brussels, and manager Marc Wilmots was pleased with his side's performance, even if they did not hit the heights suggested by the pre-match hype surrounding their coterie of Premier League stars.
Wilmots said: "I am very happy with the victory against a difficult opponent.
"We had massive support here tonight [Friday] and now we can go home and work on our game against Croatia.
"It was a difficult game, the sending off changed the context of the game and when Wales were playing with 10 men we had to use the space on the wings, which we did and all in all I am happy.
"Obviously there were some things I was not happy with also, but I am happy with the three points and starting our campaign with a win."