Veteran centre-back Walter Samuel won the game for the Nerazzurri with a headed goal in the third minute, but that was far from the end of the drama on Sunday night.
Riccardo Montolivo was denied the chance level due to a controversial call from the official, but Milan only had themselves to blame for failing to capitalise when Yuto Nagatomo's dismissal early in the second half for a second booking put Inter on the back foot.
"The ref made some mistakes and I'm not saying that because we lost," Allegri said.
"A good ref like him should know how to take these decisions with the game running. I'm sorry for the players as we did well. The team conceded nothing apart from the goal."
Allegri felt his team were denied a decent penalty shout at the San Siro, but it was Montolivo's disallowed goal that was the main bone of contention post match.
Urby Emanuelson collided only softly with Inter's goalkeeper in the area while challenging for a lob forward, leaving Montolivo to toe-poke the loose ball into the back of the net, but Valeri had already blown his whistle for what he considered an infringement by the Dutchman.
"Emanuelson did not foul in the lead-up," Allegri said. "We could also have had a penalty and Juan Jesus could even have been sent off in the first half.
"The ref had a bad night and that can happen to anyone."
Montolivo was also unimpressed with the decision.
"I think it's been the best game of the season, we deserved to win," he said.
"There was only one team on the pitch, especially after Nagatomo got sent off.
"It's a shame about the disallowed goal. On the pitch, the impression I had was that it should have stood.
"It's disappointing because it was a great goal, from far out and a tough shot to make. It's tough because we deserved it."
The defeat was Milan's fourth in seven games at the start of the new season to leave the Rossoneri languishing in the bottom half of the table.
Allegri is coming under increasing scrutiny in the wake of the club's poor start to the season, but has ruled out wholesale changes in his approach. However, he concedes his team's weakness at defending set-pieces is something that needs to be addressed.
"The team played well and we can't always change the way we play otherwise it creates confusion," he said.
"We've conceded seven goals from dead balls. Something has to change.
"We're far from the top spots, but the race is long and this team will rise again."
While the problems are mounting in the red and black half of Milan, the blue and blacks are thriving on the back of a third straight league win - and a third back-to-back success in derby clashes.
Andrea Stramaccioni's Inter are in fourth place, four points behind leaders Juventus heading into the international break, and the coach hailed the resolve of his team for hanging on for the points after Nagatomo's dismissal.
"It's a great result of course, even more so when you consider how the game went," Stramaccioni told Radio Anch'io Lo Sport.
"Playing the entire second half with 10 men forced us to show the ability and mental strength we have here at Inter and that bodes well for the future because, although there's lots of work to be done, our character is a huge advantage.
"Was it a six-pointer? It would have been had we won normally, but I think it's worth three wins considering how we stood up with 10 men to AC Milan's attacks on our goal."