The home side were struggling against Neil Lennon's men and in need of some inspiration as the game entered the final stages.
However, they got a huge slice of luck when defender Cristian Zapata's drive in the 82nd minute appeared to be heading wide before it deflected off Parkhead left-back Emilio Izaguirre and ended up behind Fraser Forster.
The Hoops goalkeeper was beaten again four minutes later despite making a great save from Mario Balotelli's free-kick, as midfielder Sulley Muntari knocked in the rebound.
Much was made of AC Milan head coach Massimiliano Allegri's lengthy injury-list which included the absence of Kaka and captain Riccardo Montolivo but the home side, and indeed they looked disjointed for most of the match.
Much of that, however, was down to the performance of Celtic.
Forster had to make several saves early on and one near the end of the first half, but in between, the visitors enjoyed more than their fair share of possession.
There was a confidence in the way the Scottish champions kept the ball albeit it without threatening Milan keeper Christian Abbiati too much.
After the break Muntari missed a great chance while Hoops' in-form striker Anthony Stokes hit the top of the bar with a free-kick.
The two late goals, though, were a sickener for Celtic who take on Barcelona in their next group fixture at Parkhead at the start of October with Ajax also waiting in the wings.
However, on their overall display, they can go forward in the group with some confidence.
Allegri complained on Tuesday of a long list absentees including Kaka and captain Riccardo Montolivo but he could still afford to leave Robinho on the bench as he chose former Manchester City forward Balotelli and Alessandro Matri in attack.
Midfielder Joe Ledley dropped out with a groin injury to be replaced by Charlie Mulgrew while Georgios Samaras came in for Derk Boerrigter, who started on the bench.
Milan, seven-times winners of European club football's top trophy, took control of the game after Kris Commons fired a free-kick from 25 yards high over the bar in the first minute.
Forster parried Balotelli's powerful volley clear four minutes later, with midfielder Antonio Nocerino driving the rebound wide.
Moments later, right-back Cristian Zaccardo tested the goalkeeper with a drive before Forster and defender Virgil van Dijk combined to block Matri.
Forster then had to concede a corner with another decent save from Matri's flashing header after Cristian Zaccardo was allowed to get in a cross from the right.
Just after the half-hour mark Nocerino sent a dipping volley just over the bar but five minutes from the interval a careless pass towards his own goal from wide on the right from Milan midfielder Valter Birsa went straight to Stokes, who turned Zapata inside the box but his shot took a deflection for a corner, which again the home side defended comfortably.
Seconds into added time in the first half, Forster had to scramble down low to his left to save a low drive from Balotelli which deflected off Hoops defender Efe Ambrose and which appeared to heading for the corner of the net.
The visitors started the second half brimming with belief and came close five minutes after the restart when Samaras drove just past the post from 25 yards.
The Milan fans were becoming increasingly frustrated as Celtic encroached into their box with menace although the visitors escaped again in the 62nd minute when Muntari missed a sitter, heading Zapata's cross over from three yards.
Celtic had their best chance in the 67th minute when Commons played in skipper Scott Brown with a clever pass but Dutch midfielder Nigel de Jong, who had earlier conceded possession, retrieved the situation with a great saving tackle and the Scotland midfielder's shot lacked the pace to trouble Abbiati.
In the 79th minute Celtic substitute Teemu Pukki, on for Commons, was brought down by De Jong 30 yards from goal but Stokes's free-kick clipped the bar on the way over before the late frustration which started when Zapata's drive from 30 yards took a touch off Izaguirre before nestling behind Forster.
More woe visited Celtic when Brown was booked for a foul on Balotelli, whose curling free-kick from 25 yards was brilliantly saved by Forster, only for Muntari to bundle the rebound in.