The Gialloblu were good value for their half-time lead at the Stadio Ennio Tardini with both Parolo and Antonio Cassano cutting through a disorganised Milan team to score impressive individual goals.
But something clicked with the visitors over the break and, shortly after Alessandro Matri hit an overdue first goal for his new club, defender Matias Silvestre dragged Milan level with a header.
Massimiliano Allegri's men pushed hard for the winner but it would not come, Parolo instead grabbing the glory with a free-kick at the death.
Parma climbed above Milan and into seventh place, while a fourth league defeat of the season leaves the Rossoneri in 10th and 11 points short of the Champions League places.
Milan put the first attacking move together, Valter Birsa feeding Mario Balotelli in the area before Antonio Mirante gathered, but the first half would belong to Parma.
In the 11th minute Jonathan Biabiany dropped the ball into the Rossoneri box where Ignazio Abate cleared it to Parolo, who beat Gabriel with an angled drive.
Parma could not find a quickfire second but ran rings around their confused opponents while trying to do so.
Andrea Poli did miss a fine chance to level on 33 minutes, though, his snapshot at the end of a counter-attack clattering a post.
Balotelli soon went down looking for a penalty, but referee Paolo Valeri was distinctly unimpressed and booked the Milan striker.
He then came up short on a rebounded Robinho shot and Mirante blocked Birsa's through-ball to the Brazilian as the visitors rallied.
Cassano burst their bubble, however, scoring Parma's second one minute into first-half stoppage time after sweeping up Biabiany's pinpoint pass.
Allegri read his players the riot act over the break and his team-talk worked.
His men pushed and probed Parma for 15 minutes before Matri opened his account in Rossonero colours with a daisy-cutter that took off and beat Mirante to the far corner.
In the 63rd minute, Milan drew level.
Silvestre, like Matri a summer signing at San Siro, found himself well-placed to nod home Cristian Zapata's rebound, leaving the Parma defence gobsmacked.
Almost immediately Riccardo Montolivo missed the chance to make it 3-2 to the visitors, his sighter leaving Mirante cold only to touch a post and bounce wide.
Brazilian playmaker Kaka was by now on the pitch as a substitute and soon began causing trouble while Parma rallied to close ranks.
His shot from Birsa's cross ended up in the crowd and, after being served by the Slovenian again eight minutes from time, he fired straight down Mirante's throat.
As time ticked away, Milan began to come to terms with the fact they might leave Parma with only a single point.
It fell to Parolo to readjust their thinking.
Four minutes into stoppage time Zapata conceded a free-kick on the edge of the visitors' box and Parolo, with a little help from the slow-moving Gabriel, steered the ball over the wall and home.