The hosts hit back from behind to claim victory and leave the Rossoneri – who had Philippe Mexes sent off in stoppage time for attacking Stefano Mauri – without a win in their last five Serie A games.
Jeremy Menez had given Milan a fourth-minute lead which lasted until two minutes after the break when Parolo netted his opener.
Five minutes later Miroslav Klose put Lazio in front and Parolo’s second nine minutes from time effectively sealed the win which sends the hosts up to fifth.
New signing Duje Cop grabbed a late winner as Cagliari moved out of the bottom three with a 2-1 win at home to Sassuolo.
Luca Rossettini gave the hosts the lead in the 20th minute but put through his own goal with 15 minutes remaining to give the visitors hope.
But Cop volleyed home three minutes later to lift Gianfranco Zola’s men – and the visitors ended the game with 10 men after Francesco Acerbi was sent off for a second yellow card.