The Gialloblu were unbeaten in 17 games until losing to Juventus in midweek and their second straight defeat allowed Lazio to close to within two points of their current position of sixth in the Serie A table.
Jonathan Biabiany cancelled out Senad Lulic's opener before Michael Ciani's own-goal brought Parma level once again following Miroslav Klose's second-half effort, but Italy winger Candreva decided the outcome at the death.
Parma's defence wobbled early on - Felipe losing control of the ball for Stefano Mauri to shoot wide - and that lapse of concentration allowed Lazio to seize control and score after 15 minutes.
Lulic only needed one attempt to fire Abdoulay Konko's cross beyond Antonio Mirante.
Parma responded brightly and, after Biabiany shot over from close range and Federico Marchetti denied Ezequiel Schelotto, the Frenchman found a way through.
With 26 minutes on the clock Biabiany levelled the scores by heading home Antonio Cassano's rebounding effort.
Schelotto almost forced a quick turnaround when skipping past two opponents to feed Afriyie Acquah, only for the Ghanaian to come up short on the ball, and Marco Parolo's free-kick soon stung Marchetti's palms.
Candreva shaved paint off a post as Lazio pushed back before half-time and, one minute before the break, Klose drove a daisy-cutter across the goalmouth and just wide.
When play resumed, a choreographed free-kick between Cassano and Parolo saw the latter whip the ball round the left side of the wall, where Marchetti was able to punch clear.
In the 67th minute, Lazio snatched the lead once again.
Lulic slipped the ball into the area and Klose took two steps past Mirante before picking out the bottom corner.
As time began to tick away in Rome the chances began to dry up for Parma and Parolo's long-range blast past the near post looked like being the best opportunity the frustrated Gialloblu would get.
So they were grateful in the 81st minute when substitute Ciani put through his own net.
It was an unfortunate moment for the French defender as he sliced Raffaele Palladino's cross beyond the flailing Marchetti to give Parma a late lifeline.
Suddenly everything had changed at the Stadio Olimpico and Lazio were the side breathing a sigh of relief when Gianni Munari powered a header agonisingly wide from Christian Molinaro's early cross.
Six minutes of stoppage time meant a nervous conclusion to a match both teams desperately wanted to win and, three minutes in, Candreva ruined Parma's day.
Substitute Keita Balde Diao made a killer contribution by delivering a low assist to the Italy winger, who outwitted Mirante to slip home a dramatic winner.