Antonio Di Natale broke his almost three-month scoring drought to put the Zebrette ahead with an early penalty, but Antonio Candreva levelled with a spot-kick of his own just after the hour-mark.
Lazio, who had been reduced to 10 men before that equaliser when Ogenyi Onazi picked up two quick yellow cards, then fell behind again when Emmanuel Badu scored midway through the second half.
However, an own goal from Andrea Lazzari and a last-gasp Hernanes effort made it seven points from three games for Reja's Biancoceleste.
It was a meeting of two clubs struggling to do justice to their expectations so far this season, Udinese on the brink of the bottom three and Lazio still well off the pace for a place in Europe next season.
The hosts looked the keener to give their campaign a boost with Roberto Pereyra twice testing Etrit Berisha in the opening five minutes before Luis Cavanda did them a favour by bringing Maicosuel down inside the penalty area.
Di Natale converted the penalty for his first goal since the end of October, and only the fifth of the season for a player who has exceeded 20 goals in each of his last four campaigns.
Sergio Floccari missed two good chances to level, but Udinese should have gone in at the break with a two-goal advantage when Maicosuel was faced with only Berisha to beat, but sent his shot wide.
Badu wasted another great chance early in the second half and Udinese's failure to put the game under lock and key came back to haunt them, even if they played the last 35 minutes with an extra man after Onazi collected two yellow cards in five minutes.
Lazio levelled with a penalty of their own in the 62nd minute with Miroslav Klose brought down in the area by Maurizio Domizzi and Candreva sending the ball into the top corner from the spot-kick.
Udinese responded well though and, within five minutes, they took the lead again when Badu took the ball down with his chest and sent an unstoppable volley past Berisha.
Klose and Hernanes wasted chances which seemed easier to take and the writing appeared to be on the wall for the 10-man visitors, but they did not give up and, eight minutes from time, they got their reward with the equaliser.
Senad Lulic's cross was pushed out by Zeljko Brkic onto the leg of Lazzari with the ball cannoning in to the unguarded goal from near the penalty spot.
There was a late twist too with Hernanes stealing all three points for Lazio with a precise shot into the bottom corner, sealing the second win in three Serie A games for Reja to lift them temporarily to seventh.