The Italian side were also awarded three penalties in breathless tie at the Stadio Olimpico.
Martin Stranzl's first-half spot-kick put the Bundesliga side ahead before second-half strikes from Sergio Floccari and Kozak turned the pendulum Lazio's way.
Stranzl then missed his second penalty, which also saw Lazio defender Andre Dias dismissed after picking up a second yellow card, before three goals in the final six minutes brought a pulsating match to an end.
Thorben Marx levelled for Monchengladbach from their third penalty of the night and Juan Arango's strike two minutes from time appeared to have given the Germans the edge, but Kozak's header in the fourth minute of stoppage time gave Lazio three away goals and a slight advantage going back to the Stadio Olimpico.
The hosts took the lead in the 17th minute when Dias fouled Patrick Herrmann in the area and Stranzl made no mistake with the resulting penalty, powering his shot into the top right-hand corner of the goal.
The Serie A team looked to respond, Stefan Radu's long-range free-kick tipped round the post by Marc ter Stegen.
However, it was Monchengladbach who came closest to breaching their opponents' defence again when Arango's 25-yard free-kick hit the post and Herrmann's follow-up shot was caught by Lazio goalkeeper Federico Marchetti.
Ter Stegen was again called on when he pushed Hernanes' 25-yard shot round the post and just moments before the break the same player's free-kick found Radu who headed over.
The second half continued in the same vein as the first with Lazio pressure paying off in the 57th minute.
Antonio Candreva's cross from the right found Floccari in the six-yard box and he powered the ball into the top corner.
And the Italians turned the game completely on its head when they took the lead seven minutes later, Senad Lulic's cross from the left met by Kozak who slotted a low shot home.
Just five minutes later the hosts were given a golden chance to equalise when Dias fouled Stranzl and was booked for a second time to reduce Lazio to 10 men.
Stranzl took his second spot-kick of the night, sending it towards the bottom left-hand corner, but it was superbly saved by Marchetti
Monchengladbach were not bowed and were inches from equalising when Stranzl's shot from the centre of the area hit the post and Hermann put the follow-up over.
However, with six minutes to go they did equalise - from their third penalty of the night.
Lorik Cana fouled Luuk De Jong and rather than Stranzl, Marx stepped up so slot home.
The game swung Monchengladbach's way again when Arango fired a 25-yard free-kick into the bottom corner with just two minutes left.
However, that was not the end of the scoring as Hernanes' ball from the right found Kozak and he headed high into the net to salvage a draw for the visitors.