Kolb was indifferent for much of the game but produced two touchdown passes in the final quarter, including a 15-yarder to Andre Roberts with 22 minutes to play to force OT.
Jay Feely then won it with a 46-yard field goal as the surprising Cardinals moved to 4-0.
Peyton Manning produced a vintage performance to lead the Denver Broncos to a 37-6 win over the Oakland Raiders.
Manning finished with 338 yards and three touchdown passes to help the Broncos improve to 2-2.
Aaron Rodgers capped a strong performance with the match-winning touchdown pass in the fourth quarter as the Green Bay Packers beat the New Orleans Saints 28-27.
Rodgers threw for 319 yards, including the key play to Jordy Nelson in the fourth, one of four TD passes he produced.
Billy Cundiff kicked the winning field goal with just three seconds left to play as the Washington Redskins beat the Tampa Bay Rays 24-22.
Andy Dalton helped the Cincinnati Bengals overcome their injury problems and the Jacksonville Jaguars, throwing two touchdown passes and running for a score in their 27-10 win.
In the day's final game, the Philadelphia Eagles beat the New York Giants 19-17.