His crucial strike happened just minutes after Diego Maradona's men looked to have suffered yet another World Cup qualifying setback.
Palermo, whom Maradona has recalled to the national team after a 10-year absence, tapped home at the far post in driving rain to send his coach diving across the sodden turf on his stomach in celebration.
Hernan Rengifo had headed an equaliser for Peru in the 90th minute to cancel out debutant Gonzalo Higuain's opener early in the second half.
The win moved Argentina up to fourth place in the South American qualifying table, a point ahead of Uruguay, whom they play on Wednesday, and two clear of Ecuador.
Maradona sprung a surprise with his team selection, leaving big-name strikers Carlos Tevez and Sergio Aguero on the bench and instead handing Higuain his first cap.
Argentina went into the match having lost their last three qualifiers to Paraguay, Brazil and Ecuador, but if ever there was a chance to restore some confidence in the side this was it - a home match against the team bottom of the group.
And they did everything but score in a first half they dominated from start to finish.
Gabriel Heinze stole in at the back post in the sixth minute to head a deep cross from the right low across goal and wide when he should have hit the target, but the assistant's raised flag got him off the hook.
A minute later Angel Di Maria was sent clear down the left but, despite being in acres of space and under no pressure, he could only stab his weak cross far too close to the keeper.
Higuain missed the best chance of the half when he was picked out by Di Maria's low ball in and contrived to shoot over from six yards with the goal gaping.
The Real Madrid man saw another far more difficult chance saved by the legs of Peru keeper Leao Butron as the home side continued to see opportunities come and go.
Lionel Messi was starting to pull the strings in the final third, one glorious drop of the shoulder and drag back leaving Rainer Torres on the seat of his shorts. On the half-hour mark the Barcelona man sent in a dinked left-footed cross from the right from which Pablo Aimar looked to score with an audacious backheel, but he did not get a strong enough contact and the ball went wide.
Benfica's Aimar then sent a low cross along the six-yard line which evaded everyone before moments later Messi picked up the ball just inside the area on the right and blasted a left-footed shot which swerved wide of the far post.
Di Maria's cross from the left was then spilled by Butron but Peru survived and went into the break all square as their goal continued to live a charmed life.
Maradona turned to imposing striker Martin Palermo rather than Tevez or Aguero at half-time, but his side almost found themselves behind scarcely a minute into the second period.
Nolberto Solano's corner picked out Juan Vargas who crashed a first-time left-foot volley off the top of the crossbar from well outside the area.
It was Argentina, though, who took the lead in the 48th minute, Aimar stabbing a pass through into the right of the area, which Higuain ran on to and slotted into the far corner.
Peru had a penalty shout waved away in the 58th minute when Solano's shot appeared to be blocked by the arm of Emiliano Insua.
And Argentina looked to grow more nervous as the second half wore on, their chances drying up as torrential rain started to pour down.
The home side seemed prepared to hold out for the final whistle and were made to pay for not going for the jugular when they conceded in the final minute.
Moments after keeper Sergio Romero had made a superb point-blank save, Rengifo nodded home a cross from the left from barely two yards bring Peru level.
But just as Argentina's World Cup dreams looked to be slipping away, Palermo came up trumps for his under-fire boss.
With no fewer than eight Argentinian players in a packed Peru penalty area, the Boca Juniors striker found himself in the right place at the right time to convert the easiest of chances after a low ball across the box came all the way through to him.
There was more drama to come as Rainer Torres tried his luck straight from the kick-off, but his effort struck the bar and Argentina and Maradona claimed a vital win.