The game looked set to end goalless when Gayle picked up the ball wide on the left, dribbled past Fabian Delph and curled a shot into the top corner.
The travelling Palace fans erupted in celebration of their second away win while boos rang out around Villa Park as the hosts slipped to a fourth straight defeat.
Both teams had created chances in the second half, with former Villa midfielder Barry Bannan seeing his shot pushed onto a post by Brad Guzan.
Aston Villa boss Paul Lambert was still without injured pair Christian Benteke and Ron Vlaar but Gabriel Agbonlahor returned from suspension to captain the side.
Palace were without suspended top scorer Marouane Chamakh so Yannick Bolasie came into Tony Pulis' team.
Andreas Weimann and Libor Kozak both kept their places in an attacking Villa line-up, but the first half was a story of frustration for the hosts.
They had plenty of possession but could not turn that into attempts on Julian Speroni's goal, with the Palace keeper more than equal to a couple of dangerous crosses.
It was not until the very end of the half that Villa really threatened to break the deadlock.
Karim El Ahmadi's volley was deflected on its way over the bar and from the corner Kozak's header looked destined for the top corner until Adrian Mariappa got his head in the way.
Palace were content to soak up pressure a lot of the time but when they did break forward they caused Villa problems.
Headers from Bolasie and Jason Puncheon were held by Guzan, while on more than one occasion a lack of precision meant the visitors were unable to capitalise on promising positions.
The tempo increased at the start of the second half, and it was Palace who probably should have taken the lead in the 48th minute.
The lively Bolasie raced down the right wing and pulled the ball back for Puncheon, who was in lots of space on the edge of the box, but the forward got the ball stuck under his feet and could only poke a shot towards goal that was beaten away by Guzan.
Villa responded and in the 54th minute a spell of pressure ended with Delph picking out Weimann with an excellent ball, but the Austrian chose to shoot first time and it was straight at Speroni.
Palace broke straight down the other end and Puncheon picked out Bannan just inside the box.
The Scot turned and struck a shot towards the bottom corner that was superbly pushed onto a post by Guzan.
Villa were having the majority of the play, though, and, after an Agbonlahor run ended with the striker shooting straight at Speroni, Weimann shot over when well placed and smacked the ground in frustration.
Bolasie was a real dangerman for Palace and in the 64th minute he got to the byline before pulling the ball back across the box for Puncheon, who was again denied by Guzan.
That was the forward's last action as he was replaced by Jonathan Williams, with Pulis also sending on record signing Gayle for Cameron Jerome.
Villa switched to a back three as Leandro Bacuna, Jordan Bowery and Alexander Tonev entered the fray, and it was Bowery who looked to have come closest in the closing moments with an 89th-minute header than was tipped over by Speroni before Gayle's late, great intervention.