The central defender prodded home from close range seven minutes after the break to ensure that the Gunners could not be overhauled by north London rivals Tottenham in the fight for fourth place in the Barclays Premier League.
But the result was in doubt until the final kick of the game, with Tottenham's home win over Sunderland meaning a late equaliser for Alan Pardew's men would have meant Spurs and not the Gunners getting a shot at Europe's premier club competition.
Newcastle had motivation of their own, after owner Mike Ashley's offer of £1million to be shared between the club's non-footballing staff if they ended a difficult campaign with a victory.
But ultimately, a third successive defeat at St James' Park, albeit despite a much-improved display, saw the staff go empty-handed and 38-year-old goalkeeper Steve Harper denied the perfect finish to his long career on Tyneside in front of a crowd of 52,354.
The Gunners enjoyed plenty of possession with Santi Cazorla the fulcrum, but with Newcastle defending competently, they found chances few and far between.
Theo Walcott, who was largely effectively contained by makeshift full-back Mapou Yanga-Mbiwa before the break, whistled a sixth-minute shot over Harper's crossbar and Koscielny glanced a header harmlessly wide from Cazorla's 12th-minute corner.
But in the meantime, Magpies striker Papiss Cisse had gone significantly closer, whipping a first-time left-foot effort just over after a fine run and cross by Yanga-Mbiwa.
For all Arsenal were able to retain possession for long periods, they were unable to penetrate with Cheick Tiote harrying in the middle of the park and Yoan Gouffran closing down space further upfield.
Gouffran dragged a left-foot shot across the face of goal with 13 minutes gone and Jonas Gutierrez saw a shot from distance deflected safely into Wojciech Szczesny's hands four minutes later.
There was little to choose between the sides, but the Gunners had a glorious opportunity to take the lead with 14 minutes of the half remaining.
Aaron Ramsey picked out full-back Kieran Gibbs in acres of space on the left and he cut inside Mathieu Debuchy before squaring to Cazorla, who steered his shot high over.
Yohan Cabaye tested Szczesny from long range with a dipping effort seven minutes later, but Hatem Ben Arfa, who produced several spellbinding solo runs during the first half, could not hit the target after cutting inside five minutes before the break.
Harper, who was warmly applauded by the home fans as the clock ticked into the 37th minute - his shirt number - was called upon for the first time within 60 seconds of the restart, and he will have been mightily relieved to find himself equal to the task after handing the Gunners their chance himself.
The goalkeeper's scuffed challenge fell to Ramsey, who slid the ball into the path of Walcott, but the England international could not beat Harper as he dived low to his right.
However, Arsenal had emerged in far more determined mood and they went ahead with 52 minutes played when Per Mertesacker headed down Walcott's free-kick for central defensive partner Koscielny to volley past the helpless Harper from close range.
Vurnon Anita, on for the injured Cabaye, sparked a promising 58th-minute attack for the Magpies which saw Jonas Gutierrez set up Cisse, but his shot was blocked at source by Koscielny.
It took Mertesacker's intervention to deny Gouffran a 64th-minute equaliser after Anita had picked out Mathieu Debuchy on the right, and the Frenchman was soon replaced by compatriot Sylvain Marveaux as Pardew looked for a way back into the game.
Walcott had a glorious opportunity to make sure at the death when he carved his way through the home defence, but with just Harper to beat, he stabbed the ball against the foot of the far post, but ultimately Koscielny's effort proved enough.