It was home defender Joleon Lescott who headed Roberto Mancini's men into the lead before the break, but France international Koscielny popped up at the other end eight minutes from time to grab a point with a powerful strike.
And Mertesacker was full of praise for Thomas Vermaelen's understudy, who was making his first league start of the season.
The German told Sky Sports 1: "We had to be strong today [Sunday], it was a very difficult match for us.
"We have a great squad this year and a lot of great centre-backs at the moment.
"Thomas is at home a little bit ill and Laurent came in and did fantastically. It was a solution.
"We are a strong squad and we have to stick together. We did well today [Sunday] and we have to keep on going."
City manager Roberto Mancini bemoaned his side's collective lapse in concentration in the closing stages.
"I don't know if we lost our concentration but it's strange," he said. "Eight, 10 minutes to go to finish the game, we should have had our concentration 100%.
"We had the game under control. [It was] only for the last 10, 15 minutes... I don't know why we don't continue to play. We were too deep. The problem is we need to close when we have a chance to score."
The draw moved the champions on to nine points from five games.
Asked to assess his side's start to their title defence, the Italian replied on Sky Sports 1: "It's difficult. I think we did not start with some players. We had some problems with this. I think we have two points less [than we should], probably."
Gunners boss Arsene Wenger was quick to echo Mertesacker's praise of goalscoring centre-back Koscielny but regretted not being able to take all three points away from the Etihad Stadium.
He said on Sky Sports 1: "I'm not worried about Koscielny - he is top class.
"We have three top-class central defenders.
"We were in danger a few times on the counter-attack but overall we were solid.
"We are a bit frustrated because we feel we had the chances early on and late in the game. But you can see that we are a team and we are a strong team.
"Manchester City score a lot from set pieces and they did it again today [Sunday]."
The Frenchman remains optimistic over Arsenal's chances of winning a first league title since 2004.
"It's down to consistency and what happened today should strengthen our belief," he added.
"But we are not the perfect team yet and there is still room for improvement.
"I believe we have a chance."