The Shakhter Karagandy striker's goal, which arrived in the 92nd minute, gave the minnows an unexpected win on the road and allowed them to move level on points with their hosts.
The Czechs started the night third in Group B, a point behind Bulgaria, and with the second-placed side facing leaders Italy, the chance to steal a march was there.
They started well and dominated early on, but fell behind to Karlen Mkrtchyan's stunner. They would level through Tomas Rosicky, but Ghazaryan had the last word to secure a famous Armenian win which moves them on to nine points also.
Their success went against all the pre-match predictions and as if to make it all the sweeter, they had been forced to ride an early wave of pressure.
Just seventy-four seconds had passed when Petr Jiracek had the first shot of the game, working Roman Berezovsky, before David Limbersky hit wide.
Jiracek then rattled in another shot which went away from goal and both Josef Husbauer and Michal Rabusic chimed in with wayward efforts which at least kept the pressure on.
The weight of traffic heading Armenia's way was overpowering but the home side could not cash it in, with Rabusic hitting wide after Berezovsky could only parry Plasil's shot.
Armenia were fortunate to be level but, somehow, they took the lead with 31 minutes gone, punishing the Czechs for their generous finishing.
Anzhi Makhachkala striker Mkrtchyan was the man on the spot, cracking in a brilliant 30-yarder to put his side in front with a Petr Cech helpless.
Few could believe what had happened and Armenia looked to make the most of their moment, with Levon Hayrapetyan having a follow-up effort charged down.
The Czechs were clearly stunned and Rabusic's header over was the closest they went to a response before the break, as Armenia impressed, with Henrikh Mkhitaryan seeing a shot charged down.
Rabusic went close with another effort after the interval but was lacking in support and the chances became more evenly spread, with Ghazaryan hitting wide at the other end.
The Czechs would get the goal their early if not later play deserved, however, with old-stager Rosicky pulling them level from the edge of the area with 20 minutes left.
Arsenal man Rosicky and then Husbauer tried to complete a total turnaround, but it was not to be and Ghazaryan had the last say from 12 yards.