After defeats in their first two Group H fixtures to AC Milan and Barcelona, the Scottish champions grabbed their first three points to leapfrog Frank De Boer's side and take them off the bottom of the table.
A James Forrest penalty two minutes before the break and a deflected Beram Kayal drive early in the second half put the home side into a comfortable lead.
In the 88th minute Hoops substitute Nir Biton, who replaced Mikael Lustig, was shown a straight red card by referee Ivan Bebek for his challenge on Thulani Serero before Ajax substitute Lasse Schone grabbed an injury-time consolation.
With AC Milan drawing 1-1 at home to Barca, Celtic are within two points of the second-placed Italian club after with Ajax at the bottom of the table with one point ahead of the Hoops' trip to Holland in a fortnight.
The former Celtic skipper, who revealed right-back Mikael Lustig had a recurrence of a hip complaint said: "I feel we can get results away from home but we are going to have to do very well.
"I knew the group was going to be difficult but I was happy with the performances in the first two games.
"It was encouraging going into the double-header against Ajax.
"We were in every game and that is no mean feat with the calibre of clubs we have been against us.
"If we had lost the game last night, with no points after three games, we would have been out, we would have been chasing our tails but we have given ourselves a great chance now.
"A point would not have been the end of the world but we got the win and it is wide open again."
De Boer insisted his side deserved more from the game but remains confident that they can win the return game.
The former Rangers player said: "Celtic didn't create anything although they scored twice, once with a penalty.
"We dominated the game but only scored once and should have equalised before they got their second.
"It is going to be very hard for us to get second place but I have confidence in the third place.
"We have one point and if we win at home we will be one point ahead of Celtic.
"It will be difficult but if we play that way again...eight times out of 10 the best team wins and I thought we were the best team.
"But only goals count and we will have to do it in the Arena."