Rafael Benitez's side were made to work hard for the points against a resilient Nerazzurri side, whose hopes of holding out evaporated once Luca Cigarini was sent off soon after Higuain had sent the home fans into delirium.
Jose Callejon wrapped up victory late on to extend Napoli's 100 per cent start to the season.
Marek Hamsik, who said he needed a rest after international duty, was left on the bench by Benitez - with the Spaniard keeping one eye on a Champions League opener against Borussia Dortmund on Wednesday.
Goran Pandev took his place and the Macedonian had the first chance of the game in the 26th minute with a left-foot shot that sailed just wide of the post.
Apart from that effort, Napoli struggled to find gaps in an Atalanta side who had arrived in Campania with the clear intention to frustrate their hosts.
Benitez did not make any changes in personnel at the break, but he must have found the right words to spark Higuain into action, the Argentinian going close in the 49th minute with a shot just over the bar.
He broke the deadlock midway through the second half, turning to fire into the bottom corner and beyond the reach of Andrea Consigli, who was let down by his defence for the first time all night.
Atalanta were then let down by Cigarini, who earned a second yellow card for a foul on Callejon, leaving them with 10 men for the final half an hour.
That was enough to end Atalanta's hopes of finding an equaliser and, shortly after Higuain wasted a chance to finish them off, Callejon obliged with a low shot in off the left-hand post from Blerim Dzemaili's backheel.