Bojan Krkic opened his account for the Giallorossi in the 20th minute before Pablo Osvaldo added a second after half an hour.
German Denis opened the game up with a goal in the 48th minute, but Fabio Simplicio settled Roma's nerves with the winner nine minutes from the end.
The win lifts the Giallorossi temporarily into a share of the Serie A lead ahead of the rest of the league's fixtures this weekend.
Roma looked much more confident after last weekend's win over Parma and Francesco Totti sent in the first shot in the 18th minute with Andrea Consigli choosing the safest option of pushing the ball behind for a corner.
Roma took the lead in the 20th minute when Daniele De Rossi lifted the ball over the Atalanta defence into the path of Krkic, who needed a few touches before sweeping a left-footed shot across Consigli into the far corner.
Another ball over the defence, this time from Aleandro Rosi, set up Osvaldo for Roma's second in the 31st minute.
The Roma striker waited for Consigli to commit himself and got a bit of a lucky bounce off the 'keeper before slotting the ball into the open goal.
Totti almost made it three direct from a corner on the stroke of half time, but the ball rebounded off the near post, through the legs of Consigli and along the goal line before missing the far post.
Atalanta tried to recover and Denis headed them back into the contest in the 48th minute.
Roma suddenly retreated and focused on defending their lead as Atalanta grew in confidence.
But Simplicio ended the visitors' hopes of grabbing an equaliser in the 81st minute when he latched onto Miralem Pjanic's pass and lifted the ball past Consigli to seal the win.