The Serbian cruised through the first set 6-1 but was unable to take any of his three break points in the second before eventually losing it on a tie-break.
He was back to his clinical best in the third, though, breaking twice to seal a 6-1 6-7 (3/7) 6-2 win in just under two hours.
Seventh seed Tommy Haas will play no further part in the tournament, beaten 6-4 4-6 6-2 by Russian Mikhail Youzhny.
Third seed Jo-Wilfried Tsonga was initially in difficulties against Denis Istomin, dropping the first set but recovering to win 4-6 6-1 7-6 (7/3).
Nikolay Davydenko won an all-Russian clash against Alex Bogomolov Jr 7-5 6-1, Jurgen Melzer saw off Pablo Andujar 6-1 4-6 7-6 (7/3) and Frenchman Julien Benneteau retired when trailing 6-1 2-0 to Italy's Andreas Seppi.
In the women's second round, German Julia Goerges claimed the scalp of eighth seed Sam Stosur after a tough three-set battle.
The 23-year-old, ranked 21st in the world, took the first set on a tie-break but had to withstand a fightback from the Australian before eventually prevailing 7-6 (7/2) 4-6 7-5.
Tenth seed Caroline Wozniacki had to battle from a set and two breaks down to beat Su-Wei Hsieh 6-7 (5/7) 7-6 (7/3) 6-0.
Third and fifth seeds Agnieszka Radwanska and Angelique Kerber were not troubled as they reached the last 16.
Radwanska, from Poland, beat home wild card Zhang Shuai 6-2 6-3 and Kerber made even shorter work of Lara Arruabarrena-Veci, winning 6-2 6-0 in just over an hour.
Seventh seed Li Na won almost as emphatically, 6-1 6-2 against Nadia Petrova, while ninth seed Marion Bartoli won 6-4 6-3 against Japan's Ayumi Morita.
Former world number one Jelena Jankovic, outside the seedings for this tournament, hammered Monica Niculescu 6-2 6-1.
Great Britain's Laura Robson lost 7-5 6-3 to fellow qualifier Lourdes Dominguez Lino, while Romina Oprandi beat Camila Giorgi 7-5 6-2.
World number one Victoria Azarenka had no problems in seeing off Sabine Lisicki 6-4 6-2.
There was a major casualty in the men's draw, however, when second seed David Ferrer retired against Lu Yen-hsun.
The Taiwanese player was preparing to serve for the first set at 5-4 before Ferrer pulled out.
Fifth seed Richard Gasquet eased past Matthew Ebden 6-2 6-4, while Feliciano Lopez was a 7-6 (7/3) 5-7 6-2 victor over Ryan Harrison.