The Waratahs' shock loss to the Western Force last week ensured their worst start to a Super Rugby season since 1998, placing immense pressure on Michael Foley's men to perform in front of the Sydney faithful.
But they were made to fight every inch of the way by a Sharks side that was intent on playing an enterprising brand of rugby.
Two first half tries helped the visitors to a 17-13 lead at the break and they went on to enjoy a handy 24-13 lead after 10 minutes of the second stanza.
NSW, who were equally as entertaining, refused to give in though and fought their way back into the contest through tries from full-back Bernard Foley in the 53rd minute and centre Tom Carter in the 57th.
But a Patrick Lambie penalty goal with eight minutes to go put the visitors back in front at 30-27, before young NSW winger Tom Kingston crossed for the match-winner with just five minutes remaining.
The dry and sunny conditions were perfect for some quality running rugby and the Sharks were the first side to take full advantage when a spread to the left resulted in dashing winger Lwaza Mvovo looming up inside centre Meyer Bosman to score under the posts in just the third minute.
Lambie converted and the team that had previously enjoyed just one win from nine Super Rugby appearances in Sydney had an early 7-0 lead.
Brendan McKibbin kicked a penalty goal to get the Waratahs on the board after 12 minutes, but Lambie responded with a 45m three-pointer four minutes later to push the Sharks out to a 10-3 lead.
However, McKibbin - a replacement in the starting side for South African Sarel Pretorius - took full advantage of some well constructed Waratahs phases to dive over the ruck from close range for his side's only try of the opening 40 minutes.
The Durban-based unit quickly regained the ascendancy though, taking possession back from the re-start and producing some fine driving forward play for skipper Keegan Daniel to crash his way over in the 25th minute, opening up a 17-10 lead for the visitors.
McKibbin added another penalty goal for the hosts soon after as they went in trailing 17-13 at half-time, having seen winger Adam Ashley-Copper blow a great opportunity to snatch the lead as he spilt a clever one-arm Tatafu Polota-Nau pass with the line wide open eight minutes before the break.
The Sharks refused a couple of opportunities for penalty goals in the pursuit of maximum points and their gamble paid handsome dividends in the 49th minute when giant hooker Bismarck du Plessis pealed off a rolling maul for a strong try.
But with so much at stake, the Waratahs could not let this one get away from them and when a rampaging Wycliff Palu charge provided space for Foley to cross out wide in the 53rd minute, they were more than back in the contest.
The home side finally took the lead four minutes later when the hard-running Carter crashed over under the posts, with McKibbin's simple conversion edging the 'Tahs out to a 27-24 advantage against seemingly tiring opposition.
Lambie's second penalty goal in the 63rd minute put the Sharks back on level terms before Rob Horne failed to release, allowing the young number 10 to add a further three points and give the visitors a 30-27 lead with just eight minutes remaining.
But there was a match-winner left in the 'Tahs and it came about after some excellent offloading from the Timani brothers, who provided Polota-Nau some space to fire a ball out to Kingston who crashed over in the right corner.
Berrick Barnes sent the crowd into raptures with a sideline conversion to confirm the heart-stopping bonus point win.