The Germans were stunned by first-half goals from Steffen Hofmann and Nikica Jelavic before substitute Christopher Drazan sealed the result in the 76th minute.
It is a great result for the Austrians and gives them hope of qualifying from a tough Group C which includes Hapoel Tel-Aviv of Israel and Scottish giants Celtic.
Rapid are playing their group stage games at the 53,000-seater Ernst Happel Stadium due to a huge demand for tickets and with the vociferous backing of their fans they almost got off to the perfect start.
With less than three minutes on the clock, Yasin Pehlivan's effort from just outside the area flew narrowly wide.
Branko Boskovic went even closer in the ninth minute but Frank Rost did well to save his 25-yard screamer.
And the Hamburg goalkeeper was on hand again moments later to deny Nikica Jelavic.
The visitors had to wait until the 25th minute for their first sight on goal and Helge Payer comfortably dealt with Marcus Berg's weak effort.
Berg unleashed a better effort a minute later but sent it inches wide of the left upright.
Rapid took the lead 10 minutes from half-time when Piotr Trochowski fouled Hofmann in a dangerous position just outside the area and the midfielder curled his free-kick into the bottom corner.
They doubled their lead soon after when Veli Kavlak picked out Jelavic who slotted home from close range for his sixth European goal in as many matches.
Hofmann could have extended his side's advantage five minutes after the break but sent his effort straight at Rost.
Hamburg slowly started to come alive and should have reduced the deficit shortly after the hour mark as first Mladen Petric headed over the bar from 12 yards before, seconds later, Eljero Elia blazed over.
The visitors were playing better and both Jose Ze Roberto and Jonathan Pitroipa went close only to be denied by Payer.
They were to rue their missed opportunities as hope of getting anything from the game all but evaporated in the 76th minute when Drazan slotted home from 12 yards.