A 3-1 win over Hoffenheim on the final day of the season took Hertha above Cologne and into 16th place, earning them a relegation play-off against the side who finish third in the second division.
Rehhagel said: "All we have done is what we had to do.
"I have told my lads in the dressing room that they can enjoy this for one night, but then we have to prepare for the play-off on Thursday."
Just one goal for Hoffenheim late on would have ended Hertha's dreams as they held on a slender 2-1 lead, but the result was finally secured when Raffael raced the full length of the field to score a third.
"Can you imagine what would have happened if we had conceded?" Rehhagel added to ZDF television.
"Things were not perfect in the second half, but we fought like lions."
Hoffenheim coach Markus Babbel congratulated Rehhagel and wished his former club all the best.
"I am delighted for him that he has managed to get over this first hurdle and I am keeping my fingers crossed that they win these two play-off matches," he said.
"They are a fantastic club."