Rangnick left the club for health reasons on Thursday with Seppo Eichkorn taking temporary charge of the team for Saturday's Bundesliga match.
And Holtby wished Rangnick a speedy recovery while saying his shock departure had inspired the team to pick up three points for him.
"It is good that Ralf Rangnick can now relax and find inner peace," he told Sky television.
"That is the most important thing at the moment. The team showed a reaction for him today."
Eichkorn admitted he had not spoken to Rangnick since he left the club, preferring instead to "leave him in peace", but he did use the coach's bravery in confronting his illness - an exhaustion syndrome - to motivate his team.
"On Friday, I touched on his strength a little to make the team understand," he said.
"Today, we just tried to prepare for the game in a normal way. We tried to bring normality into the situation, but that was very tough on Thursday."
Eichkorn's work received praise from the club's director of sport Horst Heldt, who admitted this win and the way the caretaker coach had mastered the situation helps him in his search for a new coach.
"Our current coaching team has certainly done well in what was not a simple situation," he said.
"Therefore, we don't have to make a panic appointment now.
"We have the time we need to search for a new coach. This victory showed that we do not lack leadership."