The Wolves beat Hoffenheim 3-1 to move out of the bottom three and Allofs, who only arrived from Bundesliga rivals Werder Bremen on Thursday, is convinced Kostner is the right man for the job.
"We are not looking for a new coach," he said. "I have seen the training sessions for myself and there is no issue regarding the coach.
"My priority now is to observe a lot and hold a lot of talks before making decisions.
"We don't want any short-term solutions, but ones for the mid and the long-term."
Allofs appeared to put 13 years with Bremen behind him quickly as he celebrated his side's goals and a victory which took them above their opponents.
"When you win and lead 3-0 at one point, then it is somewhat more relaxing," he told Liga total television.
"We were all a bit tense on the bench."
Hoffenheim coach Markus Babbel was even more tense, particularly during the first 45 minutes, but an improvement in the second half warmed him.
"It was a really good second half and I can't criticise the team for that," he said.
"But we must not sleep through the first half - we've got to be better there.
"We made too many mistakes which led to us conceding, and you get punished. We don't have the right balance yet."