The Scots' dismal 2-1 defeat by Wales at Hampden Park on Friday night left them at the bottom of Group A with two points from five games and ended any hopes, faint as they were, of reaching Brazil next summer.
While even the most optimistic member of the Tartan Army could be forgiven for looking at the section and wondering where their first win will come from, the Wigan defender believes there is enough within the Scottish squad which landed in Belgrade on Monday to make it happen against the Serbs.
"I think there is enough confidence and ability," Caldwell said. "It is just a case of going out and showing that on the night and getting it right over 90 minutes, getting a win under our belt and then trying to build on it.
"Friday was a massive disappointment for the group and for the country. But in football you have to keep going. There is no point dwelling on the past.
"We have to recover together and stick together.
"It is difficult, we want to win games and every time you lose a game a bit of confidence is taken from the group but we dust ourselves down.
"We have recovered from the Wales match and we look forward to tomorrow [Tuesday] night with great confidence again.
"As players we put expectation on ourselves to perform and be the best we can be, so every game we go into we have expectations on our shoulders and we try to live up to that.
"We look at Tuesday night as an opportunity to win a game and that is what we will be trying to do.
"So far we haven't but we will go out and give it everything we have got."
Caldwell is hoping Scotland can tap into the disappointment which enveloped Serbian football after their 2-0 defeat to neighbours and bitter rivals Croatia on Friday.
The former Celtic defender said: "Their result would have been a huge blow to them.
"It is probably the equivalent of a Scotland versus England game and if we had lost that it would have been a big blow to us, so we have to try to use that to our advantage.
"We have to start the game a lot better than we did on Friday and get the crowd on their backs.
"It is up to us to keep the ball and frustrate them early on and try to turn the fans against their own team
"We know that, but we have to go out there and do it."