Victor Moses levelled in the 68th minute to cancel out Wesley Hoolahan's early opener as the visitors dominated the second half.
"It was a sloppy start, we have to look at ourselves for that," Caldwell said on Sky Sports 1.
"We let Norwich into the game and gave them a goal start. We came out for the second half and had a right go at them but couldn't get that second goal.
"We know on our day when we play well and work together as a team we're not bad and we showed that today [Sunday].
"We've still got great belief, we showed that today [Sunday]. We're together as a unit, that can only be a good thing."
Moses felt his side deserved the victory for the second-half display.
"I thought we deserved the three points," he said. "We had a lot of chances. In the first half they had the chances, in the second we were all over them."
Wigan boss Roberto Martinez took heart from the way his side got themselves back in the game.
"It took us 50 minutes to get into the game," he said on Sky Sports 1. "We know that Norwich at home can play with real momentum and we got heavily punished in that period, but after that we grew into the game and in the end we created the best chances.
"You saw the reasons we don't win football games, because we didn't show the composure and the little bit of luck you need in front of goal."
Asked what his side now needed to stay up, he said: "What we need is to perform in the manner we did today [Sunday]. We want to win, but we need to know how. Forget about the table, we're going to take every game as a unique opportunity to show what we can do."
Norwich boss Paul Lambert admitted he was content to take the point.
"If you can't win the game, make sure you don't lose it and credit to the team, they kept going," he said on Sky Sports 1.
"It's another point. We had a terrific start and I thought some of the football we played was very good, but I will take a point."