The Black Cats missed out on a place in the quarter-finals of the cup competition after they were knocked out by npower Championship side Middlesbrough in a hard-fought Wear-Tees derby that ended 1-0 at the Stadium of Light.
Scott McDonald's 39th-minute close-range strike was enough for Boro to progress as Sunderland lacked the cutting edge that has blighted them all season. They have drawn six of their eight Barclays Premier League games so far this season.
The loss also ends the Black Cats' dominance in the fixture as, until the game, they were undefeated against Boro in six games stretching back to January 2006.
"It was a massive opportunity for us and we've let ourselves and the fans down with the performance," Colback told safc.com.
"We have to deal with the pressure as players. The fans had a right to get on our backs tonight because it wasn't good enough.
"There were a few times where we didn't play well last year and we always came out fighting in the next game.
"We've got to handle that, come out on Saturday and make sure we get the three points."
Colback believes Sunderland came undone after they were unable to put an early score on the board and take control against their lower league opposition.
Scoring goals has been the Achilles heel for Martin O'Neill's side this season, with Sunderland only finding the net 10 times in 11 games in all competitions so far.
O'Neill showed his attacking intentions against Boro by partnering form striker Steven Fletcher up front with Louis Saha, but his attacking line was still unable to unlock Middlesbrough's defence.
"Going into the game we were looking to start well and get through but it didn't happen on the night," Colback said.
"We knew it would be tough. They have been going well and it was a derby game.
"We let Boro get into the game and get the goal and that goal hit us harder than it should have done."
For their part, Middlesbrough headed home with nothing to worry about.
They are riding high in the Championship and head to Charlton at the weekend looking to improve on their current position of third.
Added to that, they are one of three non-Premier League teams left standing in the cup, but manager Tony Mowbray is cautious of getting too carried away.
"We're delighted to be through, cup competitions are all about getting through to next round," he said.
"But I'm a realist. We could go to Charlton and lose 3-0 and everyone will say the wheels have come off. I know there will be setbacks and when they come, the team have to remain strong, be mentally tough and keep going and believing in themselves.
"The Championship is a difficult league, as you'll see on Saturday when we'll have a very tight game at Charlton.
"We're winning those tight games at the moment. But lose a couple of games and you're mid-table. Every point is hard-fought."