Carragher, 35, on Thursday announced he will quit football following his 17th Premier League campaign after spending his entire career at the Merseyside club.
The former England international, who earned 38 caps for his country, is second only to Ian Callaghan in the club's all-time appearance table with more than 700 Liverpool games under his belt.
McAteer, who played alongside Carragher between 1995 and 1999, believes the Reds stalwart will be sorely missed when he does finally hang up his boots for good.
McAteer told Sky Sports News: "It's not just a sad day for football but it's a sad day for Jamie Carragher.
"I'm sure it's not been a decision he's taken lightly.
"He knuckled down, worked very hard and has become a credit to the game and the football club and that's why it's going to be a sad loss.
"When you have a player that has been around as long as he has it will leave a hole."
However, McAteer remains confident Carragher will remain part of the Liverpool set-up following his retirement.
He said: "The club are working closely now with the ex-players. He was very good with the media during the Euros and he knows the game inside out.
"I'm sure Liverpool won't miss the chance to see him there at the club in come capacity."
Jamie Redknapp, another member of the Liverpool squad during the 1990s, called Carragher an "icon of the Premier League".
"People always talk about you being a long time retired, but Jamie must know it's the right time to do it for his family and we have to respect that," Redknapp told Sky Sports News.
"Jamie's been thinking about this for a while now and his career and when is the right time to decide. The career that he's had, he's an absolute icon of the Premier League and for Liverpool and they are going to miss him.
"Only Jamie knows his body. He's the only person who knows what it's like day-in, day-out, how he recovers after a game. He'll be looking at it and think the right time is the end of the season and then he will look at his options.
"Maybe management, in the media - when he did the World Cup coverage he was exceptional. He's a real student of the game and watches every single football match on television so that's something he might want to do. He has many, many options."
Redknapp also thinks Carragher's calming influence and dedication will be sorely missed at Liverpool, and believes younger players should hold him up as an example of a true professional.
Redknapp said: "He's worked so hard at his game and the extra training he did behind the scenes and in the gym. The young players now look at all the trimmings but he is a good person to analyse for the dedication he's put in and the rewards he's got now.
"He won't want any fuss whatsoever, that's the last thing he likes. Brendan Rodgers will know he will get him 100 per cent until the end of the season."
Other figures from the world of football also took to Twitter as they reacted to the news.
Manchester United's Rio Ferdinand, who played alongside Carragher in the heart of the England defence, tipped the Liverpool man for a career in the dugout.
Ferdinand said: "Jamie Carragher has been a great servant to Lfc. Great player&a top fella! Funny guy! Nutter when we were kids but management material now."
Liverpool midfielder Lucas Leiva also posted a message on Twitter after hearing the news.
He said: "Just read Carra statement. Let's enjoy a few more months with him before he retires. It is amazing what he is done for LFC. #23"
Pundit and former England and Tottenham striker Gary Lineker added: "Jamie Carragher will hang up his boots at the end of the season. Those boots were worn by a player respected by everyone in the game."