The former Liverpool defender said he was sad to see the Portuguese depart and also accused some of his team-mates, without naming any names, of caring more about their own public image than that of the club.
Arbeloa also feels the squad need to accept their share of the blame for Madrid's failure to win more than two major trophies - the Copa del Rey and the La Liga title - in three years under Mourinho.
He told Gol Television: "Yes, it's a shame that Mourinho is leaving because he has given his life for the club. He has sacrificed a lot for this club, and he has suffered.
"He's always put Madrid first, ahead of himself and at the cost of his own image. I don't think that anyone at this club, including the players, could say the same.
"It's a shame that he hasn't been able to take more advantage of these three years by winning more titles."
Mourinho has had public disputes with some of Madrid's senior stars, and Arbeloa pointed the finger at members of the squad for the uneasy relationship between the coach and the players.
"There have been some people that have lacked maturity. There are some players that don't care about the image we have with the press, others care more about their public image," he added.
"We were looking forward to this year coming to an end because we couldn't achieve our objectives.
"We gave everything that we could have done, now all we want is to for the new season to arrive.
"In the first month of the season there were slip ups and we never believed we were capable of winning the league because we dropped so many points and lost the advantage on Barca."
Arbeloa admitted that Mourinho had not won as much as he would have hoped since joining Madrid from Inter Milan in 2010, and feels everyone, both players and the coaches, were to blame.
"I feel that we could have achieved a lot more because of the potential of the squad and the coaching staff," he said.
"We are all responsible. Mourinho has won trophies everywhere he has been and it seems strange to me that this squad, which is the best, is the place where he's won the least."