Striker Drogba left Stamford Bridge in the summer after a glittering eight-year stay which saw him win three Premier League titles, four FA Cups, two League Cups and finally, the Champions League.
He sealed that memorable success over Bayern Munich with the winning penalty, having already scored in normal time - his 157th and final goal for the club.
A move to China and Shanghai Shenhua followed but, although Chelsea have made an unbeaten start to life without him, Mourinho believes it is only a matter of time before they start to miss him.
The Real Madrid manager told Sky Sports News: "I believe Drogba is the big difference for them. I don`t think you can replace him. You can buy top players and invest like Chelsea did, but there is only one Drogba, in the Premiership and Chelsea`s history.
"Drogba means crucial moments. The final, a top match, Drogba is there. When the team is losing at home, Drogba is there. Chelsea will miss Drogba and Drogba will mean a few less points. But the team is good, with good players and they are European champions."
Mourinho kickstarted the club`s run of success by guiding them to back-to-back titles in 2005 and 2006, and whenever a vacancy has cropped up at Stamford Bridge, he has been linked with a return.
And, although he has reiterated his love of the club, he says he will be returning to England and to any club when his time with Real comes to an end.
He added: "In England I am blue. I will push a little but when I watch matches on TV.
"I cannot deny what is clear in my mind. After Real Madrid, England, that`s for sure. I keep saying the same, I know clearly.
"In England (I could manage) everyone. I am blue, but I am London blue. I am a professional."