Terim, who doubles as the Turkey coach after recently taking over the national team, was voted out at a meeting of the club's board of directors.
"In spite of the decision of the Galatasaray Sports Club and the Board of Directors to extend the term of the contract with Fatih Terim for two more years as of May 2014, when Mr Terim has rejected the offer, the Board has instead decided to terminate the contract between him and the club," said a statement on the club's website.
"We would like to thank Mr Terim for all his services and wish him all the best in his future career."
Galatasaray - who spent heavily on the likes Didier Drogba and Wesley Sneijder - have had a disappointing start to the season, with only six points from four league games, while they suffered a 6-1 home thrashing at the hands of Real Madrid in the Champions League.
However, it seems as though doubts as to the long-term commitment of Terim, who had a contract until the end of the season in his third stint with the club, led to his removal as much as recent form.