The independent Football Association regulatory commission said there was "no credible basis" for Terry's claim he had only been repeating words he thought Ferdinand had accused him of saying.
In their full written reasons for the four-match ban, the commission said they were satisfied the words "f***ing black c***" were intended as an insult by Terry. He now has two weeks in which to appeal.
The commission also stated: "There are further aspects of Mr Terry's defence that the commission finds improbable, implausible and contrived, and which serve to underline and reinforce our decision.
"The commission is quite satisfied, on the balance of probabilities, that there is no credible basis for Mr Terry's defence that his use of the words 'f****** black c***' were directed at Mr Ferdinand by way of forceful rejection and/or inquiry.
"Instead, we are quite satisfied, and find on the balance of probabilities, that the offending words were said by way of insult.
"We are able to arrive at that decision without needing to make any adverse findings against Mr Terry arising out of his decision not to give evidence.
"Accordingly, the commission finds that there is 'clear and convincing' evidence."
The commission said that character references from a number of people including black players made it clear that Terry was not racially prejudiced.
"It is accepted by everyone involved in the criminal and disciplinary proceedings that Mr Terry is not a racist," added the commission.
Ashley Cole's statement supporting Terry's version, and the role played by a Chelsea club official, has also been questioned by the commission.
Terry had been cleared in Westminster Magistrates Court in July of a racially-aggravated public order offence, partly helped by the testimony of England and Chelsea team-mate Cole.
However, the commission found that there were discrepancies in Cole's initial statement to FA interviewers of what he heard Ferdinand say to Terry compared to later statements.
Cole did not mention the word 'black' in the initial interview with the FA on October 28. On November 3, Chelsea club secretary David Barnard asked the FA for the specific word 'black' to be inserted into Cole's witness statement, suggesting that Cole may have heard Ferdinand use the term.
The commission saw an email exchange between the FA and Barnard and said that should be regarded as "cogent new evidence".
The commission said: "These highly material issues relating to Mr Cole's evidence were not addressed by the Chief Magistrate - he clearly did not have the interview notes of the FA's interviewers, or Mr Barnard's statement before him - and they do not appear in his judgment.
"Accordingly, that material can and should properly be regarded as cogent new evidence.
"Had it been before him, the commission has no doubt that the Chief Magistrate would have examined Mr Cole's evidence as to what he claims he heard Mr Ferdinand say to Mr Terry on the pitch very carefully indeed, or scrutinised it even more closely than he may have done."
It adds: "All of this causes the commission to have very real concerns about the accuracy of Mr Barnard's recollections, and the motivation for the assertions that he makes in his witness statement about what Mr Cole said during the FA interview of him, particularly his alleged use of the word 'black'."
Ferdinand had suffered "hateful abuse" as a result of the case but had acted with dignity, said the report.
The commission stated: "The victim impact statement of Mr Ferdinand makes it plain that he has been badly affected by the incident. He has been the subject of hateful abuse and adverse comments, but has acted with restraint and dignity."
The FA had told the panel that Terry's stature as club and England captain at the time was an aggravating feature in the case.
"His conduct undermines the FA's efforts to promote inclusivity, equality and diversity and in combating racism in football through the Kick it Out campaign," the FA had told the commission.
Chelsea would not comment on the contents of the written reasons.
A Chelsea spokesman said: "As we said last week we recognise that John has the right to appeal.
"In view of this it would be inappropriate for us to comment further on the matter at this time."
Chelsea manager Roberto Di Matteo insisted he did not believe Terry capable of discrimination.
Di Matteo said: "From my judgment on this man, I've known him for many, many years since we played together.
"I've never had any doubt about the fact that his comments wouldn't be of any kind of discrimination against any kind of ethnic party."
Asked if he still stood by Terry, he added: "At the moment, he's our captain and he's available to play."
The anti-racism charity Show Racism the Red Card said the Terry affair had "tarnished" English football.
Paul Kearns, the organisation's assistant chief executive, told Press Association Sport: "We welcome the release of the report today and it is clear the issue has been taken very seriously by the FA and we would always call for the strongest possible punishment for racist abuse.
"The written reasons are a very detailed analysis and set out the clear reasoning behind the decision.
"I think the case has tarnished English football - there are no winners in any case of racist abuse and football has clearly been a loser."
Kick It Out, the group that campaigns against racism in football, said in a statement: "The written reasons are welcomed and acknowledge some important consequences of this episode, namely the trauma caused to the Ferdinand family.
"It's a period that has polarised the game, undermined the good work being done at all levels, and it must never be allowed to happen again.
"We note the rationale for the outcome and there is still a right to appeal, which we respect. In addition, any sanction by Chelsea, as Mr Terry's employer, is subject to the club's own internal disciplinary procedures.
"The words used are completely unacceptable, irrespective of how many times they were levelled. As campaigners we remain committed to the strongest action being taken against discrimination and supporting the victims of it."