He also claimed the lack of goal-line technology in football is "killing the game" after Spurs' controversial semi-final defeat to Chelsea at Wembley.
Tottenham were thrashed 5-1, but their task was made much harder when Atkinson allowed Juan Mata's 'ghost goal' to stand despite the fact that Benoit Assou-Ekotto cleared the Spaniard's shot before it crossed the line.
With Spurs already 1-0 down thanks to Didier Drogba's opener, they were compelled to throw men forward, and even though Gareth Bale pulled a goal back, they were ultimately hammered thanks to late goals from Ramires, Frank Lampard and Florent Malouda.
The defeat denied Adebayor his first appearance in an FA Cup final and he blames only one man.
"To be honest, we have to blame the referee," he said.
"I think we have done well and he has made a lot of mistakes and made a lot of wrong decisions."
He added: "We are all disappointed. But what do you want me to tell you? How many goals scored this season were disallowed that were onside?
"To be honest, I'm just tired of it and everyone is talking about goal-line technology.
"They have to make something happen because it's killing the league, it's killing the FA Cup, it's killing the game, it's killing everything."
Midfielder Scott Parker claimed no-one in the ground but Atkinson - who did not consult his assistant - thought the ball had crossed the line.
He added: "It's not even a debate, is it, really? I think it's clearly obvious.
"It's not even one we can sit here and say, 'Did it or didn't it?'
"It was nowhere near the line. I had a perfect view. Four players were covering the line, so how the ball could've got over the line, I don't know.
"The linesman said he didn't make the decision. The ref took it upon himself."
Spurs boss Harry Redknapp revealed Atkinson approached him after the game to apologise for giving the goal.
Redknapp now faces the tough task of picking his players up for their remaining five matches.
Spurs are clear of fifth-place Newcastle only on goal difference and Redknapp says his players must put the incident behind them.
He added: "I've got to find a team among that group who'll have the nerve, the will power and the desire to get that fourth spot now.
"We have to go to QPR next week for a tough game and show we can bounce back. It's important. It's a test of character now. They have to show some character."