Arsenal's 1-0 victory over Newcastle meant Tottenham had to settle for a fifth-place finish despite their own 1-0 win over Sunderland, which came courtesy of Gareth Bale's 89th minute wonder-strike.
Had Marriner awarded Villas-Boas' team a penalty in the first half, the Portuguese thinks it could have been Spurs, and not Arsenal, who finished in fourth, however.
In the 21st minute with the score 0-0 at both grounds, Bale entered the Sunderland box and went down after colliding with Sebastian Larsson.
The Swede used both arms to knock the 23-year-old to the ground, but rather than award Spurs a penalty, Marriner handed Bale his fifth yellow card of the season for simulation.
Villas-Boas thinks had Spurs taken a first-half lead from the spot, that would have applied too much pressure on to Arsenal, who did not score until the 50th minute.
"I don't want to put the spotlight on the referee, but I think it (was) absolutely decisive," the Tottenham manager said.
"Apart from the distance that separates the grounds, the emotions are felt very strongly. It would have certainly been felt at Newcastle, and it would certainly change the running of this game too.
"Sunderland would have to be more adventurous, and so there would be more space."
Villas-Boas has repeatedly defended Bale this season amid accusations that he dives, and he thinks Marriner got the decision wrong.
"It was extremely disappointing decision in my opinion for the fact that Andre was on the conference at start of season and we spoke about Gareth's incidents," Villas-Boas said.
"It was a poor decision. It could have potentially changed the nature of other game too."
Marriner also failed to award Spurs a penalty in the second half when Clint Dempsey's shot struck Carlos Cuellar's out-stretched arm in the Sunderland box.
While Bale's goal was worthy of earning Tottenham a place in the Champions League, the harsh reality is that the team started sowing the seeds of their own downfall a long time ago.
On March 3 after Spurs won the derby at White Hart Lane, the Gunners went unbeaten until the end of the season while Tottenham dropped points in five matches.
Not winning against Chelsea and, to a lesser extent Liverpool, were forgivable, but teams who qualify for the Champions League do not generally lose to Fulham at home - as Spurs did in March.
Still, it would be harsh to castigate Villas-Boas too much. After all, since he took over he has lost the services of Luka Modric, Rafael van der Vaart and Ledley King while he has also had to do without key players like Sandro and Younes Kaboul because of injury.
Despite his troubles, the 35-year-old ended the season with 72 points - more than Spurs have ever accumulated since the Premier League began.