The Portuguese is under serious pressure at Stamford Bridge after their loss to Liverpool, and his cause was not helped by Neville's remarks, which were made in his capacity as Sky Sport's pundit.
According to reports in the Sun, Neville had slammed Luiz's performance, saying that the Brazilian defender looked like he was being controlled from the stands by a ten-year-old on his PlayStation machine.
Villas-Boas was upset at the comparison, saying: "Gary Neville was a fantastic defender but it is a stupid approach to an opinion.
"If that's the way he wants to take the game that is ridiculous."
The manager was also unhappy with the number of goals that Chelsea have conceded in recent weeks, saying: "The amount of goals we are conceding is something that worries me.
"So we have to analyse what will make us better in terms of defensive organisation."
"When you are conceding goals in situations like this and being so low in the league, you focus on the details that you have to get right.
"That's exactly what we're doing and we know where we have to improve. We don't concede a lot of opportunities but our opponents are being more efficient than us.
"But I still trust our defenders because we have immense talent at our disposal and we all believe we will get it right."
Despite being a massive 12 points off Premier League leaders Manchester City, the 34-year-old still believes Chelsea can lift the Barclays Premier League trophy, insisting: "There is no running away from our responsibilities, which is to win trophies. When you are at a club like Chelsea, you cannot call for a period of transition because you must never forget that your fans expect you to win titles".
"We are still in contention for four competitions and we still have the possibility to win them all.
"At the moment the view is that we need to win every game if we are to maintain a title challenge.
"We are on a bad run and no one wants to experience defeats like this.
"Manchester City are playing well, United are playing well and are in front of us. Arsenal are playing well and recovering, Newcastle are playing well and so are Liverpool and Spurs. Now we have to play well, too, to get the results we need.
"It is a long time since 12 points were last recovered but that doesn't mean we can't do it."
There has been criticism of his tactics, with some saying the Chelsea manager should have shut up shop after Daniel Sturridge had equalised to cancel out Maxi Rodriguez's first-half goal for Liverpool, but Villas-Boas was typically defiant.
"In our position, salvaging a point is not what we are looking for", he proclaimed.
"We had the opportunity to go third today and we were trying to shorten the distance at the top and had the game in our control. The Premier League is behaving rather differently than in previous years and there are a lot of teams challenging for the title this time.
"Liverpool showed aggression, commitment and strength and they pressed very well in the first half. In the second half it was exactly the opposite - we came back strongly and managed to equalise within 10 minutes of the restart.
"But just when things looked to be going our way Liverpool made the most of a good opportunity they had and managed to take an important three points.
"The decisive moment for me was how Glen Johnson got free on the opposite side of the ball and got through on goal to score."
There has also been talk about the possibilty of Guus Hiddink returning to the Chelsea hotseat, should things not work out for Villas-Boas, especially as the Dutchman had led the London club to a FA Cup victory in his six month tenure previously.
But Villas-Boas said: "Guus had a very important period at the club - but it was six months at the club.
"It does not mean that he can always be as successful as that in a club of this dimension of tremendous expectations.
"Guus is one of the top managers in the world, a manager that is always available when a top team arises.
"But at the moment this is not a vacancy to be taken", the Portuguese defended.