The former Manchester United defender is currently splitting his time between his coaching role with England and his job as main pundit for Sky Sports.
Neville has earned high praise for the articulate and insightful manner in which he delivers his analysis.
But, without meaning to, sometimes he goes too far.
Never was that more true than following Chelsea's defeat at Liverpool last season, when he likened Luiz's performance to "being controlled by a 10-year-old on a PlayStation".
It was taken as a signal Neville did not feel Luiz was a good player, which the 85-times capped full-back is adamant is not correct.
"Once or twice I have overstepped the mark and said things in innocence but I can see why it gets portrayed in a certain way," he said.
"The David Luiz comment was a moment in time, describing a performance within a game.
"It wasn't 'I don't think he is a good player and he never will be a good player' like it has been interpreted for the last two years.
"He was sensational against United at Old Trafford and there are incredible merits in him.
"But through that statement it is like people think I have turned off him as a player."
As his playing days drew to a close, it had been anticipated Neville would head straight into management.
Instead he opted for life in front of a TV camera and claims he still would not be involved with coaching had Roy Hodgson not made his surprise offer ahead of Euro 2012.
"There have been two or three times in the last 18 months where I have been offered roles as a manager," he said.
"But it doesn't feel right at this moment in time. It would not be the right decision.
"I have completed my A licence and I am doing my pro-Licence at the moment. I am really enjoying that.
"And I love it with England. If it had not been for that I wouldn't be coaching.
"I see a lot of people rush into coaching too quickly.
"In two years they are finished. They are disillusioned with everything because they are in the hands of some crazy men."