Rangers owner Tony Fernandes on Sunday gave his backing to under-fire Hughes - the second time he has done so in a week - despite the club being bottom of the Barclays Premier League with two points from their opening seven games.
But Nelsen knows QPR have to show a vast improvement after poor defending contributed to Saturday's 3-2 defeat at West Brom.
Nelsen said: "We have to live up to expectations. It's our livelihoods, we have to win games.
"It takes time to gel and injuries to players in key positions haven't helped.
"The only real bright spot for us is that at least this run is happening now. There's a long way to go yet.
"But we know each and every one of us have got to be a lot better. We're letting the manager down here.
"We just have to win matches. I think every player has let the manager down at some stage this season."
Nelsen is aware QPR have to cut out the soft goals which led to their downfall against the high-flying Baggies, before a second-half revival almost earned them a point.
He said: "It doesn't matter how good our second half was. We conceded two very soft goals at the start.
"You just can't concede those sort of goals at this level. You get punished and it leaves us with a mountain to climb after that.
"It took us until the second half to play to our potential. You just can't do that in this league. We're leaving ourselves too much to do.
"It gives us a mountain to climb. Teams aren't really earning their goals against us to tell the truth.
"We have to start from 20 yards behind all the time. We have to remedy that and as soon as we do it'll be a lot easier to get ourselves up to win games.
"Unfortunately, we have had injuries to our wide defenders, but I don't want to make excuses."
Nelsen has also challenged midfielder Adel Taarabt to start showing the kind of impressive performance he produced against Albion, and against West Ham on Monday, on a regular basis.
He said: "He's very easy on the eye. He can create stuff out of nothing.
"With Adel, he has those flashes of brilliance which no-one else has. It's consistency with him.
"Hopefully he can string out some performances week in, week out. Then he'll be a valuable asset."