After a troubled start to his managerial reign, Moyes got his most significant win since replacing Sir Alex Ferguson as Robin van Persie's first-half header proved enough to defeat his own club.
It meant United hauled back their opponents advantage to five points, whilst extending their own unbeaten record to nine games.
Moyes was clearly jubilant at the final whistle.
However, he is adamant there is plenty of work left in front of him.
"I am not singing and dancing because we won the game, I am thrilled because the boys put in a great performance," he said.
"It's another step in the right direction but we have a lot of big steps. It is going to take a while for me to get it the way I'd like.
"We are going to get a few bloody noses along the way but this Premier League has shown it is not just Manchester United who are going to get a few of them, there are other teams as well."
Van Persie opted not to celebrate when he scored against his old club in the corresponding fixture last season.
However, after receiving the abuse of the visiting fans this time around, the Dutchman did not hold back after rising to nod home Wayne Rooney's corner.
"Great players score great goals in big games - and he is a great player," said Moyes.
"I genuinely didn't hear what they sang, so I don't know if it inspired him."
Phil Jones was deservedly named man-of-the-match after a towering effort, first in midfield, then, following Nemanja Vidic's departure, in central defence.
Chris Smalling excelled too, whilst there was a typically industrious performance from Rooney, whose all-action contribution characterised the United effort as a whole.
"The energy, the way he worked and some of the stuff he did was magnificent," said Moyes.
"He gave the ball away a couple of times, but he did some brilliant things."
Moyes is not content though, not with his team still outside the Champions League spots in what is turning into a most bizarre of seasons.
"I want to be top," he said.
"Why would you want to be five points off it?
"We are where we are because we have had a few games where we didn't do as well as we should.
"We have to try and claw back the points as the season goes on."