Former United striker Welbeck sent Arsenal through to the semi-finals of the FA Cup with a 61st-minute winner after Wayne Rooney had equalised Nacho Monreal’s opener.
To pile yet more misery on United boss Louis van Gaal, he watched Angel di Maria get sent off towards the end of the 2-1 defeat for pulling referee Michael Oliver’s shirt as he reacted angrily to being booked for diving.
Replays showed Di Maria was pulled back by Aaron Ramsey before he tumbled to the ground, but the way in which he reacted will not please United boss Van Gaal.
Di Maria can have little reason to complain. The ??59.7million signing snatched at Oliver’s shirt as he turned away after issuing him with a booking.
The dismissal is the latest episode in a long list of disappointments for the British record signing, who Van Gaal hoped would bring silverware to Old Trafford.
With United 10 points behind Chelsea in the league, that dream is almost certainly over. After a mildly encouraging performance in the first half, United wilted and looked blunt after the restart. Welbeck, the Longsight lad who left his boyhood club for north London last summer, provided the killer touch after a clanger from Antonio Valencia.
Van Gaal claimed he controversially ditched Welbeck last summer because the striker did not score enough goals.
While the forward has hardly been prolific at Arsenal, he will have enjoyed showing Van Gaal what he is missing here.
For Van Gaal, the dream of silverware may be over, but Arsene Wenger’s side are now very much favourites to regain the FA Cup after registering their first win at Old Trafford in nine years.
A look back at recent history suggested the capacity crowd inside Old Trafford was in for a treat.
Meetings between the two teams have thrown up some classics over the years. Spectacular goals have been scored, intense battles have been fought, red cards have been issued and even pizza has been thrown in the tunnel.
But, with neither team the major force they were 10 years ago, this match had of a low-key feel to it at the start.
Both started cautiously. Marouane Fellaini managed to treat United fans to an equal measure of joy and despair in the 13th minute when he turned brilliantly in the box and then clattered into Per Mertesacker and gave away a free-kick.
Welbeck looked cautious on the attack, but it was not the same down the other flank where Alexis Sanchez drilled a low shot which tested David de Gea.
Di Maria cut a frustrated figure on the right. He cut in from the flank and fired in curling crosses, but United’s front two of Rooney and Fellaini failed to take advantage. The one time Fellaini did take the ball down well, he laid the ball off to Ashley Young inside the box, but he fired over.
Arsenal raced up the other end on the counter. Alex Oxlade-Chamberlain sprinted past his marker before trying and failing to beat De Gea on the lob.
The impressive England midfielder did much better moments later when he helped the Gunners take the lead. He picked the ball up off Mesut Ozil, tip-toed his way past three United players and squared the ball to Monreal, who stroked the ball past De Gea.
The 9,000 Arsenal fans inside Old Trafford fell silent four minutes later when Rooney equalised.
Di Maria sent over another dangerous dipping cross. This time Rooney had peeled away from his marker Laurent Koscielny and he speared a diving header past Wojciech Szczesny from close range.
Arsenal appealed for a penalty after Marcos Rojo pulled Welbeck back, but referee Oliver waved play on.
Rooney was now brimming with confidence. He plucked a long ball from the sky and scooped a delightful 20-yard pass to Di Maria, but he could only stab the ball at Szczesny.
Di Maria found another way through the Arsenal defence before the break after a cutting pass by Valencia, but again Szczesny denied the Argentinian.
Van Gaal brought Phil Jones and Michael Carrick on for Luke Shaw and Ander Herrera at the break and Arsenal were also forced into a change when Ramsey replaced Oxlade-Chamberlain, who had suffered a hamstring injury.
United started to look more threatening. Fellaini created problems as he tried to plough his way through the middle. Di Maria used his pace to get in behind the Arsenal defence, but he could only blast wide.
Then, with just under half an hour to go, Valencia handed Arsenal a gift-wrapped ticket to Wembley.
The right-back sold De Gea short with a backpass after Jones had chested a long ball down to him and Welbeck sprinted through and tapped into an empty net after beating De Gea to the ball.
The former United striker thought about not celebrating for a second, but then pumped his fists with joy.
United fans responded by demanding their team attack, but they looked inept. De Gea stuck out his left hand to deny Cazorla and Arsenal looked comfortable.
Di Maria rounded off a miserable for the hosts when he was dismissed for tugging Oliver’s shirt after he had cautioned him for diving.
Adnan Januzaj appealed for a penalty soon after, but Oliver also booked him for simulation and Arsenal were able to coast to victory.