Two James Collins headers, both from crosses by the returning Joe Cole who was back in Hammers colours almost 10 years after his departure to Chelsea, looked set to earn the home side a 2-1 victory.
But Ryan Giggs played a tremendous long pass into the path of his fellow substitute Robin van Persie who rifled in a stoppage-time equaliser to secure a 2-2 draw at Upton Park.
United manager Ferguson always hoped there would be late drama, and insisted he never thought the game had got away from his players.
"You never do that with our team," Ferguson said.
"When you're 2-1 down with a couple of minutes left you just hope you get a break, but the manner of goal - the pass from Ryan Giggs and first touch from Robin van Persie and his finish - is absolutely world class, absolutely fantastic."
Ferguson praised the influence Van Persie is having at United in his first season with the club.
His goals have been vital, but Ferguson sees more to the Dutchman's game.
"He's at a mature age now. He's brought that maturity to our front line," the Scot said, "and his performance after coming on was tremendous."
Cole was substituted before the United late show denied him the taste of victory on his first appearance since signing from Liverpool.
"It was a great start on a personal note but it still feels like a missed opportunity," Cole told ITV1.
He recognised United would keep going until the final whistle, and appreciated the quality of their leveller.
Cole said: "They're a quality side and they've done that countless times.
"When you've got world-class players like Van Persie and Giggs, they can combine like that.
"I thought we defended really well. Good concentration, good heart and good discipline.
"We were unlucky not to win."
Now a replay beckons.
"We've got to go to Old Trafford again and that's a really tough game but we've done it before so we'll have another go," Cole said.
Reflecting on his performance, Cole added: "It was a nice return.
"I didn't like all the fuss, I just wanted to get out there and play football.
"I was nice to get involved and get the minutes under my belt. It's been a long time since I played properly.
"Three or four games and I'll be back to my best."