Irons midfielder Joe Cole had delivered a vintage display on his return to his boyhood club following a free transfer from Liverpool this week, rolling back the years as he set up a double for defender James Collins with superb crosses from the left flank in each half to turn the tie around after Tom Cleverley had fired United into the lead on 23 minutes.
However, as the clocked ticked into stoppage time, Dutchman Van Persie - who had started on the bench - ghosted into the right side of the penalty area following a superb crossfield pass from fellow substitute Ryan Giggs, before drilling a low shot into the bottom corner to rescue a 2-2 draw and force a replay in a thrilling FA Cup third-round tie.
Ferguson said on MUTV: "We missed some good chances, with headers over the bar and (Shinji) Kagawa had one blocked on the line, I thought it was not going to be our day.
"But you have to say the pass from Ryan Giggs, then the first touch from the ball from Robin and his finish was world class."
Ferguson was delighted to have made sure they did not suffer a disappointing early FA Cup exit.
The United boss continued: "We want to stay in it and have a go at it this year.
"We have had some terrible draws over the years and been knocked out early and we don't enjoy that.
"For us to be losing 2-1 in that game is amazing with the way we played.
"But when you play West Ham you have got to defend those crosses and set pieces, because they are brilliant at that, the best team in the league, but we did not do that.
"Twice in the same position, (it was) James Collins from a cross by Joe Cole and we have lost two goals.
"I could say it is a travesty, but they are good at it, but with some of the chances we missed, we should have been home safely."
West Ham manager Sam Allardyce admitted it was "more than a tad disappointing" not to have been able to close out what would have been a memorable victory.
He said: "You have never beaten Manchester United until the final whistle blows and I have learned that in the past.
"When Matt Jarvis broke down the left-hand side late on and put in a great ball for Mark Noble to head down to Matt Taylor I was going, 'Please put it in', because Manchester United weren't going to come back from that.
"But if we didn't put it in (I knew) it would be nail-biting until the final whistle because somewhere along the line they might just carve out a chance with the quality that they have got.
"It was just a brilliant goal they scored.... the ball from Ryan Giggs, who smashed a half-volley over James Tomkins' head from about 60 yards, and Van Persie's first touch and finish were just unbelievable."
Allardyce felt his team could take great credit from their efforts.
He said: "It was a great performance and in that dressing room we almost feel like we have lost when you get done in the last minute.
"But we are still in the hat, we have a difficult January now. That game gets fitted in, and we have some important league games before and after that, so we have to get the players fit who are out injured."
Allardyce felt Cole's performance could not have been better.
The West Ham boss said: "It's obviously a fantastic debut for him and a great reception from the fans, which must have made him feel really good, but the whole team has played really well."
Allardyce, meanwhile, said he was close to settling a damages claim against Steve Kean out of court.
Kean, during his stint as Blackburn manager, was filmed making apparently disparaging comments about his predecessor Allardyce.
Allardyce, who was sacked by Blackburn in December 2010, said of his claim: "No, I don't think it's final but it's close."