Substitute Diafra Sakho broke City hearts with an 81st-minute winner, rising at the far post to beat goalkeeper Frank Fielding after Andy Carroll had done well to cross from the right.
The outcome was harsh on the League One high-flyers and Allardyce admitted to be fearing the worse early on.
“I expected us to play a lot better than we did,” Allardyce said.
“In the end one moment of quality has put us in the next round and that is the most important thing.
“All credit to Bristol City for never being overawed and taking their league form, which has been outstanding, into the FA Cup.
Twice the visitors were forced to clear the ball off the line as Matt Smith caused the Hammers rearguard all sorts of problems, but Allardyce’s men stood firm, and the West Ham boss believes withstanding the physicality of the home side was key to victory.
“Keeping a clean sheet made all the difference and it was a hard-earned victory for us,” Allardyce added.
“They had their best spell in the opening 15 or 20 minutes, but we battled all the way through to make that one bit of quality count.
“With the size Bristol have in their team, we knew we had to protect ourselves as best we could from set-pieces. We put players on each post because we felt at some point they would get headers at our goal and it proved a wise tactic.
“How far we can go in the competition will depend on the draw. We will have to face Premier League opposition at some point if we progress, but I would rather leave that until the final!”