Kevin Nolan's injury-time equaliser earned the Hammers a point but Allardyce felt his newly-promoted side should have won given the number of opportunities they created.
Nolan saw an acrobatic volley brilliantly saved by Sunderland goalkeeper Simon Mignolet while Carlton Cole, Ricardo Vaz Te and James Collins all failed to hit the target with close-range headers.
It is a familiar concern for Allardyce.
"At home particularly (last season we) ended up drawing games by doing what we did today - missing too many chances," Allardyce said after the 1-1 draw.
"We have a kink in our armour by not converting the chances we created into goals. That is a concern of mine.
"If we showed the clinical finishing that we showed against Fulham we would have won the game handsomely. We had headers - Cole over the bar, Vaz Te wide, Collins hit the bar from six yards out, no further.
"In the end it is not good enough.
"The good thing is we are creating those opportunities - and so many as a newly-promoted side - I just need to work on the lads improving their finishing."
The key difference between West Ham's performance against Fulham and the way they huffed and puffed against Sunderland was the absence of Andy Carroll.
Cole played the target man role and he does not bring the likes of Nolan into the game as well as Carroll, who was involved in two of West Ham's three goals against Fulham.
When the target man is linking with his runners, Allardyce's style of play can look dynamic but against Sunderland it was one-dimensional and predictable.
Sunderland, who had taken a 10th-minute lead with Steven Fletcher's fourth goal in three games, dealt well with almost everything that was launched at them.
Allardyce injected some variety into West Ham's game just before the hour mark through Matt Jarvis, who caused Sunderland constant problems down the left flank.
Eventually, it was his persistence to chase a long ball and hook it back to Modibo Maiga that allowed Nolan to beat Mignolet on the turn.
And with a daunting autumn fixture programme, it was vital West Ham earned something from a game they had dominated.
"Points on the board for us are critical. Arsenal is game seven then Manchester City, Chelsea, Manchester United, Tottenham, Newcastle.
"If we can accumulate enough points and continue the growth of confidence that we are showing now, then we can go into the game with less pressure on ourselves.
"I am frustrated as our players never finished the job off but we did not look like a newly-promoted side.
"To take a game to a side of Sunderland's experience, to dominate, get stronger and stronger and finally get the draw shows that we are committed and passionate about what we do."
Sunderland had to settle for a fourth consecutive Premier League draw which, including the end of last season, extends their run of games without a win to 12.
"I am not overly concerned because it is over two different seasons. We have to start winning. We got close today. We need to try and win next week," said Sunderland manager Martin O'Neill.
"That's four games we have played now. In the last 20 minutes of each of them we have been under pressure.
"We need to find a way to keep possession better.
"We knew we would be put under pressure. The disappointment for me was that for most of the time we dealt with it manfully.
"There are so many ways to play football now. It is within the game and we have to keep going.
"On the break - and I accept they were on the break - we had some really decent chances to put the game beyond them."