The champions had to settle for a third draw from five games in a 1-1 stalemate with Arsenal at the Etihad Stadium after Laurent Koscielny equalised eight minutes from time.
City took the lead in the 40th minute with a header from Joleon Lescott but were flattered by it and a point was the least the visitors deserved.
The game came five days after City suffered a last-gasp defeat by Real Madrid in their opening Champions League game, leaving Mancini less than happy.
The Italian is particularly concerned his side have not yet kept a clean sheet, with Koscielny's strike the seventh they have conceded in the league already.
Mancini said: "I'm very sorry today [Sunday] for the players because it was difficult after Madrid but I'm worried because we conceded a goal. We need to improve and we need to work.
"In the last four games we've conceded three goals from set-pieces. This can't happen. I'm frustrated but the players are frustrated about this. If you want to win, clean sheets are important."
The result left City in seventh, four points behind leaders Chelsea and three adrift of Manchester United, and Mancini urged his team to forget about last season.
He said: "What we did has finished, it's history. Now we are like all the other teams and I don't like that we drew three games. Probably we're missing two points, from the game against Stoke maybe."
Mancini, though, shrugged off an incident after the game where he appeared to push striker Mario Balotelli, a late substitute, down the tunnel.
He admitted the Italian is probably unhappy about his lack of playing time, but said: "I don't remember what happened after the game. I don't know if he was asking me something, it was not important. Tomorrow [Monday] I'll ask him."
Arsenal largely dominated the first half and would surely have taken the lead had Gervinho not taken a truly dreadful touch when played through by Aaron Ramsey.
The Gunners' failure to capitalise cost them in the 40th minute when Vito Mannone flapped and Lescott headed in David Silva's corner.
Mannone partially redeemed himself with a fine save from Edin Dzeko's shot while Sergio Aguero wanted a penalty at the start of the second half when he felt he was tripped by Koscielny.
The Argentinian was making his first appearance since suffering a knee injury on the opening weekend of the season, and he would have doubled City's lead with 10 minutes to go but for a smart stop from Mannone.
It proved a crucial moment as Arsenal broke down the other end, Santi Cazorla forced a flying save from Joe Hart and then sent in the corner from which Koscielny fired into the top corner.
Watch: Man City and Arsenal share spoils... More Videos
It was a tremendous finish from the defender, who was only playing because captain Thomas Vermaelen had been struck down by flu.
Both sides had chances to win it late on but Mannone saved Vincent Kompany's overhead kick and Aguero fired wide from the follow-up while, not for the first time, Gervinho missed the target when well placed.
City were certainly better after Mancini strengthened his midfield at half-time by replacing Scott Sinclair with Jack Rodwell, and he said: "The first half Arsenal played more because we had one player less in the middle.
"The second half we were better for 30 minutes but in the last 15 minutes we dropped too deep.
"I think we didn't play very well but we had some great chances to score at 1-0 and you need to score. We need to improve this. When we have a chance we need to score and close out the game."
Arsene Wenger felt Arsenal were unfortunate not to get more than a point but took great heart from the performance of his side, who again showed the pre-season doom-mongers may have been overly pessimistic.
The Frenchman said: "I'm pleased because I feel it was the minimum we deserved today. I'm pleased as well with the quality of our performance and our spirit. I'm a bit frustrated because I feel there was room to do more.
"We had early chances and of course very late chances, especially the chance of Gervinho at the edge of the box where we missed the target.
"But overall I hope the game will reinforce our belief, our confidence and reassure us about our potential in this league."