Gary Hooper's goal on 30 minutes proved enough to give the Canaries a much-needed victory, which handed under-pressure Chris Hughton some breathing space following seven Barclays Premier League defeats before Saturday afternoon.
However, the Eagles - boosted by last weekend's away win at Hull - were probably worth a point, particularly for their second-half efforts, and had seen a shot from Barry Bannan headed onto the crossbar by Norwich defender Martin Olsson while, with just a couple of minutes left, Cameron Jerome missed his kick when in front of goal.
"You have to take your chances. We did not have the breaks today," said Pulis, whose appointment was confirmed just ahead of the Hull game.
"How the hell (Norwich defender Sebastien) Bassong gets that (clearance) over the crossbar from there, it was unbelievable, and then the kid who stops Barry's shot from going in with his header, Cameron had a really good chance, so we have had the opportunities.
"But I can't fault their work-rate, they gave it everything, but you need a break and we did not get that."
Sunderland's goalless draw at Aston Villa pushed Palace back to the bottom of the table, but they are not yet cast adrift ahead as the south London outfit prepare for key home games against West Ham and then Cardiff.
"There are certain clubs we cannot compete with, and there are others we have to compete with so against those you have to give a performance," added Pulis, who left Stoke at the end of last season.
"We have come away from home and done that because we dictated the game for long periods.
"The quality at times, I was very pleased with.
"But it is going to be tough. We have a mountain to climb in a lot of respects, but it is not the first time I have been in this position."
Hughton's role, meanwhile, has come under scrutiny following inconsistent results on the back of a £20million summer spending spree.
The Norwich manager, though, felt his side - hit by injuries to key midfielders Anthony Pilkington and Robert Snodgrass last week - produced the required response.
"It feels like a big win for us," Hughton said.
"We played very well in the first half and scored a good goal, but we knew they would come back.
"It is never comfortable viewing when you look at the league table and you are in a position you do not want to be in, so whatever places we have jumped tonight is a nice feeling."
England Under-21 winger Nathan Redmond, signed from Birmingham, was the instigator of the goal for Hooper, with a determined run down the right before Johan Elmander swept the ball into the penalty area.
It dropped through to Wes Hoolahan at the far post, and he squared it for Hooper to roll the ball into the net.
"I was pleased with his overall performance," said Hughton of the striker signed from Celtic in a £5million deal.
"He is getting stronger and fitter after missing the first few weeks of the season and his overall contribution was good, so I am really pleased for him."
Irish playmaker Hoolahan made his first Premier League start of the season.
Hughton said: "There is no doubt his best position is more central and for this season so far I have looked at other options, with two up front.
"It has been tougher for Wes, but the feeling we have for him is the same because he is a super player and has just lost out on some choices by myself.
"He has been frustrated, but he has handled it very well and trains very well every day."
There was even a cameo for disgruntled Norwich forward Luciano Becchio, reprimanded by Hughton after speaking out over lack of opportunities, sent on with 11 minutes to go.
"You want players to be knocking your door down, wanting to play, and the door is never closed," Hughton said.
"The moment somebody thinks it is not their time and they are not going to be playing then something happens, but that is part and parcel of the game."