This is because of their on-field performances due to the off-field controversies that have engulfed them.
The Stamford Bridge club have been embroiled in the fall-out from the Mark Clattenburg affair, with the club having accused him of using "inappropriate language" towards John Obi Mikel last weekend, which has come hot on the heels of the John Terry racism scandal.
Saturday's 1-1 draw at Swansea, where Chelsea were denied victory by a late Pablo Hernandez strike, saw Roberto Di Matteo's side surrender top spot in the Barclays Premier League table.
But Bertrand believes the result is a rare blip, and that off-field distractions have deflected attention from their excellent start to the season.
"We've got off to a very good start this season and I don't think we have got enough credit for the start we have had," he said.
"It's all been negative stuff. With the draw today [Saturday] we are a bit disappointed but we just look forward to the next game."
The fixture in south Wales was the final instalment of Terry's four-match domestic ban for racially abusing QPR defender Anton Ferdinand.
Di Matteo is unsure of whether to throw the former England international back into the side for the Champions League meeting with Shakhtar Donetsk in midweek, but Bertrand admits the return of such an experienced defender can only help Chelsea.
He said: "A player like JT would be a loss to any club in any league. But it has not affected us too much, thankfully, because we have very good players to fill in. Gary Cahill has been outstanding every time he has come in.
"But JT could still play for any team so it will be a massive boost for us when he does come back."
While Victor Moses grabbed his first league goal for Chelsea since moving from Wigan, Fernando Torres drew another blank.
The Spaniard has now gone nearly seven hours without finding the net, although he has found the net nine times in his last 21 games.
He was well marshalled by Swansea captain Garry Monk, who impressed in his first Premier League start under Michael Laudrup.
But Monk, who has played for Swansea in all four divisions, defended Torres and believes the pressure of the £50million fee Chelsea paid to bring him to Stamford Bridge from Liverpool has placed extra pressure on him.
Monk said: "It was good to play against him. I came here on a free when we were in League Two and he's cost £50million!
"That's what you're dealing with - living in different worlds. It's great to pit your wits against the best. And he cost £50million for a reason and I just enjoyed it. I didn't give him a yard and just kept with him.
"He clearly has quality, but, yes, there are hundreds of players overpriced.
"Yet that is not his fault that Chelsea paid that sort of money for him.
"Obviously with the price-tag comes a lot of responsibility. He will feel that on his shoulders to perform every week.
"Once you don't perform, everyone is on your back. You could see that with Suarez at Liverpool, he's a good player.
"Even though he is Marmite to a lot of people, but out of everyone he is the best."