Jose Mourinho's side looked to have got their Barclays Premier League title challenge back on track with three straight wins but their momentum was halted by Saturday's 3-2 loss at the Britannia Stadium.
The biggest frustration for the Blues was that they appeared in complete control after Andre Schurrle's early goal.
But they did not capitalise on numerous chances and goals from Peter Crouch and Stephen Ireland either side of half-time turned the game on its head.
Schurrle quickly levelled but Chelsea were hit with a sucker punch in the 90th minute when substitute Oussama Assaidi exploited space down the left and found the top corner.
Cahill said: "It's massively disappointing because it's so tight at the minute that you've got to keep chalking up results.
"A game which we felt we could get points from, we've lost, which is very disappointing.
"When it's at 2-2, it swings either way and we've got to push for the win because we need to try and get the three points, but it opens you up a little bit and that's what happens."
Cahill was also unhappy Jonathan Walters was not penalised for a foul on Cesar Azpilicueta before setting up Ireland for Stoke's second goal.
He said: "Early on, I thought we were in control. We had a few chances. We created quite a bit in the first half and I felt, if we had got a second goal, we would have gone on to win it comfortably. But it wasn't to be.
"One-nil up, they came back into it, and at 1-1 it's a different game.
"I thought the turning point was the second goal. I thought it was a foul on Cesar. That's the way I saw it and it was such a big turning point.
"It was one of those headers you had to win, otherwise they were through, and I felt he (Walters) just looked at the player rather than the ball and just nudged him for the foul, so it should have been a free-kick."
It was the second game in succession in which Chelsea have conceded three goals following their 4-3 win over Sunderland in midweek.
They have now conceded more goals in the league than in the whole of Mourinho's first season in charge in 2004-05, when they won the title.
Set-pieces have been a particular concern, and goalkeeper Petr Cech was culpable for Crouch's goal, coming out for a corner but being caught in no man's land.
Cahill said: "Losing games is a concern. We've kept a lot of clean sheets this season, but in the last two games we've let too many goals in. That's unlike us and it's down to individual mistakes.
"We talked about it after Sunderland and there were unfortunate incidents at the set-pieces. We've had a look at it and they were not major, major mistakes. Maybe it's just a bit of bad luck but obviously, away from home, we've got to keep it tight.
"I feel we've done that most of the season, but this result was poor for us."
Midfielder John Obi Mikel admitted complacency had played its part after Chelsea began so brightly.
He said: "I thought we were going to win it. The first 10 or 20 minutes we thought it was going to be a very easy game because in the first half we played really well.
"In the second half, we came out, conceded a goal and it changed everything and we had to fight for it. We were pushing for a winner, left a little space and they got the winner. It was a great goal."
Chelsea have the chance to get back to winning ways in the Champions League on Wednesday when victory over Steaua Bucharest would guarantee them top spot in Group E.