By Gabriel Tan
The Cheetahs went ahead in the 48th minute when Azhar Sairudin netted with a blistering effort from the edge of the area, but Geylang looked to have salvaged a share of the spoils five minutes from time when Jozef Kaplan scored from the spot, after Mun Seung-man had been fouled by Ante Barac inside the area.
However, Hougang refused to give up and went on to grab all three points in injury-time when Jordan Webb picked up the ball inside the Eagles box and danced past a couple of challenges, before drilling in an effort at Yazid Yasin's near post.
Speaking after the game, Bacina praised his players for recovering from the psychological blow of conceding so late in the game and forcing a winner.
"I have to congratulate my players for the effort they showed," the Croat said. "We never gave up and showed our character.
"I have to disagree with the referee's decision to award Geylang a penalty and it was hard to come back after that, but my players did well."
Bacina, who has displayed a dislike for using luck as an excuse several times this season when his side fell to narrow defeats, conceded his side were perhaps fortunate to finish the game as victors, but believed they deserved it.
"Many times this season people have told me we were unlucky to have lost games and I never like to accept that," he added.
"Maybe you could say we were slightly fortunate today [Wednesday] but we deserved that piece of luck."
Watch: Highlights - Geylang 1-2 Hougang United... More Videos
The Cheetahs boss also admitted the win over the league's bottom side was all the more important considering it comes straight after their disappointing exit from the RHB Singapore Cup quarter-finals at the hands of Gombak United.
"We definitely felt the pressure and we knew how important this game was for us," Bacina conceded.
"Sometimes, it is harder to recharge mentally than physically, so it was important we got this result."
Geylang United assistant coach Noor Ali revealed he was disappointed to lose to an injury-time goal, but was pleased with the effort shown by his players.
"Of course, losing the game in the last minute was tough," the former Singapore international said. "The game was very even and I feel a draw would have been a fair result.
"I have to give credit to my boys. They played well and created plenty of chances."
Noor did however admit the manner of their defeat highlighted the need for Geylang to play with more maturity, as they look to climb off the foot of the table.
"The boys have to learn from this game," he added. "We cannot afford to play around in our own half especially in the dying stages of the game.
"Hougang have quality players like Jordan, who did nothing for the whole game, but he had one chance at the end and he scored.
"We cannot afford to give our opponents opportunities like this."
Following Wednesday's result, Hougang are now up to eighth on the table, while Geylang remain bottom, behind 12th-placed Woodlands Wellington on goal difference.