In a match marred by a seroious looking neck injury to Gombak midfielder Mustaqim Manzur, the Bulls went ahead after only seven minutes at their Jurong West Stadium, South Korean forward Jung Hee-bong firing home to register his first goal of the campaign.
Not to be outdone, Hougang responded shortly before the break courtesy of Mamadou Diallo's strike after 43 minutes.
However, Hafiz Rahim handed Gombak the lead for a second time in the 69th minute, before Hougang's chances of at least a point were all but extinguished by Syaqir Sulaiman's red card with six minutes to go.
Despite the victory Gombak coach K Balagumaran was far from happy after a match in which luck more than played its part.
"Actually, I'm not impressed with how we played," he said.
"The way we used to play was good football but we didn't get the results. The flow of the football wasn't the same as previous games.
"Even in our last match, which we lost 3-0 to Brunei DPMM, we played good football. Tonight we had a bit of luck."
The win sees Gombak climb to eighth in the table. Tanjong remain 11th with four points from the seven games they have played so far.
In Sunday's other S.League match, Home United inflicted a first defeat of the season on Harimau Muda, winning 2-1 thanks to goals from Shotaro Ihata and Frederic Mendy. Ahmad Hazwan replied for the Malaysian outfit.