Georgios Samaras' 25th-minute strike had put the Hoops ahead and they looked in control as the second half unfolded.
However, with 15 minutes remaining Hapoel playmaker Nemanja Vucicevic slammed in the equaliser after being set up by Samuel Yeboah.
A point would have been welcomed by the Parkhead side as they struggled in the tense final but even that was not forthcoming as substitute Maaran Lala, on after the break for Zurab Menteshashvili, took advantage of more disarray in the Hoops defence to bundle in the 88th minute winner.
The visitors trooped off the pitch as shocked as the hardy band of Hoops' supporters who had travelled over to see their club's first game in UEFA's newly-revamped competition.
Most of talk beforehand in a humid night in Tel Aviv was of Celtic skipper Stephen McManus making his first start of the season for the Parkhead club since returning from a knee injury and of striker Scott McDonald, who was perhaps surprisingly dropped for Samaras.
However, the Greece international vindicated Tony Mowbray's decision with an impressive first-half performance which saw him give the visitors the interval lead after hitting the bar with an earlier effort.
Mowbray's idea to have Aiden McGeady, Shaun Maloney and Scott Brown supporting Samaras was positive but nevertheless it was Hapoel who might have taken the lead with less than a minute played when left-back Dedi Ben Dayan's cross to the back post found Bebars Natcho.
The midfielder found enough height to reach the ball but from barely six yards out he cleared Artur Boruc's crossbar.
The Parkhead side battled hard to get a foothold in the game and their first effort on goal after seven minutes saw Gary Caldwell, playing in midfield, head Danny Fox's inswinging free-kick in to the side netting.
In the 12th minute, with Celtic settling into the game, Samaras rose highest inside a packed six-yard box to power a Maloney corner goalwards, only to see Hapoel keeper Vincent Enyeama help the ball on to the bar with his fingertips.
The home side survived the brief moment of panic before coming close again themselves seconds later.
Natcho sent Vucicevic racing down the right and the midfielder cut inside Fox before driving a left-footed shot in from 16 yards which Boruc did well to block with his feet.
The complexion of the game, though, changed more markedly in the 25th minute when the visitors took the lead through their lively Greek striker.
In text-book fashion, he controlled a through-ball by Caldwell with his chest and took himself clear of the static Hapoel rearguard before dispatching a left-footed drive from 16 yards past Enyeama.
For the rest of the half, Celtic played with more confidence while the Israeli side dropped down the gears.
UEFA's decision to experiment with an extra referee's assistant behind each goal was tested in the early moments of the second half when Ben Dayan lobbed a speculative ball in from the left for striker Yeboah to attack.
The covering McManus got his foot to the ball but it appeared to then strike his hand however, none of the officials, including referee Paul Allaerts, were impressed, despite loud appeals for a penalty.
Seconds later Boruc was called in to action to parry a powerful long-range strike from Hapoel midfielder Gil Vermouth, as the home side and their fans sensed there could be a way back in to the match.
Niall McGinn, who made his Celtic debut against Dundee United on Saturday, made his first European appearance for the Hoops as a 65th minute replacement for the tiring Landry N'Guemo.
But in the 75th minutes, and out of the blue, the home side levelled through Vucicevic.
Yeboah, who moments earlier had headed over the bar from close range, made his way to the by-line before cutting the ball back in the area and with the visitors' defence in disarray, the Serbian playmaker blasted high past Boruc.
The Parkhead men were reeling and they met with disaster with two minutes remaining when Vermouth cut the ball back for Lala to knock in from close range, completing Celtic's misery.