The Welsh side may have salvaged a bonus point from last weekend's 31-26 defeat at Wembley, but the seeds of that loss were sown in conceding an early 10-3 lead before trailing 23-13 at the interval, leaving Holley's side with too much to do as they rallied late on.
Defeat on home soil tonight would leave the Ospreys facing a battle to reach the quarter-final stage from Pool 5, they currently trail Sarries by three points at the halfway stage.
The Swansea-based side are unbeaten at home in Europe since December 2005, but Holley says if that run is to continue his side will need to come out of the blocks a lot quicker than last weekend.
He said: "We didn't see too much that we didn't expect from Saracens last week, it's just that we didn't deal with it.
"They come with a big driving lineout and a kicking game, an aerial assault.
"Defensively in the first half we started slowly and gave them a head start.
"You can't afford to do that, it was the difference in the game really.
"We'll be working hard to ensure we don't give them that head start this week."
In Pool 2 Cardiff Blues can put one foot in the last eight by repeating last week's win over Edinburgh when they head to Murrayfield.
A win in the Scottish capital would maintain the Blues' 100% record in the competition and move them to the brink of a first quarter-final appearance since 2009.
But backs coach Gareth Baber insists his side are not resting on their laurels.
He said: "We are not comfortable, that is not how we see it.
"If we go there and lose it's not the end of the world but people start to look over their shoulders and that's where the pressure comes from.
"We have been afforded an opportunity to nail the group, but it's not a comfortable position to be in.
"Edinburgh will be hurting after last Friday and will want to come out and test us and create opportunities for themselves. They are going to fancy themselves at home and we will have to work hard to negate that and impose our game plan."
Tomorrow night's other Heineken Cup fixture sees Biarritz look to bounce back from their shock defeat to Treviso when they host the Italians in the other Pool 5 contest.
Saturday's action sees Leicester look to get back on track after suffering their heaviest European defeat for eight years at Clermont Auvergne last weekend.
The 30-12 reverse, which saw George Chuter and Manu Tuilagi sin-binned at a crucial juncture during the second half, as the Tigers dropped to third in the pool, behind Ulster who travel to face Aironi this weekend, and they realistically cannot afford a repeat of that result at their Welford Road fortress.
In Pool 4 Bath need an unlikely win over defending champions Leinster in Dublin if they are avoid being all but eliminated from the competition, while Glasgow will be looking to keep themselves in the mix for qualification when they travel to Montpellier.
London Irish play host to Racing Metro in Pool 2 following their impressive bonus point win in France, while Gloucester and Connacht face off at Kingsholm in Pool 6.