Tim Sherwood was handed an 18-month contract in December last year following Andre Villa-Boas' departure.
However, Gullit has suggested it is a "done deal" for Van Gaal to take over when he leaves his current post after this summer's World Cup finals in Brazil.
The former Chelsea and Newcastle manager told BBC Radio Five: "I know for sure he was at Van Gaal's house, the chairman. I think it's a done deal, but you never know."
Sherwood took over from Villas-Boas as caretaker after a 5-0 home defeat by Liverpool and steered Spurs to victory in three of their next four Barclays Premier League games.
However, last Sunday's 3-2 win over Southampton, in which they trailed 2-0 at one point, was their first in five in all competitions and just the 10th in the 21 games of his reign to date.