At the turn of the year, Spurs looked certainties for Champions League qualification, but their form dropped towards the end of the campaign and, once again, Arsenal just pipped them to fourth position for the second successive season.
One of the most telling blows to Tottenham's Champions League push came in March, when Fulham won at White Hart Lane thanks to Dimitar Berbatov's second-half goal.
Spurs' season began to unravel soon after as they dropped points against Wigan and Chelsea, and Walker has not forgotten the impact of the loss to the Cottagers.
"Last season when Fulham came to White Hart Lane they beat us, so we owe them something for that," he said.
"I don't like losing any game, but I think that game was the turning point from last season."
If current form is anything to go on, Fulham will prove to be the ideal opponents for Tottenham.
The Cottagers have lost their last six games and their last run-out was the dismal 3-0 defeat to West Ham, which cost Martin Jol his job.
Fulham's players will be out to impress their new manager Rene Meulensteen, however, and Walker knows it would be foolish to write the west London club off.
"It will still be difficult," he said.
"No game is easy. It's a London derby. They are fighting for things just like we are fighting for things.
"We need to dig in like we did against Manchester United on Sunday, show enthusiasm and desire that we want to go out and win and if we do then I think we will get the result."
Spurs bounced back from their 6-0 defeat at Manchester City with a creditable 2-2 draw against the Premier League champions last weekend.
Although Tottenham have a packed schedule coming up, manager Andre Villas-Boas will be loath to make serious changes to his team for the Fulham game after Sunday's encouraging result.
Still, the Portuguese is pondering a change in formation from 4-2-3-1 to the 4-4-2 system which some fans yearn for so much.
"We can probably look at it," the former Chelsea boss said.
"It depends. We have been successful in some ways, with one off the striker, but there is a possibility."