Andre Villas-Boas' position as Spurs boss has come under scrutiny this week after his team followed up their loss to Newcastle with a humiliating 6-0 reverse at Manchester City.
Tottenham will slip closer to the bottom half of the table if they lose to in-form Manchester United on Sunday, but Vertonghen sees no reason to panic even though the team have already lost four times.
The Belgian remembers how Tottenham and Villas-Boas came under pressure at roughly the same time last season following three straight defeats.
Spurs responded by losing just one of their next 16 league games and Vertonghen is hoping the team can go on a similar run this term.
"You cannot write us off," the Spurs defender said.
"We did not do well in this period last year with losses against Wigan and City and Arsenal. But we had incredible months in December and January and I think that is what we are aiming for now.
"We want to do well in those two months and then I think we can be back easily because the top half of the table is so strong that everybody can win."
Tottenham's confidence increased on Thursday night when they beat Tromso 2-0 in freezing conditions deep inside the Arctic Circle.
Still, the performance was hardly vintage Spurs and you do not have to be a football expert to know that United will pose a much stiffer challenge than Tromso, who were relegated from Norway's top-flight three weeks ago.
The Norwegians have no-one who is anywhere near as good as Wayne Rooney, who had a helping hand in four of United's five-goal blitz in Leverkusen in midweek.
Rooney will have taken encouragement from the way Sergio Aguero and Alvaro Negredo tore Tottenham's flat defence apart last weekend.
Vertonghen thinks the Spurs back four may be in for an even tougher ride this weekend when they come up against Rooney and Robin van Persie.
"Rooney and Van Persie are quite similar to the City strikers," the former Ajax man said.
"They move well and play better and better together now.
"They are at least the same level as Negredo and Aguero. They are unbelievable.
"They are the most important players of the team but the strength of Manchester United is that they have always been stronger as a team together."
Vertonghen insists Tottenham will not be petrified about the prospect of facing Rooney and co, however.
The defender believes confidence has been restored somewhat following the victory in Tromso.
"It was important for the group to come away with a win in difficult circumstances," he said.
"We are so motivated to do well on Sunday, especially after the City game.
"I am very confident for that game because the supporters will be behind us and we will be so motivated to do well after such a loss."
Tottenham landed back in England on Friday morning following their trek to the northern reaches of Scandinavia, but they had no time to rest.
Villas-Boas and his squad were back on the training pitches at Enfield in the afternoon to prepare for the United game.
The trip to and from Tromso took six hours in total, and the temperature barely crept above zero throughout their stay.
Villas-Boas raised a few eyebrows by fielding a strong team against the Norwegian side, but Vertonghen is sure his manager made the right call.
"We will make sure the next couple of days we will be physically ready for Sunday," he said.
"Not everybody who played against Tromso will play on Sunday.
"Anyway, these trips are always quite good (for team bonding) and we have a young group with a good atmosphere so I didn't want to miss this trip."