England's number one has lost his place in the Manchester City side after a series of high-profile mistakes.
That has raised questions over his position in the national team and concern at the apparent lack of alternatives.
Cottee feels it is time to give some others a chance, starting with Celtic's Fraser Forster, and would do so even if Hart was in brilliant form.
Cottee, speaking at the launch of the William Hill-sponsored Football Supporters' Federation 2013 Awards at Leyton Orient FC, said: "I don't think Joe should play in these two games anyway.
"We know what he can do, we know how good he is. He is not in the team at City at the moment, and we understand that, but I think it is time for Fraser Forster to be given a chance.
"He has done very well at Celtic and is playing Champions League football.
"I'd give him the two games because if we do play Joe in the World Cup and he gets sent off in the first minute of the first game, or gets injured, you need a goalkeeper with international experience.
"I've nothing against Joe, I just want to see someone else play in goal because we need to give them a chance."
England boss Roy Hodgson was expected to name a number of youngsters in his squad for the two matches at Wembley on November 15 and 19.
That is an approach Cottee, the former West Ham and Everton striker who won seven caps for his country, would back.
The 48-year-old said: "I'd like to think Roy will give the squad players a game.
"I don't really see the point playing the likes of Rooney, Gerrard, Lampard. We know what they can do at international level.
"I think it will be a good opportunity for Roy to tinker with the squad and maybe try different formations, different players."