Carroll, 23, was signed on a season-long loan from Liverpool towards the end of the transfer window and made an instant impact on his debut against Fulham.
The day was ruined for the England striker when he was forced off with little over an hour gone after suffering a hamstring injury that could see him miss up to six weeks.
Allardyce is likely to turn to Cole to replace the former Newcastle striker when he takes his side to Norwich on Saturday and wants the 28-year-old to show a lethal touch in front of goal.
He said: "He is an obvious replacement and we would like Carlton to do how he finished off last season.
"He scored a hatful of goals for us and helped us get back into this lucrative Premier League so we would like him to step in and start scoring the goals he did towards the back-end of last season.
"We hope he can hit the goalscoring trail when he gets the opportunity on Saturday."
If Cole is selected in the starting XI at Carrow Road it will be his 150th start for West Ham and he will be hoping to establish himself in the side before Carroll returns from injury.
With the club still yet to put a definitive time-frame on his hamstring injury Allardyce is hoping he could welcome Carroll back earlier than expected.
"We have got no time on it really," he said. "It will be a maximum of six weeks but he is improving well so we continue day-by-day and look at every day and see what type of improvement we have got and go from there.
"If there is a possibility we can get him back quicker we will be trying our best."
The loan signing of Carroll was one of the marquee deals of the transfer window and the arrival of the England international, coupled with other new arrivals, has left Allardyce delighted with the some of the options he now has at his disposal.
"The last three signings that the co-chairmen okayed were brilliant for us," he said.
"I think Matt Jarvis, Andy Carroll and Yossi (Benayoun) are all of great talent in their own positions and all of a different talent that makes us a much better team and a much better squad.
"Hopefully when they get the opportunity to play they will show those talents from the very first opportunity they get.
"We have got Matt Jarvis who is the provider of more crosses than anyone else in the Premier League during his time at Wolves.
"We've got Yossi Benayoun with his quick feet and his ability to beat players in and around the final third - and we have got Andy Carroll who plays up front and in his own right is a fantastic centre-forward through his sheer talent, determination and his size.
"Those three players will make our side look good for the rest of this season."
Although the transfer window has now closed Allardyce also confirmed that he has three trialists currently training with the first team.
Experienced defender Mikael Silvestre, Ghana international John Mensah and former Portsmouth utility player Anthony Vanden Borre are all being given a chance to earn deals at the club, although Allardyce has yet to make up his mind on any of them.
"They have been training with us but there is no decision on whether we will be pursuing their services or not at this moment in time," he said.