The former England striker recently re-signed for the Hammers on a short-term deal, having previously been released over the summer following seven years at Upton Park, and he came on as a substitute in Sunday's 0-0 draw at Swansea.
Boss Sam Allardyce brought the 29-year-old back to the club having failed to land another forward before the transfer window closed and with big-money signing Andy Carroll yet to figure this season due to a foot injury.
Allardyce has since lost Ricardo Vaz Te for the foreseeable future after he dislocated his shoulder at Swansea, giving Cole the chance to lead the attack against the Sky Bet Championship leaders at Turf Moor.
"The next game we've got is Burnley and hopefully we can get goals there," he told West Ham TV.
"Everyone knows I need minutes for match fitness. Sunday was a good start on my way to recovery. I'm now hoping to get the minutes and the goals to get us through to the next round if I can.
"It was really good to be out there again. I was disappointed we didn't win but we were away from home so you can't ask for much more than getting a clean sheet.
"The boys have been doing well and we've got to give credit to them for only conceding one goal away from home so far this season. It's a huge credit to all the playing staff and the management as well.
"I'm chuffed to get back and to be involved in it again. Any footballer will tell you that have to work hard and the manager has noticed that I was working hard and has given me a new deal.
"Hopefully I can push on from here and add to the team and play a significant role."
Another player eager to impress is Jack Collison, who will be in the squad after returning from a month on loan at Bournemouth.
The 25-year-old midfielder, who has been plagued with knee problems throughout his career, has not ruled out going back to Dean Court if he cannot secure a regular spot with the Hammers.
He told the Daily Echo: "I will be looking to play in the League Cup tie and then see where I am after that. I would like to force my way into the West Ham team but if it's not to be, it's not to be.
"I have had a fantastic time here and would definitely be open to coming back and trying to improve on what I did."