Odemwingie, who scored both goals in Monday night's 2-0 home win over Southampton, has resolved his differences with Nigerian coach Stephen Keshi and set his sights on appearing in the tournament, which begins in January.
Midfielder Mulumbu is also set to be absent from the Baggies when the competition gets under way due to Congo's qualification.
But Odemwingie feels the Baggies can cope with the absence of two of their most influential players as they look to build on their fine start to the Barclays Premier League campaign.
He said: "It is a different situation now to maybe my first two years here when my goals were very important.
"Now any player can play well and we have good players who can play for a whole month at Albion and they will do well without me.
"Youssouf is also a very important player for us in the middle but we have good numbers of players. We have Yassine El Ghanassy who is still waiting his chance to show something from one of the wings.
"We have (Chris) Brunt, (James) Morrison, (George) Thorne. We have players who can play at this level and Albion took into consideration when recruiting players that a few of us might go (to the African Nations Cup).
"I know it's going to be a bit of a loss, us leaving for a few weeks, but we hope the players there can follow the team's performance."
Odemwingie is relishing the prospect of trying to help his country to glory and recognises it may be his last stab at winning the tournament.
He said: "Will I play in the African Nations Cup? Yes, yes. I will put the country and the fans before anything if there are any disputes but I'm a very straightforward person.
"If things are not okay I don't pretend and say things are fine, but definitely I'll put our football fans first in any decisions.
"I know I will miss a chunk of Albion's season but it is hard not to go to this African Nations Cup. It could be my last."