Palace beat an inexperienced Manchester United team 2-1 after extra-time at Old Trafford on Wednesday night to reach the last four while Cardiff had beaten Blackburn 24 hours earlier.
The Londoners will play the first leg at home before going to Wales for the return and the victors will face the winners of Manchester City versus Liverpool.
The ties will be played in the weeks commencing January 9 and January 23.
Eagles boss Dougie Freedman told Sky Sports 1: "It's going to be a very difficult tie for us.
"Cardiff are an exceptional team with good quality players and it's probably made it a harder draw for us."
Carling Cup semi-final draw
Manchester City v Liverpool
Crystal Palace v Cardiff City