Caulker met Craig Bellamy's corner to decide the match and lift the Bluebirds ahead of their opponents in the table.
The Swans pushed for an equaliser, but could not find a way through and had goalkeeper Michel Vorm sent off at the death for bringing down Fraizer
Campbell when the striker was through on goal.
Caulker, who spent the 2011/12 season on long at Swansea, told Sky Sports 1: "It was really intense, especially the last 10 minutes.
"I thought the boys did well and stuck together, it's a great win.
"It's a big three points and we'll go again against Aston Villa on Saturday."
Bellamy added: "It's always a hard-fought game and today was no exception.
"Next month we got really tough fixtures ahead of us. We really needed three points."
Michael Laudrup was disappointed to lose the game on a set-piece.
He said: "It was a very tight game, we knew it would be. You can't give Cardiff the chance with set-pieces.
"One set-piece, a corner, and they scored a goal. That was more or less all the chances they had in the whole game.
"It hurts to lose a derby, but that's how it is."
The Swans boss felt his side were in control of the game before the goal.
He said: "I think we dominated well in the first half in midfield. We only had a couple of chances, but we kept them away from any chances.
"My thought was during the second half to bring in one or two more offensive players and then they scored."
The Dane accepted Vorm deserved to see red, adding: "He's last man, so it's what can happen."
Cardiff boss Malky Mackay's position has been in the spotlight in recent weeks because of new owner Vincent Tan's decision to get rid of his head of recruitment Iain Moody.
Asked if this win had eased the pressure on him, he said: "There's pressure on me every week, every Saturday. It's a tough uncompromising league, but it's the only place to be."
But he admitted the victory would have "put a lot of smiles on people's faces".
He added: "It's a good day all round, obviously a hugely important day locally and also as far as we're concerned.
"Tactically we played very well. I thought to a man we were excellent in that regard and exposed them as well."Former Swansea defender Steven Caulker hailed a "great win" after his header earned Cardiff a 1-0 victory in the Barclays Premier League's first south Wales derby on Sunday.