The United States international's goals helped the Cottagers triumph in a London derby in which the npower League One leaders' performance belied the scoreline.
Dempsey gave the Cottagers an eighth minute lead after connecting with a deflected through ball from former Charlton midfielder Danny Murphy to flick home.
Boosted by up to 7,000 visiting fans, Charlton attacked with intent and only fine goalkeeping from David Stockdale prevented them netting an equaliser.
Yann Kermorgant, Danny Green and Bradley Wright-Phillips were all denied by the England man, before Dempsey netted in the 61st minute.
The 28-year-old secured progression to the fourth round with nine minutes remaining, converting from the penalty spot before substitute Damien Duff scored a late deflected effort.
Fulham benefited from a strong line-up at Craven Cottage as manager Martin Jol stuck by his word not to name a weakened side, with Kerim Frei the only change to the side that defeated Arsenal on Monday.
Counterpart Chris Powell kept largely the same side that defeated Brentford earlier this week, with John Sullivan and Matthew Taylor drafted in.
The Addicks started brightest by the Thames and only a late block prevented Danny Hollands' header breaking the deadlock inside four minutes.
Fulham were under a lot of pressure during the opening exchanges but soon benefited from their attack-minded philosophy.
Moussa Dembele forced Sullivan into a superb one-handed save in the sixth minute and the Cottagers took the lead two minutes later.
Murphy, who was receiving a lot of stick from the travelling support, saw a through ball deflected into the path of Dempsey, who flicked home.
The goal did not dampen Fulham's enthusiasm as they continued to press forward, with Frei and John Arne Riise looking particularly dangerous down the left flank.
The latter saw a 20-yard effort blocked by Taylor, before Charlton pressed forward in search of an equaliser.
After some neat interplay Kermorgant hit a neat curled chip that forced Stockdale to parry wide, although Johnnie Jackson could only fire the rebound into the side-netting.
Charlton were looking increasingly dangerous as the half progressed and a stinging Wright-Phillips drive forced Stockdale to save with his feet.
Green was unfortunate to not see anybody convert a superb cutback as the visitors continued to press, before receiving the afternoon's first booking for clipping the heels of the ever-dangerous Frei.
Riise and Dempsey flashed shots wide as half-time approached, before Bobby Zamora saw a weak header saved by Sullivan.
The second period burst into life after a slow opening in the 54th minute when Sullivan tipped wide a 20-yard Murphy effort.
Dempsey headed over from the resulting corner, before Stockdale twice thwarted Charlton in quick succession at the other end.
The England man produced a superb reaction save to deny Wright-Phillips from close range, before tipping over Green's follow-up from outside the box.
Jol soon brought on Duff for Frei in a bid to change things and only a low Sullivan save stopped the Republic of Ireland international having an immediate impact.
The home side did not have to wait long for a second goal, though, as Dempsey netted his in the 61st minute.
The Texan midfielder played a one-two with Costa Rica international Bryan Ruiz, before directing the ball past Sullivan.
Charlton seemed to go into their shell after Dempsey's goal, with Ruiz, Zamora and Philippe Senderos all coming close in the 10 minutes that followed.
Wright-Phillips and Kermorgant saw efforts blocked as Charlton briefly threatened a revival, before the home side put the tie out of reach.
Ruiz was brought down in the box by the onrushing Sullivan, allowing Dempsey to tuck away the resulting penalty and claim the match ball.
The majority of the 20,317 fans inside Craven Cottage had another goal to celebrate with three minutes remaining as Duff saw an effort deflect in off Taylor.