The 26-year-old appeared to stamp on Chelsea midfielder Juan Mata towards the end of the first half of the game at the Stadium of Light, which the visitors eventually won 2-1.
Referee Lee Probert failed to punish the full-back at the time, but after reviewing video evidence the FA have decided to take disciplinary action.
If found guilty, Bardsley faces a three-match suspension.
An FA statement read: "The FA has charged Sunderland defender Phil Bardsley with violent conduct following an incident in his side's home game against Chelsea.
"Bardsley has until 6pm on Wednesday 14 September to respond to the charge."