The game will be Ferdinand's first at Stamford Bridge since accusations he suffered racial abuse from Chelsea skipper John Terry last October, who now faces a court case in July to answer those allegations. Terry denies the charges.
Ferdinand, 27, said he was more interested in securing a positive result as QPR look to remain in the top flight, rather than listen to any jeers from the home crowd.
"There is a big rivalry between the two clubs anyway but, with everything that has happened, I know there will be some fans targeting me," he told London Evening Standard Sport.
"The Chelsea fans believe what they want to believe. They have their own view on what's gone on and if they are going to give me stick, that's their problem.
"All I can say is that it won't upset my game. If they think that's what's going to happen, then they have got another thing coming."
The Football Association cancelled the usual pre-match handshakes before the two sides met in the FA Cup back in January and the Premier League have taken the same measures ahead of Sunday afternoon's clash but Ferdinand does not see the pre-match ritual as an essential part of football and will treat the match the same as any other.
"My view is whether you shake hands with a player before or after the game doesn't really matter to me. It is about playing football," he said.
"My team-mates have said nothing to me about it because we are just treating it as a normal game. It is not about me or the current situation, it is about QPR and staying in the division.
"It is not for me to talk endlessly about handshakes - what matters is what happens in the course of 90 minutes."
Manager Mark Hughes confirmed this afternoon [Friday] Ferdinand had been instructed by his lawyers not to take part in any handshake.
He said: "We knew that Anton had been told that he wouldn't be allowed to shake hands so we understood that part of it and obviously discussions between ourselves, Chelsea and the Premier League were ongoing but I think they have come up with the right decision.
"From our point of view we have just been focusing on preparing for a very important game and the wind has been taken out of people's sails regarding the handshake issue."
He added: "That is what we were conscious of before the cup game.
"The game itself was a little bit of a damp squib because of what occurred before the actual game with the discussions and debate about who is going to shake who else's hand so the game itself was like a side issue.
"I don't think that is correct and I'm glad the decision is out of the way now."
Chelsea boss Roberto Di Matteo said Chelsea "respected" the Premier League's decision.
But he was in favour of the ritual in principle, adding: "To show respect to your opposition is a good thing, so I'm not generally against it."
Di Matteo revealed he would remind his players of their responsibilities ahead of a derby that was dogged by controversy in October's reverse fixture.
He said: "I always speak to my players. They will be focused on Sunday to try to win the game. The rest is for other people to worry about.
"We have to channel our energies to win the football match - that's all."
Di Matteo had no concerns about the state of mind of Terry, who was sent off in Chelsea's Champions League semi-final triumph in Barcelona, ruling him out of the final.
"I don't think that's going to influence him," Di Matteo said of his skipper, who was also abused by QPR fans in January's cup tie.
"He's got a few years under his belt now and has dealt with setbacks before. He'll be very good."