Mayweather's current record is 47-0 and many people think the Filipino southpaw might be the man to hand him his first professional defeat.
The two boxers’ camps have been in talks to make the fight happen, with both sides insisting they are keen to meet in the ring and that it isn't their camp behind the delay in naming a date.
De La Hoya said: "Mayweather will not fight, because he thinks that if he leaves the sport undefeated, (he will be considered the best ever). But he won't be," the boxer-turned-promoter stressed. "People will respect you more when you face the toughest, the strongest, the most dangerous."
He added: "What matters is the legacy we're leaving behind. In my day, during (Julio Cesar) Chavez's time, we saw the best fight with the best. I have that pride.
"I wanted to fight with the best, win or lose, but it was against the best. I cannot understand these things."