Graves, who was appointed ECB chairman in March, said he fully expected England to beat the West Indies, a side he described as ???mediocre???.
The teams drew the first Test in Antigua after the tourists failed to bowl their opponents out and begin the second Test in Grenada this week. The three-Test series wraps up in Barbados on May 1.
The West Indies will again be without several key players due to Indian Premier League duty, notably big-hitting batsman Chris Gayle.
But Atherton feels they still represent a solid challenge to England and that Graves??? remarks will only put more pressure on Alastair Cook???s side.
???I think they are unnecessary comments,??? said Atherton, who captained England in 54 Test matches between 1993 and 2001. ???International sportsmen are under enough pressure as it is anyway. Why give the opposition a motivational tool?
???The West Indies have been very aware of the comments that have been made about them. They are not the best Test team by any stretch of the imagination but nor are they pushovers in their own region.
???They have won seven out of the last 10 Test matches they have played here, they have a new coach and their work ethic is definitely rising.
???They have a seam attack that is decent, areas of weakness certainly in their batting but England will have to play well to beat them.???
Atherton has also stressed the need for the team???s opening batsmen to get themselves among the runs.
Cook has not scored a century in his past 33 Test innings and managed just 11 and 13 in the first Test against the West Indies.
The majority of the scoring was made by the middle order, with Ian Bell (143) and Joe Root (83) anchoring the first innings and Gary Ballance (122) leading the way in the second.
???It is important because as England go through to the summer against tougher opponents, New Zealand and Australia, one of the key components of any successful team is an opening partnership,??? said Atherton.
???You want it to be solid, settle and productive so that gives the middle order breathing space.
???The middle order is ticking along very nicely at the moment with Balance, Root and Bell all getting more than a hundred runs in the game and they just need the opening partnership to fire.???