Four-time champions Italy, two-time winners Uruguay and dark horses Costa Rica will join the Three Lions in Group D in Brazil.
Not only that, Roy Hodgson's side will have to travel thousands of miles and play in the sweltering heat of the jungle as they bid to make it through to the knockout stages.
Football Association chairman Greg Dyke made his thoughts clear by sliding his index finger across his throat after the draw, but Milner does not feel England are doomed.
In fact, the Manchester City midfielder believes that Group D could actually benefit the team going forwards.
"It's not easy, but it's a World Cup and everybody has earned the right to be there," Milner said.
"They are all good teams, and if you go there with a so-called easier group, you might not come out of the box as fast as you need to.
"We know we have to be on top of our game in the first game of the group, and that could be an advantage, hitting top gear straight away, and then carry through the tournament like that. It could work in our favour."
Before Milner heads to South America, though, there is the small matter of helping City to success.
Manuel Pellegrini's side have been guilty of Jekyll and Hyde displays so far this term, with a 100 per cent home record but just two wins to show from eight away matches.
The latest frustration came in Southampton on Saturday, where City were held to a 1-1 draw after a poor performance.
"Looking at overall form away from home, I don't think it has been too different," Milner said.
"The results have been different, but in the majority of games we have played away, the way we have dominated possession and deserved to win the game is similar.
"But we have somehow ended up losing games through mistakes and missed chances. Saturday was one of the exceptions, because we didn't play well and got a point out of it.
"We took the lead and that is a big thing, but it's disappointing as we started the better, but after the goal we took our foot off the gas, and didn't stay on top of them."
For injury-hit Southampton, the 1-1 draw with City helped stopped the rot.
Having suffered three successive defeats for the first time since Mauricio Pochettino's arrival, Saints impressed on Saturday despite missing seven first-team players through injury.
"It's really important," Croatia international Dejan Lovren said of the draw. "I think the most important thing now was keeping our game and keeping our style.
"We didn't panic and I think that's the important thing, to keep our style, to have the confidence.
"Okay, it was a little bit difficult after three games we didn't win, but we were showing the character and I think we have a good squad, a positive squad.
"Everybody is in the head prepared for everything, so we need just to believe and to play our game."