The Gunners head to Germany trailing 2-0 from the first encounter at the Emirates Stadium and Graham believes they are incapable of overcoming that deficit against the defending champions.
Once Arsenal have played Bayern they face Tottenham, Chelsea and Manchester City in a key phase of their Barclays Premier League title challenge.
"One of the questions facing Arsene is whether they play their best team and try to win the tie, which is near impossible, or concentrate on the league," Graham told BBC Radio Five Live's Sportsweek programme.
"Personally I'd play a weakened team because I think the second half of the first leg was like a practice match for Bayern.
"It took a lot out of Arsenal and they have some very tough games coming up. They need fresh legs and fresh minds.
"This is one of the big decisions managers have to make."
Arsenal playmaker Mesut Ozil missed a first-half penalty at the Emirates Stadium amid the ongoing deterioration of his strong early season form.
"When he first came into the team he was outstanding and gave a lift to the rest of the players," Graham said.
"But his form has dipped very much and I think the pace and intensity of the English game have caught up with him. He looks sad now and mentally he needs to come back."