The 18-year-old £10million summer signing from Southampton had opened his Arsenal account in last week's Carling Cup win over Shrewsbury and produced another impressive display when firing the hosts in front after just eight minutes.
Brazil defender Andre Santos' first goal for his new club put Arsenal well in command, only for David Fuster to power in an unmarked header before the break.
The Gunners - who had manager Arsene Wenger serving the final match of his European touchline ban and started with captain Robin van Persie on the bench - were nervy at times, as Olympiacos captain Vassilis Torosidis crashed a curling 25-yard effort against the bar, but it was the livewire performance of Oxlade-Chamberlain which caught the eye.
Rice believes the youngster can follow in the footsteps of England winger Theo Walcott, out injured, to play a big role for both club and country in the years ahead.
"From Arsenal supporters' point of view, they are going to be seeing a lot of this boy," Rice said.
"He has a big, big challenge to now get in front of Theo. I know that Theo is a very strong-willed guy as well and he won't give in easy. It all bodes well for England anyway."
Rice added: "Alex can go inside, he can go outside, he's got that injection of pace and I think what he needs now is to be consistent in his play.
"I am sure that is something he will be working on because he's certainly not a stupid boy.
"He has good people around him and they are telling him all the right things.
"Alex is very friendly with Theo and no doubt he will give him the benefit of his experience as well."
Oxlade-Chamberlain was playing for Southampton in League One last season and enjoyed his first taste of top-flight European action.
He said on Sky Sports 2: "It was a really good experience.
"It's a big step up, but I want to play at the top level and I have plenty more time to play in the Champions League."
Rice admitted Arsenal had been somewhat sloppy in the way they allowed Olympiacos back into the match.
"You can't turn around and say we had all of the game because that would be unfair," he said.
"The first 20-25 minutes we were really in it well, but after that for about 15-20 minutes we maybe lost our way and they came back into it. From a defender's point of view, the goal was really poor.
"Second half we had to knuckle in, fight and get our shape right.
"It is unfortunate we didn't get the third, but we also had opportunities on breakaways in the second half.
"They came, they waited for us, they broke us up in the middle of the park and in fairness they did well.
In the end, though, Arsenal did just enough to secure their second victory of the week and restored some much-needed confidence ahead of what is sure to be a testing trip to arch rivals Tottenham on Sunday.
"Needless to say we will be very, very happy with the three points," Rice added.
"That is a completely different ball game at Tottenham
"The most important thing for us was to win, because it takes us closer to qualifying.
"We will think more about the Tottenham game when we all get together again on Friday."
Olympiacos had lost their opening European tie at home to Marseille, who now top Group F after an impressive victory over German champions Borussia Dortmund.
Coach Ernesto Valverde was left to rue his side's sloppy opening.
"When you have let in two goals so early, that has effected us and left a bitter taste," he said.
"We were a bit disappointed because we tried to takes something out of the game.
"After we conceded the early goals, we performed quite well.
"However, when you face a team like Arsenal, you have to exploit all the opportunities you have to score, because they have the quality players to convert all their opportunities to goals."