The Turin club were 3-0 winners when the two sides met in the league earlier this month, but Roma turned the tables on Tuesday night with a 1-0 win at the Stadio Olimpico.
Juve thought they had taken the lead a minute after the restart through a Federico Peluso header but the effort was ruled out for offside.
And the visitors were hit on the counter-attack 11 minutes from time, with substitute Miralem Pjanic racing the length of the field before laying the ball off to Kevin Strootman, who whipped in a cross for Gervinho to poke home the only goal from close range.
Roma will now play the winners of next week's quarter-final tie between Napoli and Lazio.
The other matches pit AC Milan against Udinese on Wednesday while Fiorentina take on Tuscan neighbours Siena on Thursday.