Martin O'Neill's side looked to have turned around a first leg deficit after a James Milner penalty and John Carew's second half goal had put them ahead on aggregate.
But Jelavic, who had scored for Vienna in the first encounter, had the final say with 14 minutes remaining.
It will be a massive disappointment to O'Neill and his players and will also hit the club financially via the loss of three guaranteed home fixtures and television revenue.
They were left to rue a first half spot-kick blunder by Ashley Young whose effort was saved by Vienna 'keeper Helge Payer.
It would be harsh to criticise Young. He and Milner were two of Villa's best players and never stopped working from the first minute to the final whistle.
But Vienna were well organised throughout, defending in depth when required but also posing a threat on the counter-attack.
Vienna looked comfortable on the ball and Stefan Hoffmann played in Christopher Drazan behind the Villa defence and Carlos Cuellar needed to be alert to intercept his low centre.
Villa retaliated and Emile Heskey was inches away from connecting with a fiercely driven cross from Milner after he had seized onto a flick on from John Carew.
The visitors could have taken the lead had Jelavic's pass to the unmarked Hoffmann been more accurate after a slick move.
Cuellar became the first player to be yellow carded after 20 minutes for a foul on Hoffmann.
The Vienna skipper took the free-kick himself and curled a 20 yarder only a couple of feet wide of Brad Friedel's right hand post.
Villa started to build up some momentum but after 31 minutes Young was denied from the penalty spot by Payer.
He created the opportunity himself after initially being fouled outside the box by Dober, who was subsequently booked for the challenge, and then brought down inside the area by Markus Heikkinen.
But Young drilled his penalty too close to Payer who was able to make quite a routine save to his right.
Young's frustration at the miss instantly got the better of him as he was yellow-carded - his third of the season - for bringing down Yasin Pehlivan.
Villa were in danger of losing their cool and Carew quickly followed Young in being cautioned for a challenge on Dober.
But they drew level after 38 minutes when Young earned his side another penalty - and this time Milner made no mistake.
Young was sent sprawling by Dober and there was more conviction about Milner's spot-kick as he lashed it past Payer into the corner of the net.
Villa were indebted to Cuellar for keeping them on terms three minutes into the second half.
He had time to pick his spot 12 yards out and drilled in a fierce shot which the Spanish player cleared off the line with Brad Guzan beaten.
Villa hit back and Carew back-heeled the ball just wide from a low Heskey centre after good play by Milner.
But after 52 minutes the Norwegian international marked his return to the side by giving Villa the lead.
Milner was again the instigator of the move after spotting Carew in space on the right and he cut inside before planting a low left-footed drive inside Payer's near post.
Milner then shot into the side netting after Payer had parried a shot from Young.
But then Jelavic, who had scored Rapid's goal in the first leg, somehow shot wide from three yards out after Guzan had parried
He made amends a minute later after Guzan had initially kept out a low shot from Katzer.
Payer turned over a dipping drive from Delph as Villa tried to respond to this setback. Hoffmann should have wrapped the game up but he shot over from six yards out with the goal at his mercy.