The 50-year-old takes over from Didi Constantini, who resigned last month after failing to guide the team to next year's Euro 2012 finals in Poland and Ukraine.
Koller, who won 55 caps for Switzerland during a 17-year playing career spent exclusively at Grasshopper Zurich, has put pen to paper on a two-year deal with the Austrian Football Association (OFB), with his brief to guide the country to the 2014 World Cup in Brazil.
He will take charge of the national team on November 1, meaning interim boss Willi Ruttensteiner will remain at the helm for Austria's final Euro 2012 qualifiers against Kazakhstan on Friday and Azerbaijan next Tuesday.
Koller, whose first match in charge will be a friendly against Ukraine on November 15, told a press conference: "I am a coach who wants to win the games, wants to be successful.
"The talks [with the OFB] went very well, I've looked at a few games of the Austrian national team and in those I noticed that the team is playing very good football.
"The potential is there and I am convinced that together we can go in a good direction."
Koller has extensive coaching experience having had spells in charge of St Gallen, Grasshopper, Cologne and Bochum since ending his playing days in 1996, and said he will work closely with players' clubs to maximise results.
"My goal is to work with all stakeholders, be it player, coach or manager, to keep in close contact," he said.
"If we want to reach the World Cup, we have to fit many small mosaic tiles.
"It can only work if everyone pulls together."
Constantini had been due to leave his post at the end of the year having announced his resignation after Austria's Euro 2012 hopes were extinguished, but last month told OFB president Leo Windtner that he would stand down with immediate effect, leaving Ruttensteiner in temporary control.
Austria's hopes of making next year's tournament disappeared when they were hammered 6-2 by Germany and drew 0-0 at home to Turkey last month.
They sit fourth in Group A on eight points from eight games heading into their final two qualifiers.