The Dutchman announced his decision immediately after Saturday's 1-0 win over Hamburg at the Augsburg Arena.
"We deeply regret this step," said Augsburg's president Walther Seinsch at a press conference.
"Unfortunately, we were unable to fulfil our wish and our efforts to keep him at Augsburg, even in spite of a fantastic year with the avoidance of relegation.
"Jos Luhukay informed us in mid-March that he was doubting that we would be able to reach our goals again next season and he was no longer convinced about this.
"He informed us of his final decision to leave Augsburg yesterday [Friday]."
The Dutch coach led the Bavarian club into the top flight for the first time in their history last season and, despite needing nine games to register their first win, he still managed to leave them to safety with two games to spare.
"This decision was incredibly difficult for me," he said. "Maybe it was the most difficult one of my entire coaching career.
"I had three wonderful and successful years with FC Augsburg and I would like to thank my assistant coaches, my team and the whole management and colleagues at the club for their support.
"We have achieved something sensational in these three years and laid the foundations for a successful future for Augsburg."
Luhukay took charge of Augsburg in April 2009 and led them to the semi-final of the DFB-Pokal in his first season in charge.
They only just missed out on promotion to the top flight that year when they were beaten by Bavarian rivals Nurnberg in a play-off for promotion, but he succeeded in taking them up last season.
Avoiding relegation, despite having the lowest budget of all Bundesliga clubs, was a remarkable achievement and one Seinsch wanted to thank him for.
"The amazing successes in the last three years are firmly attached to his name," he said.
"We wish him all the very best and hope to see him again in the Bundesliga. Jos Luhukay is always welcome back at Augsburg."