The England international has signed a four-year contract after the two clubs agreed an undisclosed fee.
O'Neill has been on the Easington-born midfielder's trail for several weeks after it became apparent that his days at the Etihad Stadium were drawing to a close, and his arrival on Wearside just hours after Wolves striker Steven Fletcher capped a good day for the Ulsterman.
A delighted O'Neill said: "Adding quality players to the squad has been our main aim this summer and Adam certainly fits that bill.
"He has terrific ability, great delivery and I'm sure he is a player who will excite our fans. I couldn't be more delighted to have him at the football club."
Johnson, 25, is a product of Middlesbrough's Academy and has won 12 full England caps to date.
However, he missed out on a trip to Euro 2012 and will hope his latest move could help to revive his international career.
Johnson played 96 times for first club Middlesbrough, scoring 13 goals, before he moved on to City in 2010.
But after 73 games, which produced 11 goals, he has now moved to Sunderland.