Pioli was dismissed following Monday's 2-0 defeat at Catania, Bologna's third loss in four Serie A games, with Ballardini coming in on a deal until the end of the season.
A statement from Bologna read: "Bologna announces it has appointed Davide Ballardini as the new coach of the first team. Ballardini has signed a contract until June 2014."
Ballardini had been out of a job since being dismissed by Genoa in the summer.
The 50-year-old inherits a team that sits 17th in the standings, one point and one place clear of the relegation zone.
Ballardini has plenty of experience having coached Cagliari, Palermo, Lazio and Genoa in Serie A.
He will take charge of his first game in Saturday's match against Lazio.