Sion suffered a 5-0 defeat at St Gallen this weekend and have taken only 10 points from their last 11 games under Gattuso, who became player-coach in February.
A club statement on Monday said he had been replaced by the club's under-21 management team, comprised of Michel Decastel, Christophe Moulin and Fabian Salvi.
Gattuso, who is sidelined through injury, will remain at the club as a player.
The club have also demoted eight first-team players to the reserves and replaced them with youth players.
"(Management) hope the club's youth, along with the regular staff, can seize their chance and their desire and willingness will give FC Sion the new impetus they currently need," a club statement said.
Decastel will take charge for a second time this season. He replaced the sacked Sebastian Fournier in September before the appointments of Pierre-Andre Schuerman, Victor Munoz and then Gattuso.
Sion are fifth in the Swiss Super League and remain in the running for a Europa League place.