Rossi found the net with eight minutes left after a match in which the hosts enjoyed a string of chances, particularly in the first half but paid for their inability to beat Norberto Neto.
Sassuolo, trying to find a way to climb out of the bottom three before the turn of the year, piled forward in the early going in an encouraging start.
Domenico Berardi had the first sight of goal just five minutes in but his shot was easily saved by the goalkeeper.
Simone Zaza then had two chances but could not keep either down, heading over from a corner before sending a shot from the edge of the area into the stands.
Fiorentina did not get a sniff until the 33rd minute but Gianluca Pegolo was equal to Josip Illicic's effort and the action soon returned to the other end.
Zaza had another great chance from a corner but could not find the target with his close-range header, and Marvin Compper then got in a crucial block as a goalmouth scramble ensued.
Zaza carved out another chance five minutes before the break when he weaved his way past Stefan Savic but shot straight at Pegolo.
Fiorentina improved after the break as they took the ascendancy, but it looked like there would be no reward as Rossi and Ilicic squandered chances before Pegolo made a fantastic save to keep out Massimo Ambrosini's header.
But with time ticking down, Rossi fired home from the edge of the area, finding the corner of the net to beat the goalkeeper.