Riki's early goal settled the nerves inside the Riazor and the home fans had further reason for optimism just before the half-hour mark as Iago Aspas was sent off.
Silvio doubled the lead in the 64th minute as the league's bottom side looked to make their man advantage count and substitute Diogo Salomao, who had only come on a minute earlier, wrapped up the points in the 78th minute.
Park Chu-young pulled one back for Celta shortly after, but there was no sensational fightback.
Depor still have plenty of work to do to avoid the drop from the Primera Division, but the win at least took them to within three points of their opponents and six of safety.
It also ended a nine-match winless run in the league, seven of which have been losses.
Depor got off to the ideal start as Riki put them ahead after nine minutes, volleying into the top corner from Juan Carlos Valeron's ball in from the left.
Celta threatened when Mario Bermejo headed wide from Enrique de Lucas' corner, but they suffered another blow in the 29th minute.
Aspas saw red for a foul on Carlos Marchena, leaving the visitors a man light for the final hour.
The hosts looked to take advantage, Riki seeing a shot from the left of the area tipped round a post by Javi Varas.
But they could not relax, with Celta going close when Augusto Fernandez met a Danijel Pranjic free-kick, but sent his header wide.
Depor started the second half brightly and Riki shot weakly wide from inside the area before a minute later having another effort acrobatically saved by Varas.
And they got a brilliant second in the 64th minute, on-loan defender Silvio crashing home his first goal for the club from 30 yards.
It was almost three in the 75th minute, but Varas was again alert to keep out a Bruno Gama effort from close range.
A third goal did arrive three minutes later, though, as substitute Salomao fired home a low shot from Valeron's pass.
Park pulled one back for Celta two minutes after that, tapping home Michael Krohn-Delhi's cross, but it was too little too late.