The highly-rated forward slotted home seven minutes from time to cancel out Oscar De Marcos' opener for Athletic which came against the run of play just before half-time.
The result sees Celta join 17th-placed Deportivo La Coruna on 31 points while Athletic, who hit the woodwork twice in the game but were also indebted to a superb display from goalkeeper Gorka Iraizoz, remain seven points better off in 14th.
Athletic enjoyed the better of the early exchanges and came within a whisker of taking the lead in the 11th minute when Ander Iturraspe crashed a header against the bar from a Markel Susaeta corner.
However, after that Celta took the game by the scruff of the neck and only the brilliance of Iraizoz kept the hosts at bay.
The 32-year-old custodian's first serious involvement did not arrive until the 26th minute when he was tested by Aspas, but he was back in the action barely 60 seconds later to tip over an Alex Lopez free-kick.
Five minutes later the former Espanyol player did well to push Michael Krohn-Delhi's 20-yard drive around the post before producing more heroics in the 35th minute to parry a Lopez header.
Still Celta continued to push and Aspas drew another save from the overworked Iraizoz in the 38th minute while Augusto Fernandez came close four minutes before the break with a shot that just went wide.
Then, out of the blue, Athletic hit their opponents with a devastating sucker-punch as De Marcos put them ahead, the forward latching on to a pass from Aritz Aduriz before firing into the roof of the net.
That was a cruel blow for Celta but they came out in the second half with renewed determination - only to find Iraizoz blocking their path again as he produced another fine save to keep out Fernandez.
Celta were indebted to their own custodian for keeping the deficit at one in the 52nd minute, though, as Javi Varas brilliantly denied Aduriz from close range.
Athletic looked to have weathered the Celta storm and they almost made it 2-0 in the 65th minute with a Ibai Gomez shot that hit the bar.
However, with time slipping away, Celta finally managed to find a way past Iraizoz as Aspas fired home left-footed from 12 yards out to snatch a potentially valuable point.