Celta striker Charles Dias opened the scoring in the first half, but his effort was quickly cancelled out by Mikel San Jose. Andoni Iraola put the hosts in front on 61 minutes and Benat Etxebarria looked to have made sure of the victory eight minutes later, although Santi Mina made for an exciting finish by pulling one back for Celta with 12 minutes remaining.
Athletic went out keen to make their mark on this historic occasion and took the initiative, enjoying 60 per cent of the possession over the 90 minutes, while Luis Enrique's side played on the counterattack.
Iraola had two chances to score within the first three minutes, the first a volley which was comfortably saved by Celta goalkeeper Yoel Rodriguez, the second a miscued attempt at the far post following a cross from Markel Susaeta.
Ander Herrera, playing his first match since his move to Manchester United on transfer deadline day fell through, then shaved the far post, as he tried to take full advantage of a slip from Andres Fontas.
Athletic thought they could have had a penalty when a free-kick by Benat struck the arm of Rafinha Alcantara, but the referee deemed it accidental. A minute later, the visitors took the lead thanks to an error from Erik Moran, who misplaced a pass just over the halfway line, and Celta's Rafinha took advantage of Athletic's high defensive line to race through alone, before squaring to Charles. The Brazilian kept a cool head to slot into the bottom corner, giving his side the lead and himself a place in the history books, the first player to score at the stadium.
Athletic reacted well to going behind and were level within four minutes. Benat sent a cross into the danger zone, which San Jose rose to meet but missed, but pounced on a mishit clearance by Hugo Mallo to shoot into the bottom corner.
At the other end Nolito could have given Celta the lead again, but his shot was easy for Iago Herrerin to gather.
The 25-year-old almost got his team in big trouble when he fumbled the ball at the feet of Augusto Fernandez, but the danger was cleared immediately.
The goalkeeper was then punished for his indecision, as he brought down Nolito in the area, and received a booking, while Celta were awarded a penalty. To Herrerin's relief, Charles sent the spot-kick over the bar and bouncing off the wall behind the goal where the final stand of the stadium is being constructed.
Charles had an opportunity to compensate for his miss four minutes before half time, with just Herrerin to beat, but put the ball wide as he tried to lift it over the goalkeeper.
Athletic took the lead just after the hour mark thanks to a superb interchange between Iraola and Iker Muniain. The fullback chipped a pass towards the forward on the edge of the box, Muniain controlled, waited and then returned the ball with a deft back heel into Iraola's path, and the 31-year-old put his shot away into the bottom corner.
Benat, who ran the show from midfield then got on the scoresheet, beating the offside trap to race down the right wing towards goal, turning a Celta defender and then shooting into the opposite corner with his left foot.
Herrerin then had another nightmare, failing to catch a straightforward cross, and substitute Mina gratefully headed in.
However, the home side did not become consumed by nerves towards the end of the game, and comfortably saw out the final 12 minutes.
They could have even scored again, Ibai Gomez missing a glaring chance in the closing stages, trying to bundle the ball home from three yards out following a knock down from Oscar de Marcos, and Fontas cleared it off the line.
Athletic moved up to fifth place in the Primera Division table thanks to the win, which was their third victory of the season, while Celta suffered their first defeat of the campaign and slipped down to eighth.