The promoted side put their agonising midweek defeat to Real Madrid behind them to record a top-flight victory for the first time since they last appeared in La Liga back in the 1988-89 season.
Marquez was the match-winner with his precise 62nd-minute finish helping Elche double their points tally for the campaign and inflict a third defeat in four games on Celta.
Elche thought they had claimed a memorable 1-1 draw against mighty Madrid on Wednesday when Richmond Boakye levelled in stoppage time, but a controversially-awarded penalty converted by Cristiano Ronaldo minutes later left the Alicante side empty-handed.
That was a demoralising blow for Elche, whose coach Fran Escriba earned a one-match touchline ban for his complaints after the late defeat, but the responded in fine fashion on Sunday.
They almost took a second-minute lead through Manu Del Moral only for Celta goalkeeper Yoel Rodriguez to tip the shot wide.
Yoel was also called on to save from Marquez and Gil later in the half before the latter struck the post just before half-time as Elche pushed for the opener.
Celta, without a home win since Luis Enrique took the reins this summer, also posed an attacking threat in the opening period and Elche stopper Manu Herrera needed to be alert to save from Charles, Alex Lopez and Augusto Fernandez.
The chances dried up at the start of the second half but the first opportunity after the break saw Elche net what would prove to be the winning goal, Marquez racing into the area before drilling a low shot across Yoel and in off the base of the far post.
Celta could find no way back after that as Elche's defence held firm to seal a long-awaited win.