Second half strikes from Gareth McAuley and substitute Romelu Lukaku sealed the three points and - following their 3-0 win at the Hawthorns on the opening day of the season - a first double over Reds since the 1966-67 season.
Foster saved a spot kick from Steven Gerrard when the scores were level, after Jonas Olsson was adjudged to have fouled Luis Suarez. The miss proved to be crucial as the Baggies won for the first time in eight games.
"We needed that," Foster told Sky Sports 1. "We have been on a poor run, but the performances have been decent.
"We knew we had a chance if we kept our performances up. We always knew set pieces would be a big one for us and we managed to get one."
The win takes West Brom above Liverpool and up to eighth in the table with 37 points.
"We have a tough game against Sunderland who themselves are on a decent run, we just have to take it a game at a time," Foster added.
"We got off to a great start to the season and we are within touching distance of 40.
"If you had said at this stage of the season we would be above Liverpool, we would have taken that."
Head coach Steve Clarke was delighted his side could register their first league win of the year.
"For a change something went right and it was good to come here and play well. It's a terrific win for us.," he said.
"Since the turn of the year things haven't gone our way. Some of the performances have been decent, but to come here to play a Liverpool team playing as well as they have all season and to get that kind of result and with that level of performance - I'm delighted.
"They put so much effort into every game they play and to get that reward is really great for them."
Clarke believes the penalty save was the turning point and his side gained confidence from Foster's save.
"The players realised at 0-0, and with Ben saving the penalty, that we could go on and win the game," he added.
"It gave us a bit of positive momentum again.
"We have always been one result from kick-starting our season and hopefully that came tonight, and hopefully we can kick on and chase the teams above us."