The Rangers, 0-3-1 on a nine-game home series, closed within 3-2 in the third period, but Ryan Johansen pushed Columbus' lead back to two goals with 1:32 left.
Blue Jackets goalies Curtis McElhinney and Mike McKenna, who are subbing for injured reigning Vezina Trophy winner Sergei Bobrovsky, stopped 32 of 34 shots.
Columbus (14-15-3) made it two wins in a row.
Dominic Moore and defenseman Dan Girardi scored for New York (15-16-1). Henrik Lundqvist was pulled after he stopped only 10 of 13 shots in the first period.
Philadelphia Flyers edged Montreal Canadiens 2-1 as Michael Raffl and Claude Giroux scored goals and Steve Mason made 20 saves.
Craig Anderson stopped 40 shots as the Ottawa Senators won 2-1 over the Buffalo Sabres.
Bobby Ryan and Zack Smith scored for the Senators (13-14-6), Tyler Ennis replying for the Sabres.
Martin St. Louis scored in the sixth round of the shootout as Tampa Bay Lightning beat the Detroit Red Wings 2-1.
David Backes scored twice and Jaden Schwartz added a goal and two assists as the St. Louis Blues won 6-3 against the Toronto Maple Leafs.
The Nashville Predators were 3-1 winners against the Dallas Stars with David Legwand netting a goal and adding an assist.
The Colorado Avalanche ended a two-game losing streak with a 4-3 victory over the Winnipeg Jets, Matt Duchene and P.A. Parenteau scoring in the shootout.
Calgary edged Carolina 2-1 in overtime, Phoenix beat the New York Islanders 6-3, Boston were 4-2 winners over Edmonton and San Jose defeated Minnesota 3-1.