Eremenko converted from the spot in the 75th minute after David Targamadze was adjudged to have handled and any thoughts of a Georgia comeback were effectively ended when Jaba Kankava was sent off for a second bookable offence moments later.
It left Georgia decisively eliminated and Finland needing France to slip up in Belarus to have a chance of claiming second spot in Group I
Kasper Hamalainen missed from close range for Finland in an uneventful first half, while at the other end Akaki Khubutia had a shot blocked by Juhani Ojala after Niki Maenpaa had fumbled Zhano Ananidze's cross.
When Tornike Okriashvili volleyed over from six yards for the hosts midway through the second half, the game seemed destined to finish goalless.
But Kankava fouled Eremenko 20 yards from goal, earning a booking, and when Eremenko's shot hit Targamadze's hand in the wall, the referee pointed to the spot.
Eremenko himself stepped up, sending the ball into the corner, and was promptly fouled by Kankava again, drawing the red card from the referee's pocket and effectively ending the contest.