The home side made much of the running inside their own Mainz Arena but one break was all it took, together with a fortunate deflection, for Klaus to earn Furth all three points midway through the second half.
The Bavarians also had their goalkeeper Max Grun to thank for preserving the points on a historic night for Mike Buskens' side.
Greuther Furth started the season with a 3-0 defeat to Bayern Munich and although their coach Mike Buskens said his side would not be downbeat because of it, the nerves were evident in a cautious approach to the match.
Like they did six days ago, they left just one striker up front and packed the defence and midfield, frustrating the home side who, despite their territorial superiority, found it hard to get through.
Furth were not completely negative, though, and they almost took the lead with their first counter-attack of the game in the 12th minute when Djiby Fall struck a loose ball wide of the post.
Adam Szalai got the hosts' first shot on target two minutes later, but Grun did well to palm the ball away.
Eric-Maxim Choupo-Moting missed Mainz' best chance of the first half in the 21st minute when he managed to get the ball round Grun but could only find the side-netting with his shot.
Choupo-Moting looked the most likely source of a goal and he went close early in the second half when he headed just over the bar.
Mainz pushed forward, sensing a goal, but they were caught out in the 67th minute as the visitors took the lead against the run of play.
Klaus' shot took a wicked deflection off Mainz captain Nikolce Noveski to loop over Christian Wetklo and into the back of the net.
Mainz naturally responded by increasing the pressure and Grun was kept busy in the Furth goal, turning away shots from Marcel Risse and Noveski as Furth held on.