The hosts' second win from four games leaves them just a point behind second-placed Norway, while Slovenia lag behind on three points.
Both sides had chances in the first 20 minutes, Albania pair Armando Sadiku and Ervin Bulku seeing shots from the edge of the box saved while their keeper Etrit Berish denied Zlatko Dedic, twice, and Andraz Kirm at the other end.
Sadiku headed Ansi Agolli's corner wide from six yards and Slovenia's Tim Matavz saw a long-range effort saved moments before his team fell behind.
Emiljano Vila was the provider, crossing from the left for Roshi to beat Slovenia keeper Samir Handanovic with a downward header.
Vila shot wide at the start of the second half and, after Berisha saved from Josip Ilicic, Roshi saw a shot parried away by Handanovic.
Matavz headed Ilicic's cross wide and Kirm's shot was easy for Berisha, who then also held Mirnes Sisic's header.
That preserved his side's clean sheet and three points as Kirm failed to make anything of a pair of stoppage-time corners.