Defender Grant Hanley gave the home side the lead the against the run of play with a header from a Charlie Mulgrew corner at the end of the first half and the Scots, in Gordon Strachan's first competitive game in charge, looked a bit more comfortable after the break.
However, in the 70th minute, after Scotland forward Robert Snodgrass was sent off for picking up his second yellow card of the game for a challenge on Welsh defender Chris Gunter inside the home penalty area, Aaron Ramsey levelled from the spot before Hal Robson-Kanu headed in the winner just over a minute later, with Ramsey shown a straight red card for a 'last man' tackle on James McArthur.
However, there could be no Scottish complaints about the result, which will boost Wales for the rest of the campaign - even if their hopes of qualification remain slight.
The second of Scotland's double-header is in Serbia on Tuesday while the Welsh welcome Croatia but as far as the Scots are concerned, with two points from five games, their trip appears to be an exercise in futility.
Most of the attention in the build-up had been on Gareth Bale, who shrugged off a virus and ankle knock to take his place in Chris Coleman's side alongside Celtic midfielder Joe Ledley and former Celtic forward Craig Bellamy.
Strachan raised a few eyebrows by including 21-year-old Blackburn stopper Hanley alongside skipper Gary Caldwell, with Alan Hutton and Mulgrew as full-backs.
Graham Dorrans and McArthur anchored the midfielder with Chris Burke, Shaun Maloney and Snodgrass backing up striker Steven Fletcher.
Strachan's plans were disrupted with only a couple of minutes gone when Fletcher injured himself on the way down from an aerial challenge with Welsh defender Ben Davies.
The Sunderland striker looked in some distress and he was carried off on a stretcher to be replaced by veteran striker Miller.
Snodgrass was booked in the 10th minute for lunge at Gunter, before Miller headed Burke's deep cross over at the back post.
Scotland were being outplayed and another dangerous Welsh move ended when Bale's drive from 30 yards was parried by McGregor before McArthur completed the clearance.
As the snow continued to fall, a decent move by the Scots in the 32nd minute ended up with Miller heading Snodgrass' cross over - and for the first time in the game Scotland looked as if they were getting up to speed.
Four minutes from the break a robust tackle on Bale by Hanley left the Spurs forward requiring treatment and he was given more stick by the home fans as he re-entered the fray, although he was replaced for the second half by Jonathan Williams.
The Scots grabbed an unlikely lead when Hanley leapt at the back post to power a header from Mulgrew's corner past Boaz Myhill from six yards.
Five minutes after the restart Dorrans flashed a drive over the bar from 20 yards, before Snodgrass curled a shot against the Welsh post from 25 yards after taking a pass from Maloney.
As conditions deteriorated, Wales kept coming forward and continued to rack up the corners against a hesitant Scottish defence.
In the 57th minute Wales substitute Andy King had the ball in the Scotland net seconds after replacing Jack Collison but French referee Antony Gautier awarded a free-kick for Ashley Williams' challenge on McGregor.
McGregor saved Bellamy's free-kick in the 70th minute but a minute later Snodgrass recklessly conceded a penalty - and earned himself a second yellow card - for a challenge on Gunter in the box and Ramsey hammered the spot-kick in off the bar.
The Tartan Army were stunned but they had more to worry about barely a minute later when the unmarked Robson-Kanu headed King's cross past McGregor from six yards out.
With five minutes remaining, and the game slipping away from Strachan's men, McGregor saved with his feet from King's angled drive.
There was to be no late rally from Strachan's men but there was time for Ramsey to pick up a straight red card for a last man foul on McArthur.
However, the home side failed to capitalise on the free-kick and, with that, all hopes of Rio disappeared.