DeMar DeRozan added 24 points and Greivis Vasquez and Kyle Lowry each scored 12 for the Raptors, who clinched home-court advantage in the first round of the playoffs.
Dwyane Wade scored 30 points for the Heat, who can only get to 37 wins – the number that Brooklyn and Boston are already at, putting them in the No 7 and No 8 spots in the Eastern Conference race.
Goran Dragic scored 22 points for Miami and Hassan Whiteside finished with 16 points and 18 rebounds.
But Whiteside also missed 12 free throws, going 4 for 16, and as a team the Heat went 24 of 44 from the line.
Williams hit a 3-pointer with 31.2 seconds left to put the Raptors up by five, and that just about sealed not just the game but Miami’s playoff fate.
The win was Toronto’s 48th of the season and 22nd on the road, both tying franchise records.
Meanwhile, Pau Gasol had 24 points and 13 rebounds and Derrick Rose had 22 points and eight assists in 28 minutes in the Chicago Bulls’ 114-107 victory over the Philadelphia 76ers.
The Bulls remain tied with the Raptors for third place in the Eastern Conference with two games to go for each team.
Robert Covington led Philadelphia with 22 points, Jason Richardson had 19, and JaKarr Sampson 18.
Stephen Curry scored 20 of his 34 points in the first half and Klay Thompson made three 3-pointers in the final four minutes as the Golden State Warriors set yet another franchise record in a 110-101 victory over the Minnesota Timberwolves.
The Warriors broke their franchise record with their 37th home win. They have lost only twice at home and lead the NBA with a 68-15 record.
Curry made five 3-pointers, went 11 of 21 from the floor and had seven assists to help the Warriors complete a four-game sweep of the Timberwolves.
JJ Redick and Blake Griffin scored 18 points each as the Los Angeles Clippers pulled away to beat the Memphis Grizzlies 94-86 to take a step toward securing home-court advantage for the playoffs.
DeAndre Jordan had 16 points and 16 rebounds and Chris Paul had 15 points and 14 assists to help the Clippers win their fifth game in a row.
Zach Randolph had 21 points and 13 rebounds for the Grizzlies, who lost starter Marc Gasol to a sprained left ankle late in the first quarter.
Trevor Booker scored a career-high 36 points as the Utah Jazz beat the Portland Trail Blazers 111-105, denying Portland home court advantage in the first round of the playoffs.
The New York Knicks beat the Orlando Magic 80-79 in a game that had the lowest-scoring quarter in NBA history.
The teams combined to score only 15 points in the second quarter, breaking the previous mark of 18.
Orlando 79-89 NY Knicks
Miami 104-107 Toronto
Chicago 114-107 Philadelphia
LA Clippers 94-86 Memphis
Portland 105-111 Utah
Golden State 110-101 Minnesota