The 36-year-old Croatian beat 13th seeded Tsonga 7-6 (7/3) 4-6 7-6 (7/2) 7-6 (11/9) to become the oldest man to reach the second week of Wimbledon since 1976.
The big-serving Karlovic has reached 136 aces for the tournament so far and will now have third seed Murray in his sites after the Scot beat Andreas Seppi in four sets on Saturday to set up the fourth round clash.
In a chat with media on Saturday, Karlovic said he would love to beat Murray for the first time at SW19, even though the home fans would not like it much.
"Yeah that could be not nice I guess, but that's what I'll try to do," said Karlovic.
"Whenever you win you feel in good form.
"I feel good. It will not be easy (against Murray) in the next round, but if I continue serving like this I will definitely have an opportunity.
"I know every game is different, so I always try to start serving well, and if it's a lot of aces then things get easier,” he said.
"Each point is a new one, so I don't feel like anyone can beat me.
"I just always go on to the next one, the next point. I've never beaten Murray, so now it's an opportunity to.
"I feel good, I like what I do. I'm having fun.
"So I don't know how it will turn out, but I have a chance, for sure," he added.
Catch all the Wimbledon action LIVE on FOX Sports Play