The UCI on Monday ratified the sanctions recommended by the United States Anti-Doping Agency, who concluded Armstrong and his United States Postal Service team ran "the most sophisticated, professionalised and successful doping programme that sport has ever seen".
Armstrong was banned for life and all his results from August 1, 1998 removed, including his seven Tour de France wins from 1999 to 2005.
Cavendish believes the sport is now clean and has urged Armstrong to follow in the steps of his Great Britain team-mate David Millar and his former HTC director Rolf Aldag who have admitted to doping.
"It's not fair on me having to answer these questions. If you've done something, confess," Cavendish told BBC Sport.
"That anyone can damage the sport I love right now, it's frustrating.
"I've worked with David Millar. This guy's remorseful," Cavendish continued. "He's repented.
"The team I grew up with, HTC, one of the directors (Aldag) - these guys care about the sport. They ruin their reputation to move the sport on, but other people care more about themselves."
And Cavendish believes this is an issue with society rather than cycling.
"It's not just in cycling; it's every walk of life. There's cheats in entertainment, journalists cheat, every single sport has cheats," he added.
"If you put the effort into catching them and you have a structure that does things properly, you're going to catch a cheat.
"It doesn't happen in other sports not because they are clean but because it's not got the structure cycling has. In my eyes, cycling is the cleanest sport."
British Cycling performance director and Team Sky boss Dave Brailsford believes cycling needs to learn from the Armstrong scandal.
Speaking in Paris at the announcement of the route for next year's Tour de France, Brailsford told Sky Sports News: "That is certainly dominating the headlines at the minute but I think the sport has got to move forward and this is about the future.
"We can't change what has happened in the past but what we can try and be is agents for change in the future.
"This sport needs to regain the trust that it has lost because of the past but I think when you look at next year's Tour it gives everyone an opportunity to focus on what we can change for the future.
"We can't change the past but we can change what we do today, tomorrow and by the time we get to next year's Tour."