It seems to me that there's a current trend going on—and I'm not entirely sure if it's new, actually—where a lot of people defend vocal bigots, demagogues, and rabble-rousers because of the idea that doing so implies they support "freedom of speech" even if it's not "politically correct." I also see a slippery slope where people assume that because political correctness sometimes results in shutting down reasonable debate, political incorrectness becomes a good thing almost by default.
Yes, Milo Yiannopoulos has the right to espouse bigoted, ignorant views publicly. That doesn't make them any less harmful, though, regardless of how many things he's right about. I think it's less a matter of having some correct views and more a matter of weighing the harms and benefits his beliefs could cause—and as far as that goes, Yiannopoulos is clearly on the harmful and toxic side. I think he deserves not to be censored or forbidden from expressing his beliefs publicly at all but to be strongly condemned and called out on his bigotry and ignorance.
Coddling extremists of any kind merely leads to empowering them, and I think there's a need to shut down people like Milo Yiannopoulos in public discussions through reasoning and facts, although I doubt he cares about either of those things given that he has described himself as a "provocateur."