Actually it doesn't. It protects people from unecessary exposure to risk.
If the unvaccinated person doesn't like being excluded from these activities, the remedy is simple, easy and in their hands: get vaccinated, or get a letter from a doctor explaining that they can't be vaccinated for medical reasons.
There is no reason at all why society should tolerate potentially highly infectious people in their midst when the means of reducing the risk is easily available to all.
Ideally it would protect people if we knew who had the virus (asymptomatic or symptomatic). For now, we just don't know. An illusion of protection. If it were a potential bomb threat, then yeah. Everyone duck for cover regardless where they are, who they are, and such... but this, no.
In this case, "potential" does not say anything about the person's health, vaccination status, even his political opinions (if that matters). All we have is "he is not vaccinated=he's a bomb waiting to go off." Potential is a fear word and the risk of catching the virus is dependent on how much vaccinated people put emotion and concern over whether that potential warrants segregation.
I mean, it's alright to an extent to be afraid of the unknown, but the unvaccinated (exempted for medical reasons and not) are not bombs. The issue is with the vaccinated not the unvaccinated.
How do you know which unvaccinated person is at a higher risk of catching the virus?
Are there other factors that determine whether you're (and others) are in danger or is it only that one is unvaccinated alone?