Yep, that's the nature of public speech. Just as you have the right to march with posters and a megaphone, other people have the right to yell back at you.
That's one of the things I couldn't figure out about the "turn or burn" style street preachers, who go to public events with their "you're all going to hell" signs and megaphones (some extremely loud) that they use to bascially scream at people. I couldn't imagine that approach being very effective, and instead being very counter-productive.
But then someone who used to do that explained the dynamic to me, and how it's not really about gaining converts, but instead is about the street preacher getting the inevitable blowback, surviving it, and then going back to the church with tales of what they endured, thereby demonstrating the strength of their faith.
Rather weird IMO, but it's certainly their right.