There are large swathes of society not worth listening to at all. They are the ones who are significantly intellectually and morally challenged. I'm really only interested in what decent, thoughtful people have to say, well educated critical thinkers. It turns out that they are mostly liberal humanists (this includes some theists, who basically share the same values as atheistic humanists). If you're a faith-based thinker, your conclusions have no value to me.
If you lack empathy, such as the gun people, the forced birth people (please spare me about their empathy for fetuses), and the racists, your opinions don't matter. If you're a MAGA Republican, your opinions don't matter. These are not people to share ideas with. They are people to keep an eye on, so listening to them is merely surveillance. Did you support Trump knowing who he was? If so, I don't care at all what you think is right or wrong, or what you think about politics. It's a litmus test, since in my estimation, it is absolutely impossible for a decent person of at least average intelligence to come to such a conclusion.
But having segregated that aspect of society, there is still a lot of diversity of opinion in the residual contingent. The critical thinker doesn't passively take his opinions from others. He may consider them, but unless they resonate with him and he feels that he can and should defend them, he rejects them.
This whole idea of listening to these other people as if they might have something worthwhile to say is rejected. They are not people to compromise with or cozy up to. Just decent, contemplative people, and by decent, I mean exhibiting humanist values. This does not exclude the liberal theists (Abrahamic, dharmic, pagan, whatever), who, as I mentioned, I consider decent, compassionate, and thoughtful people - fellow humanists. These other hateful, mindless people have nothing to offer, so why listen to them?
Yes. It obviously has with me. There is nothing I want to hear from them, and nothing I can teach them. I don't consider them fellow anythings, and they are not included in my "we." I am indifferent to their fates. I consider this an appropriate reaction to an unfortunate reality. No, this is not how I would like to feel about anybody. I wish they didn't exist. But they do and excluding them from my "bubble" of reason and compassion is constructive.