Disproportionate effect is legally relevant. A great many laws and policies have been overturned specifically because of their disparate impact on certain groups.
Relevant in government, employment, & public accommodation.
But a private group that is dedicated against certain politics
& acts...in this case involving terrorism & oppression...there
is less legal coercion available to government.
So again, they could have the same speakers as they'd want anyway if they avoided the whole ban, even if it was deemed technically legal.
The issue I see here is the right to exclude some groups.
Could a Jewish student group exclude Nazis, KKK, etc in your view?
I'd say yes. But a government venue could not exclude them,
even with their heinous agendas.