After sleeping on your query, yes, I think you're being a bit paranoid.
I had trepidation with labeling myself a Hindu (and spoke about it here on the forum), though my worldview certainly qualifies as such, not because I felt I'd stick out like a sore thumb, but because I had concerns that it would be an affront to those where were culturally born into the religion. But with the help of some of our Hindu members here, I realized that my concerns were unwarranted and that being a Hindu is about one's philosophies and beliefs, not about one's ethnicity.
I've attended Vedanta society gatherings and even Puja, and those with which I attended, all of Indian descent, were more than welcoming and made me feel right at home.