Unfortunately I am guilty of doing just that, which I have come to regret. Not out of any sense of embarrassment, but I feel it's something I should have kept to myself. I have some framed deity pictures and tiny brass murtis in my work cube (some? ). My Indian coworkers were tickled pink when they first saw them. My western co-workers probably think "that's Jai just being Jai" (digression: my middle name is actually Jason, so Jay or Jai works). But now the proverbial genie is out of the bottle, so to scale back or dismantle my work "shrine" (no incense or diyas though.. yet ) would probably send a negative signal.
One woman, on a bring-your-kid-to-work day, made a point of bringing her son to meet me and show him the images in my cube. I take that as full acceptance. I sometimes wear my om pendant over my t shirt, but I don't generally do it in the presence of other Hindus, if they are not wearing any religious jewelry. I feel I'm "trying too hard" and trying to out-Hindu other Hindus, especially being a white westerner.
For the most part, I could care less what other people, westerners, and Hindus alike, think. The reason I don't wander around in veshti or pottu is I don't need to be drawing attention to myself. It's just easier, but if something does come up, so what? We're off to Toronto in a couple of weeks, and will hit maybe 30 temples between there and on the way home. (We're driving from here.) It's been a while, and they don't know us at all, so we'll see how it goes.
At school very few people actually cognised I was a dedicated practicing Hindu. They knew it, but it was in the same importance scale as what kind of car I drove.