I don't know many atheists who feel a sense of aloneness though.
I'd say that's probably more because you haven't talked to them about it than anything. This is no different than homosexuality. They are both things that society in general looks down on but doesn't understand. They both tend to ostracize the people who take part in them. It's not like atheists are necessarily suicidal or depressed or anything, but when 76% of the population is Christian and everywhere you go, Christianity is promoted as the way you should live your life to the point where, because many Christians think it's wrong, gay marriage is banned, you do tend to feel ostracized.
Most of us simply go on with life and don't center it around our non-belief. That's sort of my point. An atheist who centers his life around his atheism has turned it into a religion when in fact a true atheist has no such angst.
It's not about centering on it or not. It's the fact that it makes you different, and in many people's eyes "bad". Gay people don't necessarily "center" their life around their sexuality, it's just a part of them, but it definitely tends to alienate them from a good portion of society.
From what I've seen it's not too hard to avoid people who would look down on atheists.
Then you're not living in America.
Stay away from church is the first step. Don't discuss religion at work is the second. Get those two covered and the rest is pretty simple. Personally, I have no problems with people in general and I don't bemoan the fact that I never really discuss non-belief with anyone else (except here). Life has too many other interesting things to do and see than to be wrapped up in any kind of -ism.
So, anyway, back to reality here in America where the phrase "America is a Christian nation" is as common as "How are you today". Yes, avoiding church and discussing religion at work might help lessen the exposure to people who look down on non-Christians, but you simply can't get away from that group of people unless you move to another country and pay no attention to the U.S.
I'm not really caught up in my atheism either, but the fact is many, many people are caught up in their Christianity and in thinking that non-Christians (especially the dreaded atheists) are bad, and they show it every day. I also don't bemoan anything. However, there are many people out there who don't feel comfortable saying out loud that they're atheists, and that's not the way it should be.