I am a secular atheist and believe in the maxim each to his own, so long as what a person does brings no harm to other sentient beings. The problem is some -- who view "homosexuality, adultery, abortion or ..." atheism as wrong -- take actions that would harm those who practice the offending life style, or hold the wrong views. So in propagating the notion that the above individuals are immoral, ie. bad, or offensive to God, places the onus upon extremists to do something about the problem and moderates, where possible, to place legal restrictions upon those who offend them -- ie., making homosexuality or atheism a crime, as they are in many Middle Eastern cultures. Suddenly, it is no longer a case of live and let live, but live as I do, or else! It is a slippery slope when some members of society claim moral superiority over those who don't live as or believe as they do and hold God up as the rational for their superiority, because -- of course -- God trumps all. You can disagree all you want with the correctness of the aforementioned positions, and I will defend your right to do so, but don't flash the moral superiority card at me. In some countries that card would even trump my right to exist, except in hiding.
My assertion that homosexuals, adulaters, pro-abortionists, atheists, or whatever, are not immoral, means they have every right to live a full and rewarding life. Your claim that they are immoral would appear to mean that they are bad for society and should have their life styles curtailed, restricted, and perhaps silenced, because it is not right that something that is immoral and bad in society should be given free reign to fester and spread like a disease to infect the moral. Or do you think the immoral, as you would call us, should be allowed to live openly and without restrictions from the law? (I am not a homosexual, but I am an atheist, and I am assuming you would consider this something undesirable and immoral).