I think there are several possible explanations, more than one of which might be correct.
First, I think easy access to educational material, different perspectives, etc. on the internet plays a big part. When someone has a profound question, instead of taking the word of someone else on faith, they can research the answer for their self, and fewer people require a religious answer when there are natural explanations available as an alternative. For those who were already religious, the internet can lead to finding information that contradicts what they believe and lead to them changing their beliefs once they encounter new information.
Second, I think there is becoming less of a stigma against admitting you don't have a religion. In the past, and for many people today, it was social suicide to deny the religion popular in your area. There are certainly more places today where people just don't care if you are religious or what religion you adhere to. I think this trend will continue, because I know there are still plenty of places where atheists or otherwise not religious people are afraid to let anyone find out for fear of being ostracized or picked on. As the stigma goes away more and more, people who have been non-religious all along will just admit it where they have been lying on these types of polls.
Third, I think people feel less of a need to be associated with particular religious practices or parishes. Not all these people who "left religion" stopped believing in God or stopped practicing their religion in their own way. They may not have a name for their beliefs, and they may not have a building to gather in on particular days of the week, so they don't call themselves religious anymore. But I think the number of people who are still theists or deists is still larger than these polls may reveal.
Fourth, I think people no longer want to be associated with religious organizations because, in America, some of them hae become quite embarrassing. For former members of fundamentalist Christianity, whose churches denied science, opposed homosexuals having equal rights, treat women as inferiors, and so on, being associate with that religion any longer makes them feel like a part of something they don't support or approve of. Some people may find a different religion to associate with, and some may drop it altogether and go their own way.