The
general consensus(Johnson, 2010) in science is that religiosity negatively associates with crime. In other words, the more religious the person is, the less likely they'll commit a crime or take licit drugs. These studies have been replicated in
numerous countries(Brauer, 2013) with the same results. One of the shortfalls for these studies was that most of the participants were Christians. So, it is only generalisable for Christians. Another limitation is that the mechanism of this association is unknown. Some psychologists think it may be related to social control theory and that religion teaches self-control.
If it was self-control or another facet of religiosity, then the assumption is that non-believers would be more prone to committing crime and there is some stigma towards non-believers, in some countries, that atheists are dangerous. Unfortunately, most of the studies done on religiosity do not properly measure non-belief or ignore it altogether.
One study(Jang, 2013) measured non-belief and found non-belief also negatively associated with criminal behaviour, and the spiritual-but-not-religious was positively associated with criminal behaviour. However, a
recently study(Schroeder et. al., 2017), and the one I would like to talk about, noted this disparity in these measures and consequently measured agnosticism and atheism correctly. They found atheists and agnostics have similar statistics to the highly religious. In fact, the most criminally dangerous group were the uncertain believers. It was more likely an uncertain believer would take illicit material and commit crimes(see image).
Therefore, if there is any stigma towards the morality of atheists/agnostics, science tells a different story.
I tried to find the open-source papers, but it's not always possible. The main discussion here is on Schroeder et al., but any may suffice.
Why do you think uncertain believers have a tendency to commit more crime?