Yes. I agree completely with your first sentence.
I think that's a sticky area, because I see what you're saying and I also see the least restrictive part as being a safeguard for balance in some situations. It also has a potential to work as a preventative measure for not allowing the law to over-reach, in favor of the rights of one party -- to the point of infringing on the rights of another, by forcing someone to do something they really believe is wrong, or penalizing them for not doing it, simply because some other party that could achieve the same thing in another way wants to insist/force that person to do that thing, even when they think it is wrong.
I understand that concern, too. I think that this problem can be resolved - at least in part - if we as a society stop conflating freedom of religion with freedom of conscience. Not all matters of conscience have to do with religion, and not all aspects of religion are matters of conscience.
For instance, hopefully we both agree that requiring church-run daycares to have the child-to-caregiver ratio of other daycares is not an infringement on anyone's freedom of conscience... though some states allow church daycares to be exempt from these sorts of rules.