When formally leaving the (Roman) Catholic church, this is often called 'getting debaptized'.
I formally left the Catholic church, but I never considered myself as 'debaptized'. I may not be a member of the Catholic church anymore, but the sacrament lasts.
I believe a person's baptism into God's flock is binding no matter what the religious persuasion of the person at the time. The reason is I believe there is just one God. But an infant baptism is different. Infant baptism is a reflection of the parent's intention, not the child's.