Let's not forget that there was a temple in Jewish religion, which was in fact the same thing as all of these places you cited. A place of worship. Sites for devotion are there to bring people together, and to direct the devotees attention to the divine, which the shrine or temple is designed to invite and encourage for the faithful.
The difference being that the Temple was built at God's request and to his specifications.
No building of the Christian faith ever included anything resembling a physical building. The "church" was the people....the building didn't matter. But when the congregations increased in size, buildings became necessary....but they were to be modest meeting places for Bible education and instruction. The gigantic and palacious cathedrals of past centuries stood in stark contrast to anything Jesus commanded. It seems as if there was a competition to see who could outdo whom with the most ornate and elaborate edifice....and all whilst the poor among them were starving......?
Where was Jesus when they were shedding innocent blood? I'm afraid he had long left the building.
When does it cross over into idolatry? When a focus of our devotion becomes an excuse to ignore the principles of your religion and place your beliefs ahead of love. The bible itself, is one example where many faithful turn it into an idol, worshiping their beliefs about it, over the needs and sufferings of others. It's idolatry, when it's used to promote your ego, instead of inspiring faith.
It crosses over into idolatry when we put "things" before God....when we concentrate on physical things like buildings and the accumulation of wealth and power before we consider how we worship and whether it is authentic to God, not just to ourselves.
Keeping the masses in ignorance for centuries and torturing confessions out of 'heretics' as an excuse to silence them was par for the course.....there is no Christianity there....nor can there be in anything that sprang from them.
The needs and suffering of others is not fixable by human means as we are all well aware. Humans are too selfish to care about their fellow man. Only God can fix what is wrong because he will eliminate the cause, not just provide a band-aid.
The Bible can be an idol if it is misused or ignored which is what I see most of the time. It was never designed to beat people up, but to educate them and give them hope and dignity. Jesus showed us how to do that too and encouraged his disciples to do the same.
Egos have no place in Christian worship, but God can use inspiring speakers to build up the faith of others.