You'd need to develop very strict behavior protocols to make such visitations a success. Catholics, Orthodox, Jews and Buddhists have means of visiting one another. That has already been worked out. These each have a large contingent of people who do not have any problem with the other three religions. That would be your place to begin perhaps. I think protocols are needed. Its easy to offend people when you don't know how to behave around them. I think you should investigate how Catholics, Jews and Buddhists visit one another and what the rules are; and you should publish some etiquette rules.
Staring at the problem and thinking about it rarely leads to solutions. On large state university campuses you can observe what really divides people, but finding solutions to division finding a way around problems. If you visited enough campuses you might find a campus that has made some progress. Then you could study that success and perhaps find something there which can be replicated, to add to what you learn from the Catholics Jews and Buddhists. Its very important because campuses are absolutely caked with social fear, isolation and cliques. People there want to meet the right people, network, then get out of there. They don't generally want to know everyone, because many people are toxic, dangerous, losers, etc. My idea is to find a campus or some campuses that have managed to overcome the fear and cliques. This should lend some light to your problem.
I'm envisioning a large glass window at each place. At each meeting hall participating religious places could have a large room that visitors could stand in and observe services through such a window, and visitors could be seen, too through that same large window. Its a two way window, possibly soundproof. They could visit, and the congregants or meditators could be assured of some level of normalcy. There could still be interruptions, but they'd be less likely because of the window pane. I think this idea needs work, but its an idea. You might be able to get some participation at a lot of places.
The window room needs have a good name, because you'd need to get funding to buy these window rooms. You'd need to call it something savvy that would encapsulate its purpose.