This is a crucial point. Understanding relatively mundane text is not so difficult.Again it depends on the nature of the text. Most texts aren't deeply tied to a culture and most often a cursory knowledge of the culture in question is more than enough to understand a text. Some philosophical and historial texts might require a more deep knowledge of the culture producing them and for those are important detail for any serious critique of a specific work.
An account of a battle, or a sea voyage, or somebody's life history is far easier than Scripture. Because most of the concepts are objective and so the meaning of the words used to describe them is clear.
Then there are far more subjective concepts like God and morality and prophecy and such. There are no objective standards for truth in the religious world.
Nevertheless, people commonly mistake their own opinions for objective truth. So people often believe that their personal opinion about the meaning of some Scriptural text is The Word of God, and those non-believers who don't find their opinions authoritative are evil and rebelling against God. Oftentimes, they'll flat out deny objective evidence to protect their opinions. Like Creationists and homophobes and people like that.
Not all religious people are like that. But lots of them are. And so religion tends to drag primitive and immoral beliefs into the modern world.
Because humans have a tendency towards arrogance and religion encourages that tendency.
Tom