It is still odd that the IMF is criticized for fulfilling its role, when the critics are representatives of countries that have asked for its help and often enough failed to fulfill their promises towards it.
Not that I disagree with your overall point that countries technically are not
forced to ask the IMF for loans, but I view the IMF against the backdrop of neoliberal exploitation rather than in isolation of it. In an ideal world, no external factors would contribute to the sorts of disruptions that typically lead a country to ask from a loan from the IMF, but the reality of geopolitics and economics is far from ideal.
Many countries face economic difficulties due to mismanagement despite having the ability to avoid an IMF loan and being in charge of the ineffective policies that led to the economic problems (as opposed to being forced to adopt them), but many others face external pressure and are essentially turned into vassals of more powerful countries for various reasons.
When a country in the latter group faces economic difficulties largely or partially aggravated by factors like interventionism, exploitation, and environmentally destructive extraction of resources carried out by richer countries, it is not surprising for it to ask for a loan, in which case the IMF sets forth extremely neoliberal conditions that simply don't work well in some countries and can enable further privatization and subsequent increases in corruption and concentration of wealth. It almost seems that a core assumption of some of the IMF's policies is that similar neoliberal measures should work effectively everywhere, without much or any regard for the geopolitical and economic disparities between different countries.
The IMF is not a charity, nor do I believe it should be. I also don't think of it as "good" or "bad"; I mainly see it as an extension of the current global economic system, which has many pros but also many cons primarily in the form of exploitation, excessive concentration of wealth, and environmental unsustainability. I think that as the effects of climate change intensify, that last con may well prove to be the downfall of our current way of life and the brand of neoliberalism typically encouraged by the IMF as "reform"—in which case it would be a con arguably outweighing all or at least most of the pros.