My impression is that there was a gradual change in values as transportation, residences and daily routine also changed and created certain pressures. Even most politicians rarely use neckties anymore unless there is some clear reason why they should.
The general rise in variety of reasons for spending money is probably a factor as well. As is the wider acceptance of a variety of dressing styles, which can't help but make the perception of the need for formal dress somewhat weaker.
Then there is the advancement in materials. Some fabrics that are very cost-effective and long-lasting such as microfibre arose in recent decades and made it practical to have useful, lasting clothing that is not always formal, classical or expensive.
I would believe that some influential moments with at least one leg in fashion ended up making significant numbers of people actively avoid looking too well trimmed and/or formal. Beatniks, hippies, punks and grunges come to mind. Each new wave amounted to further encouragement to dress as one pleases instead of necessarily how others would expect or prefer.
Generally speaking, the trend in recent decades has been towards better acceptance of individuality and its expression in various ways, including fashion.
Frankly, it is only a problem if one expected otherwise.