Actually the truth was not nearly do dramatic as an asteroid.
A wild lawless town sprung up in the desert around a newly discovered water source. The structures were hastily built from a porous scorched volcanic stone that was common to the area, stuck together with wet sand and lime. It worked but made for weak structures. As time passed the lawless reputation of the town spread, and it grew, but the watering hole was being depleted.
Finally, the depleted water well had created a giant sink hole that collapsed in on itself, taking much of the town down with it, and disturbing the earth around it to a degree that many of the remaining weak structures collapsed, too. And because the water source, what was left of it, was now deeply buried, the town was abandoned. Leaving the fallen remains of charred volcanic rubble to stand as a testament to the caravans that still passed by, of the wrath of their God in the face of man's arrogant inequity.
Some archeologists used the Arabic version of the story to find the site, back in the 1980s.