Agree on infinite regression, and agree we don't have an explanation using natural causes, or supernatural causes either for the same reason (infinite regress) that you cite.
What I said was "Given the existence of the universe though, natural causes seem to be doing pretty well as an explanation", by which I meant that natural causes are doing well in other areas.
Incidentally, I don't think that rules out god/s so long as you don't insist that they must have created the universe, be eternal, and so on. A god could (logically) be part of the universe, having been created at the same time, or developed over a period of time, or whatever.
If I were looking for a god, I would look inside the universe, now. Anything else would be of little consequence anyway.
There have never been any god existing, before humans, and since every religions and cults and spiritualities are of human constructs, it is far more probable that the various deities are manmade or of human inventions, basing their gods their superstitions either as personification of nature (eg sun, moon, sea, rivers, mountains, rain, storm, etc) or that god’s resembling humans (eg that of human rulers with sceptre, crown & throne symbolizing power of monarchs, or that of human shepherd or that of a mother, etc), or that of depicting god as part human & part animal (eg the falcon-headed Horus or Ra, ram-headed Khnum, or cow-headed Hathor, or the winged goddess Artemis or Nike, etc).
Genesis 1 say, god made humans to resemble like him, but more likely humans created god in human’s image.
Even in the Bible, there have been imageries of angels with wings of some birds, or head of some sorts of animals, while the rest of bodies are “human” in form.
In Ezekiel 1, there are the four living creatures, most likely archangels or seraphs, with four wings and feet of calf, and weirder still each angel had one head with four faces, that of man, lion, bull & eagle.
Only humans would create god that resemble themselves, as being human-like, or that of animals that exist on Earth.
There are plenty of evidence that deities are all man-made.