The solution may lay in remembering that we don't possess knowledge or experience of all that exists. So for us to make the claim that cause and effect must rule not only the material universe, but all other possible material universes as well as all other realms of existence seems rather an arrogant claim, to me.
Secondly, our own universe is not ruled by cause and effect alone, as is often claimed, but is also is subject to chance. It seems to me that this should throw a wrench into the whole first cause debate as well.
So for me, the traditional first cause argument, at least in the mechanical a sense, is a very weak argument. Where it is much stronger, however, is in a more philosophical sense.
We know that all that exists is an expression of energy. Matter, space, motion, time, etc.; all these are expressions of energy. We don't know what energy is, yet, but we do know that energy does not manifest itself in any way. It is limited. It is only able to manifest itself in certain ways. And this begs the question: what is governing the way energy can and can't behave? This question is important because whatever the answer is to this question, is responsible for the nature and character and structure of all that exists as we know it. Whatever the answer to that question is, it's the source and sustanance of all the exists, including ourselves. It is "God" by most definitions of the term.
This is the "first cause" that I can't get around.