This is sound reasoning for your perspective, but I see another.
A person who was born not believing in gods, & continued all his life not believing in gods certainly passed a phase in life where he became aware he didn't believe, & gave thought to not believing. The when & how long of this phase would vary from heathen to heathen, & perhaps be indeterminate. Moreover, some of us could've been predisposed to be atheists because of hardwiring &/or cultural environment. (This is almost analogous to a Jewish infant being a Jew.) So I've always been an atheist, & always had such an orientation. It's not the "truth", but it's true enuf.
Btw, I wouldn't say all babies are atheists.
(It's not something I discuss with them.)