Dictionaries give multiple definitions of many words; varied meanings: historical, colloquial, slang. The trick is to choose the appropriate one. Using a popular dictionary to define technical terms can be problematic. Best to use a specialized dictionary of chemistry, biology, philosophy, &c for serious work in specialized areas.
If your goal is to obfuscate, proposing a definition inappropriate to the discussion is a good starting point.
"Atheism" unmodified, is the basic, essence of atheism: the lack of or non-belief in God. The question isn't weather babies are strong atheists, gnostic atheists, religious atheists, indifferent atheists, narrow atheists, active atheists, or whatever, and ignorance of 'X' logically implies a non-belief in same.
The question is weather a baby is a (basic, essential) atheist -- (finer gradations to follow).
Babies, grasshoppers and cabbages lack a belief in God, ergo, the definition applies. It might not be useful. It might be irrelevant to the issue at hand. It might be absurd in context -- but it applies. it's technically, semantically, lexicographically apt. The philosophical or emotional ramifications may be disturbing, but they're irrelevant.
The fact stands, Babies are, technically, atheists/weak atheists.