The idea that matter was composed of elements and atoms began with the ancient Greeks. However, atomism, as they called it, was a competing theory of matter that vied with the elements theory. It was Empedocles who first argued that all matter was made up of earth, fire, air and water.
This unfortunately common confusion is almost entirely inaccurate. In reality, in opposition to the "atomists" (to the extent such a classification is useful rather than merely anachronistic) were those who viewed all that exists in terms of a "continuum". A great deal of debate becomes muddled in modern simplistic summaries in no small part because of the terms, but even more so because of the fundamentally different ways in which the ancient Greek philosophers thought about such fundamental concepts as those of "change", "matter", "cause", etc. It is quite easy when reading e.g., Aristotle to require a set of mental readjustments to manners of speaking (particularly if one is reading only translations, which are rather fundamentally inaccurate without at the very least extensive commentary). For example, Aristotle clearly sees the nature of causation (
aitia) as concerned not only with the kind of "cause-and-effect" relationship we tend to think of as related to "cause" but more fundamentally with the nature of things/stuff (
physis). In his
Metaphysics, for example, he uses as a quintessential example of cause that of "bronze" being the "cause" of a statue. Aristotle and those before (and after) him tended to think of things we might call properties or qualities as more fundamental things-in-and-of-themselves while relegating to more secondary classes of "effects" things which we might regard as entities. Another way in which the discussion on Greek atomism can easily become confusing for those who are not familiar with the primary texts and the scholarship concerning these is the issues relating to motion. For central in our extant sources is e.g., the paradoxes of Zeno. Both the atomists and their opponents were trying to resolve the issue of how it can be that things--whether material or temporal--could (or could not) be divided indefinitely.
Empedocles could be and has been considered to be an atomist (it's not at all clear that he should be). Atomism, to the actual individuals and schools we apply this term to, did not so much mean that everything was composed up of little atoms but rather was a stance against the idea of infinite divisibility and the continuum. Thus one could believe that Nature and the Material cosmos was made up of four elements while being an atomist, as there was nothing really particularly unifying either among atomists or in the way in which they regarded the most primitive elements. Again, atomists held that there existed not only indivisible units of "stuff" in the material sense but also of events and actions or time more generally.
These four elements later were extended by Aristotle to include a fifth, Quintessence or ether.
This is mainly anachronistic and/or highly oversimplified. It is true that Aristotle spoke about something call and has been called quintessence, but his entire scheme was opposed to that of Empedocles, whom he was quite critical of. Interestingly, some of his criticisms are relevant as they touch upon the apparent failure of Empedocles' "theory" of elements to properly account for the way in which the mixtures and aggregates of elements could make up essential substances and their properties.
This was based on the fact that he believed that everything above the moon's orbit was perfect and was therefore made up of something less worldly.
More fundamental to Aristotle than elements was the division be between the matters/nature of things of the Earth and celestial things/matters.
It was not until the 1800's that the idea of the atom was revived by English Physicists like J.J Thomson and then Ernest Rutherford in the early 1900's that the structure of the atom was complete. The Plum Pudding model assumed a positively charged diffuse "matrix" that was occupied by the electrons which were known to exist at the time. This was later replaced by the Solar system model but was seen to be problematic due to the fact that accelerating electrons would cause the atom to collapse and radiate energy in the form of photons.
Yes, this was how we were taught in high school chemistry too. A central problem is that the "Solar system" model was proposed in 1911, and lasted 2 years. In 1913, Bohr proposed his shell model in which the notions of classical "orbits" become basically untenable. And even at the time, Bohr was aware that something more radical was needed.