You are giving a picture which in my opinion is misleading.
The priests were not from the lower classes of society and not required to work constantly as I understand it.
'The
priestly divisions or
sacerdotal courses (
Hebrew: מִשְׁמָר
mishmar) are the groups into which
Jewish priests were divided for the purposes of their service in the
Temple in Jerusalem...
...Lots were drawn to designate the order of Temple service for the different priestly orders.
[7] Each order was responsible for ministering during a different week and
Shabbat and were stationed at the
Temple in Jerusalem. All of the orders were present during
biblical festivals. Their duties involved offering the daily and holiday
Temple sacrifices, and administering the
Priestly Blessing to the people. The change between shifts took place on Shabbat at midday, with the outgoing shift performing the morning sacrifice, and the incoming shift the afternoon sacrifice.
[8]'
Source:
Priestly divisions - Wikipedia
So basically as I understand it there were plenty of people to undertake the work and they didn't work constantly contrary to the impression you give of priests slaving away 24/7 in my opinion.