The least we would ever want to go is 3,000 or so... just to maintain a healthy genepool.
But I don't think that's quite what you meant by the question. I would like to see at least half our current population as a goal.
The numbers you are looking for are summed up with the equation IPAT... I = P*A*T
I (Impact on environmental carrying capacity)
P (Population)
A (affluence/consumption)
T (technology)
One commonly used example is agriculture... agriculture in developed nations is set to begin declining as technology allowed us to extract groundwater faster than nature could replace it. By the time my son is my age, agricultural output is projected to shrink by at least 10% and at worst 35%. (depending on mitigating factors like continued drought in the southern and central states)
We can shift some food production back east, but will it be enough to keep a surplus for shipping around the world? I have no idea.
Here is a link you may find helpful... it shows resource consumption by nation and discusses some of the issues with consumption.
AAAS Atlas of Population and Environment
wa:do