Your argument might well apply to parent-child sexual relationships, but what of sibling-sibling sexual relationships (involving consenting adults of a similar age)?
I view it as morally wrong, even if the siblings or close relatives (i.e. uncle and niece) consent to the union and refrain from bearing children. That is certainly less irresponsible and reprehensible than incestuous reproductive sex but it is still taboo in my eyes.
St. Augustine condemned endogamy, because he argued that (in addition to the health risk if offspring are involved), marriage served the purpose of bringing people together who would not otherwise be united in bonds of love and amity.
When you married a close relative, you were thwarting that purpose because you were not bringing two different families together, since you and your spouse were already related.
Therefore, as an important tool of social utility, endogamy would have to be rejected as bad for the common good of society, leading as it does to isolationism and nepotism within kinship groups. Such powerful kin groups hold back state-building and democratization to this day in some parts of the world.
Why would we aspire to create closed communities? Enlarging the orbit of one's social relations and bringing different "
tribes" of people together is more desirable than a bottleneck.
Where reproductive sex is involved, it becomes much graver and reckless in nature. There is a remarkable letter from
Pope Gregory the Great (540 – 604) in the sixth century AD, which explains the church's opposition to marriage within the kin group as follows:
A certain secular law in the Roman State allows that the son and daughter of a brother and sister or of two brothers or two sisters may be married (i.e. first cousins). But we have learned from experience that the offspring of such marriages cannot thrive [in terms of health/life expectancy/infertility]. Sacred law forbids a man to uncover the nakedness of his kindred. Hence it is necessary that the faithful should only marry relations three or four times removed, while those twice removed must not marry in any case, as we have said
For some inexplicable reason, the Catholic Church was one of the first institutions in global history (if not the first, actually) to recognise that marriage between close cousins raised the children's risk of being born with birth defects or other health problems.