Anyone who is genuinely interested in arguments against polyamorous relationships needs to take a look at
the findings of the Supreme Court of British Columbia. Basically, it found that at least one form of polyamory, polygamy, typically disadvantaged children and women.
What makes the case interesting is the testimony of one of the expert witnesses, Dr. John Witte jr., a scholar and scientist who allegedly had no preconceptions about polygamy before investigating it at the request of the court.
Witte's report can be found here.
As for myself, I do not necessarily agree with the court, but I think any really serious discussion of this issue would need to take the court's findings into consideration.
Of course, the court does not address the broader issue of polyamorous relationships in general -- just polygamy.