It's not that the consequences of denying human agency is just something we don't like or wouldn't prefer. It's that it makes ethical discussion incoherent. If you don't believe ethical discussion is incoherent - if you believe it's just and reasonable to hold people morally accountable for what they do - then you know there must be some problem with the premise that we have no agency.
I tend to think of actions in terms of cause and effect as well as the material conditions that have led to them, not ethical terms per se. My primary ethical principle is not to unnecessarily harm any conscious creatures (e.g., without needing to do so for self-defense, food, etc.).
We could talk about whether some random tribe's practice of animal sacrifice is arguably unethical, but that, by itself, would largely be a pointless discussion: I think a more relevant and useful question would be
how that practice ended up arising within the tribe and
why they still practice it. Merely telling someone that what they're doing is unethical is unlikely to change their actions or beliefs, whereas material conditions such as education, economic status, place of upbringing or residence, etc., all have an effect on people's beliefs and actions.
So, due to practical considerations and the usefulness of the concept of ethics in everyday life, I don't regard it as pointless. However, if you ask me why a murderer should be prosecuted, my answer wouldn't be, "Because he's immoral!" since a lot of immoral things are neither illegal nor even possible to outlaw. It would be because of the sheer harm to others that a murderer's actions cause.
It's similar to why I would support locking up a crocodile roaming the streets of some town despite not believing the crocodile has any free will or even a conception of morality; its mere presence would pose a risk to the residents of the town and therefore be enough to justify locking it up or releasing it elsewhere it couldn't cause harm to people.