You think it's just to murder family members who are apostates, or that it was ever just to do so? Just out of curiosity?
It is no more murder than is the death penalty administered by law (and sanctioned in the NT).
I think it is to do as the text says.
"If your very own brother. . ." indicates that no one was to be spared, even one's family.
It indicates that it applied to anyone, and that if it were your family, you were to throw the first stone,
and for those outside your family, you were to participate in the stoning.
We have light on this in the NT in 1 Co 11:20, where Paul reveals that worship of other "gods" is, in reality, worship of demons.
Demons are evil. They are spiritual plague.
God's response is to kill the plague to protect, and keep it from spreading to, the others.
This was the remedy they were to use for the plague (idolatry), when they entered the plague-infested land of Canaan.