My impression is that a lot of anti-vaxxers are misled by misinformation rather than having malicious intentions themselves. In those cases, I definitely find their death or severe illness tragic because it is so preventable.
I have family members who fall within the above category, so I don't look apathetically at most anti-vaxxers' deaths. I wouldn't find it any less difficult to lose them just because they were anti-vaxxers.
I separate people who are taken in by gaslighting in particular and lying in general and those who actively promote lies. The former are tragic, the latter are or should be criminal.