What I believe or don't believe is completely irrelevant. The fact of the matter is that the gospel writers and Acts failed to notice that Jesus' brother had a ministry and that he was supposedly martyred.
So what? The gospels DON'T fail to mention that Jesus had a brother. We don't know if acts fails to mention James' ministry, because the various people named James in act aren't always explicitly identified. We don't know how important Jesus' brother is. All that doesn't matter.
You can't logically go from claiming that the gospels and acts don't support the later christian notion of James Jesus' brother as later active as a missionary in the early christian circles to the conclusion that he did not therefore exist as Jesus' brother. The conclusion doesn't follow. It is perfectly plausible to suppose that James was relatively minor to the early christians and therefore not worth mentioning, except maybe in passing, to the early NT authors. It is equally plausible to suppose that he wasn't active until later, so the only one who would mention him is the author of Luke/Acts, who, while he clearly knew of him, perhaps deliberately chooses not to mention him.
All that is meaningless. What matters is that our earliest source mentions him and knows him first hand, he is attested to by an early non-christian source, and he IS mentioned in the gospels. What we can say about his ministry is irrelevant, and again it isn't logical to go from "later christian beliefs about James are unsupported" to "James wasn't Jesus' brother."