What do you expect him to do? Do nothing, say nothing, be nothing? What he did was good. He tried to help people.
"Truth has come and Falsehood has Vanished.. (Qur'an 17:81)"
I would not threaten you to become "agnostic". You are free to be a Muslim. I would not take away your rights of whatever religion you wish to follow.
Do I have the same rights to being agnostic, and not a Muslim or a Christian? Don't I deserve the same curtesy?
What Muhammad did, was to cause trouble, a trouble he started once he began threatening to take away the pagans' rights to worship at Kaaba.
In the early stage of Muhammad's preaching, the pagans may have mocked his teaching, they didn't threaten Muhammad, nor did they stop him from preaching or gathering followers.
The pagans' attitude only changed, when Muhammad began to changing his preaching, when he began inciting his followers into destroy the idols in Kaaba.
So really, Muhammad started this trouble first, not the pagans, because he threatened the pagans first. That's when the pagans felt the need to react.
If Muhammad never threaten them first, then Muhammad might have been left alone. Muhammad might not have been forced into exile.
Things would have been different if Muhammad had left them alone.
Persecution can happen both ways. Muhammad threatened the pagans, the pagans pushed back. Muhammad felt his life was threatened, so he left, and migrated to Medina.
Muhammad could have left Mecca in peace, but he didn't. Muhammad began arming his followers in 623, and started a series of raiding, attacking and looting Meccan merchant caravans. Again, the Meccans were reacting to another new problem started by Muhammad, and began sending an armed force to protect the latest caravan, in 624, but this triggered a war that didn't end until 630.
So it was Muhammad again, who started a new trouble with the Meccans. Those raids and looting led to war, a war that Muhammad started.
Muhammad caused other troubles in Medina, with the Jewish tribes that were living in Medina. Muhammad exiled the Banu Qaynupa from Medina, in 624, and Muhammad stealing their lands and properties. This exile triggered trouble with the Banu Nadir, and then the Banu Qurayza.
After Mecca submitted in 630, Muhammad started a new war, with Ta'if, when he began besieging the city. The Ta'if weren't at war with Muhammad, but the motive for attacking Ta'if is because the townspeople, back in 622, didn't want Muhammad brewing trouble in their pagan town, so they rejected giving sanctuary to Muhammad in 622.
So really all the wars were started by Muhammad himself. None of these were defensive wars, because Muhammad was the aggressor.