It says in the bible that Mary was pregnant before she got with Joseph, and Jesus had an elder brother too, James, so she would have had a child to at least 3 men. Strangely they do not mention that James was an older brother in the bible since it would kind of ruin the "I'm a virgin story", you find that out by reading the Gospel of Thomas. According to the bible Jesus had 4 brothers and at least two sisters.
An unmarried mother in those days would have faced some difficulty Joseph could have had her stoned to death for not being a virgin when he married her, her clan or the group she belonged to could have turned on her, maybe even killing her in a bid to protect the reputation of the group.
In this day and age women have lied about the father for much less, I think threat of death would pretty much drive someone to say anything; though I don't believe it is true either; by which I mean I think the whole bible is a story.