Regarding the OP's question, are M to F transitions more common than F to M transitions? If so, that might account for the former getting more attention than the later, and why people might make presumptions when it comes up.
Then again, like you said it probably has a lot to do with culture. After all, there often seems to be a difference between social attitudes towards gay men and lesbians, as well.
I'm not sure how much of a disparity between the rate of transitions there is. In my personal experience, it seems about equal. Trans men also tend to "blend in" more, especially after a certain amount of time on testosterone. (Not that trans women can't, but that depends on various things.)
According to statistics I heard a year ago, it's more or less 1:1, based on those people who went through with officially changing their first name. So, not any more common.
But yes, it's probably that we trans guys don't stand out as much.
Due to my cup size I have some difficulty passing as male (and having just starting seeing a therapist, it will take quite a while till I can get hormones or surgery, at least as long as I play it safe and keep to the guidelines of health insurances).
But if someone reads me as female, my "male" clothing doesn't seem any strange in this society, and if I get read as male, then I suppose people may sometimes think I'm gay due to some feminine behavior I have from my upbringing, but they wouldn't even think of me being trans. Instead, it often happens that they think I'm 10 years younger than I actually am.
Everyone knows what a drag queen looks like, and that's the picture they have of trans women (as inaccurate as it might be). But there is no real common image of what a trans man looks like, so they don't have something they can look for.
It's not only in western culture, though. There are several cultures that have more than 2 genders, but in many of them, there is only a 3rd, for trans women, and no separate for trans men. I haven't heard of a culture with a separate gender for trans men that doesn't have a separate gender for trans women as well.