Yes, I'm really a man.
Chromosomes really don't mean all that much. Sexual differentiation and development is a complex process. Many things can happen along the way to make the outcome something different than we may expect. There's various genes and hormones all involved in the process. There's various intersex conditions and more than two karotypes (chromosome combinations), so chromosomes aren't a good way of discerning biological sex. Then there's examples of people being born with reproductive tissues of both sexes. They're not true "hermaphrodites" (which don't really exist in humans and is an outdated term). But it's more like males being born with ovarian tissues (such as ovotestes) and males being born with full-fledged uteruses, while having male external genitals and a male gender identity. They often don't know until something happens to make them get checked out by a doctor and it's discovered decades down the line. So there's many different things that can happen.
With transsexuals, there's studies that have shown that our brains, at least in certain areas, are different from the brains of our presumed chromosomal sex. There's also been a gene identified that's associated with male to female transsexuals, iirc. There's also something called "digit ratio". For some reason, prenatal androgen exposure effects the length of the index and ring fingers. Males tend to have a longer ring finger than their index fingers. Females tend to have ring and index fingers that are about equal length or an index finger that's a little longer than their ring finger. Transsexuals follow suit. I'm a female to male transsexual and my ring finger is about a knuckle longer than my index finger, pointing to me being exposed to high levers of androgens while in the womb.
Transsexualism appears to be a sex differentiation disorder. it's a medical condition. Apparently something happens in utero where the brain is hit with a high level of the hormones of the opposite sex (an XX karotype fetus being exposed to high levels of androgens, for example) and this causes the brain to develop more in the fashion of the opposite sex. This can cause the child to end up having a sex/identity that's the "opposite" of their chromosomal sex.
Of course, science doesn't have definite answers on this topic. We're still learning. But the evidence points to transsexualism (which is a medical term) being a neuro-biological issue. But the evidence is that clear that those with strong sex dysphoria (because the dysphoria is really caused by the brain expecting one bodily setup but finding another and it can't handle) are best treated with hormonal therapy and/or surgery to treat the dysphoria.
As for gender identity, you know what your identity is in early childhood. It's something enduring and can't be changed. Cissexuals (those whose gender/sex identity isn't in conflict with their bodily setup and biological sex) just don't think much about it because they have no reason to - there's no conflict that causes them to question it. But transsexuals know that something isn't right from a very young age. I always perceived myself as a male, from my very earliest memories.. When puberty rolled around, I was not happy and in denial about my breast development for a time.