Some freak at a bad dinner party wrote: There is no such thing as a definite gender
When you think about it, we are all technically the same. Testes are dropped ovaries. The clitoris is an undeveloped penis. And breasts are glands that got an extra dose of hormone.
Considering that we are machines that respond to different stimuli, the only thing really making us unlike each other would be the amount of hormones and other chemicals during critical stages in development and a sustained hormone level (unless you have menopause) throughout life.
So I suppose, in actuality, we just don't understand anybody around us.