It could be that you crave it because you truly desire men as opposed to women, or that of environmental factors, but doubtful it's the fault of family. I don't really know your situation, so I cannot give an adequate assessment, but don't feel disconnected by this because it's something that is (for whatever reason) a part of your consciousness. It's who you are as an individual. I had those thoughts when young, and I've learned to embrace it as you should. It's all a part of the human condition. Perhaps you should try it, you may find that you either will, or will not enjoy it.
I myself have not had the chance to find out, but I'm sure I'd like it.
You think in this way of confusion because you are trying to confront your upbringing. The attraction is there, just learn to embrace it. In the end, you will define your true self.
I wish you well in this struggle.