I think you are confused. The average man is dramatically stronger than the average woman. Here is a sexist science fact for you: The men's Olympic weight lifting record for men is 212 kg snatch, 263 kg for clean and jerk. For women, those numbers are 151kg and 187kg. The average man in the US is 14cm taller than the average woman with proportionally longer limbs so he can run faster and reach farther.
There are very few sports where you can give up that much size and strength to your opponent can come out on top unless there is a pretty large skill difference. In competitive sports the skill difference is minimal so size, speed and strength will carry the day.