@snood,
Well, take Bruce Jenner transition to Caitlyn Jenner. When she clearly looked like a man, she should go into the men's room, when she clearly looks like a woman she should go into the women's room. It seems a fairly simple solution to me without having old women who are might not be able to wait go into the bathroom with scary dudes, or worse young girls.
I forgot the instructions on resizing pics but here is the transition, clearly able to tell when.