Intercepts and slopes not matching after removing non-significant interaction

If you got two different slopes when you ran the separate linear regressions, then sex*time is significant. If you got the same slope, but with an offset between the curves, then sex is responsible for the offset and time is responsible for the slope and there is no interaction. If the slope with time is essentially zero, then time is not important, just sex and there is no interaction. If the two regressions are the same, then only time is important, sex is not and there is no interaction. One problem you might be running into is that time is likely the big player in all these regressions, so if you map against sex, time, and sex*time, if might look like time is a huge player and sex*time is not. Also, don't put a lot of emphasis on the regression intercept. Depending on your data set, the intercept could be a long range extrapolation and insignificant differences in your slopes can result in huge differences in the intercept.