If you have been separated from your husband for over a year and you have family court proceedings in place or you have seen a lawyer to begin proceedings, then seeing another person should not give any detrimental effect upon those proceedings. Your decision to not have him stay over is a good one because then there is no 'come back' from your husband when it comes to the children or what they are being exposed to. You will be required to see a counsellor prior to your divorce hearing, which is purely in an effort to either sort out marital problems, or create an environment where both you and your husband can be 'adult' about the situation. The very worse thing that can happen is for the children to be used as 'pawns' in a 'tit for tat' emotional blackmail game. I would ask my lawyer what the ground rules are in your situation and stick to them. Your husband sounds hurt and angry. He would not be granted sole custody based upon threats. My advice to you is to keep a detailed record of what your husband has threatened and also your responses, the time and the date, just in case this turns nasty. This is advice re Australian legal system, not sure what happens over in the US.