@gbklyn9,
Let me say this as clearly as possible.
You need a lawyer. Your situation is more difficult than most. I am sure there is a way to work this out in a legal way, but there are all sorts of things that could go wrong (some of which could have some pretty bad consequences for you).
When I went through a child custody issue, I payed a lawyer some amount (I think it was $150 but I don't remember now) for him to answer my questions for an hour. This was worth any penny... make sure you write down your questions before to get them answered. If you don't pay a private lawyer... all you will get is a sales pitch, but if you pay them they give you valuable advice and you can decide how to go forward.
If you can't afford a lawyer... go to a family law practice, many of them will work with you for either free or for a very low cost.
I don't think you want to do anything in this situation without talking to a lawyer first.