Your partner has no legal rights to stop you from moving to be with your family, or to stop you from moving anywhere else.
You are already thinking like a responsible parent by wanting to put yourself in a living situation where your child will be adequately supported, and you can only do that if you are not supporting your partner as well. And you are doing this, not because you want to, but because you need to, and that shows you are already placing your child's welfare above what you might prefer.
Your partner needs to grow up if he's going to be a responsible parent as well, and part of that is demonstrating an ability, and willingness, to provide financially for this child and for the child's future. His days of being a freeloader are over, given the fact you can no longer support both him and the child who will soon be in your life. Your main obligation is to the child, who has no choice about their dependent status. Your partner, who is apparently not disabled, will have a choice to make--either he finds and maintains steady employment, to help ensure the financial security of both you and your child, or he risks losing having both you and this child in his life.
At this point in time, I don't think your partner has either parental or spousal rights. You are a free agent, and your fetus is part of your
body. Move wherever you think will be the best arrangement for you.
There are no parental "rights" that do not come with considerable parental obligations and responsibilities. But, I wouldn't worry right now about what your partner's future legal rights, regarding this child, might be. I'd tell him you're moving to be with your family now, and why, and let him know he's got 6 more months to start proving he can be a responsible parent/adult/partner, including a financially responsible one, who can maintain consistent full-time tax-paying employment, if he wants to resume living with you. He's got to get a job ASAP, and pay his own bills, and start putting money aside as well. If he does that for several months, then, and only then, can you start discussing any future arrangements between the two of you. If he chooses not to do that, well, then he's made his own choice, and you haven't taken anything away from him, he's chosen to give things up. It's not just an issue of his earning money, it's about his ability to function responsibly
I think it's important that you aren't considering your options, as an independent person, and a soon-to-be unwed single mom, because you are angry at this man, or dislike him. You've made it clear you love him, but you just can't rely on him to function responsibly, particularly after your child is born, and that appraisal seems entirely justified. Maybe your moving out, and giving him an ultimatum about the kind of responsible behavior you want to see from him in the next few months, will be just the kick in his pants he needs to help him grow up.
Having a baby is going to change your priorities, it already has, and, if he wants both you and that child in his life, his priorities are going to have to change as well. If he can't do that, before this child is even born, then consult a lawyer about whether to even put his name on the birth certificate, and what rights, if any, he might have regarding the child after it is born. But, I'd first give him a chance to prove himself in the next few months. Maybe the thought of being a father, and having another life totally dependent on him, will change him. You never know...