@Lifeisnteasy,
Are you all quite young? it seems odd to me to think of adults packing up and going to their parents.
In any case, if someone moves out, they need to realize that it could very well mean the permanent end of the relationship. Adults don't move in and out when they're fighting.
I think it's reasonable of your boyfriend to point that out to you. It's helpful to you to realize that he's at a different relationship stage than you are.