Long term relationship advice

I have been in a long term relationship for 4 and a half years now since I was 18. He is my first and only partner sexually or otherwise. We usually don't fight and get on great. We are a chilled out couple and aren't really lovely to each other (me and him both).

However him and my parents don't get along which is fine I'm ok with that, after all it's more their fault than his. But recently he is just taking everything out on me or I just feel that he is disinterested. I should say I moved about 2 hours ride away from our home where he still lives and has his own business about a year and a half ago.

Because of the friction between him and my parents he has decide to distant himself from all my family gatherings, which I agreed to, but it is very difficult also. So how it all really started was I asked to a family gathering and we had this huge row about my family and how I should respect that he wants to stay away from them because they were so bad to him, which was true but I'm also the kinda person who doesn't like going to these things by myself so I was annoyed.

So anyway we got over that and decide to just take it one at a time and that I would have to respect his decision. But since that everything I do he is pushing me away. It's like it opened up a wound, I just feel he is all the time making smart comments to me, or anything I say he disagrees with. Maybe I am being overly emotional or something I don't know.

But then it came to my birthday weekend and I told him I didn't want any presents but I wanted to go out and have dinner and a few drinks. By a few drinks I meant both of us enjoying ourself maybe getting drunk and enjoying ourselves. I will add that we don't go out as much as I'd like so this would be a treat to me. So birthday weekend he took a day of work to spend with me we went shopping and that night went for dinner but went home after to his parents house like we do every Saturday and watched tele. I know I probably should have been more clear and said I want to stay out and enjoy ourselves but I though he didn't want to. But anyway we went home, and I got a bit moody because we were back to the same situation watching tele with his parents not even talking.

So he knew I was being moody but claims he didn't know I wanted to stay out for the night. So the next day he didnt talked to me the whole day, then I went to go back to where I work and we had a full blown row because I was annoyed he wasn't paying any attention to me all day and he just I should should of just told him what I wanted to do and be more direct!

Now I love him so much and was evening planning our future together like trying to save so we could buy a house some day. I don't know if anyone will be able to help because it's all very confusing, but was I just being crazy or should he really step up and make more of an effort with my birthday and everyday? I'm just feeling very unloved this weather. It is kinda important because we are fighting that much now he says he is getting sick of it and I'm afraid if it doesn't change it's over.