I don't want to live with my in-laws. Am I wrong for feeling this way?

Me and my husband just got married yesterday. Before we got married we both moved into his grandmother's house to stay until I could find a closer job and we could save up money and find a new place to live. At first I was hesitant about staying with my husband's grandmother because I felt it was inappropriate since we were only engaged and not married at the time. When I first got there I was extremely uncomfortable. I have only known his family for three months and would only spend a few hours with them until now and I'm starting to notice and see characteristics of his family I don't like. We had only been staying with his grandmother almost four weeks and it was already a living hell and today I got fed up and left my husband at his grandmother's house. It was nice at first until his grandmother started trying to control us, meddle within our marriage trying to cause arguments and crossing her boundaries. What makes it worse is that my husband is a very strong mommy's boy so if his grandmother yells at me and disrespects me he will back her up and take her side at all costs just to keep the peace even though she's in the wrong. When we're talking about personal issues that should be between us he'll get his grandmother involved everytime. I am tired of having to compete with his grandmother and all the rest of his family. I feel like he's not putting me first. It upsets me and makes me feel a certain way because I've been nothing but kind, polite and respectful towards his family. I was starting to become so stressed, depressed and overwhelmed being there trying to make a good impression upon his family. I know my husband can't help it because he has a physical disability which he can't stand for more than four hours and has ADHD on top of that which prevents him from working and finding a job. He gets SSI but a large portion of the money goes to his grandmother and his mom. He gets 750.00 a month 550.00 goes to his grandmother for her rent and bills and he gives his mom 160.00 a month on top of that. I feel like his grandmother babies him and wants to keep him codependent on her and as a result I feel like he hasn't cut those apron strings yet but he is a grown man now and has lived with his grandmother 24 years of his life. His grandmother is his payee and controls all of his finances which doesn't give him much wiggle room. I left and am staying with my grandfather until I get myself together save up and find a place to live my husband says he is still staying there with his grandmother and is not leaving there. My husband says he is confused and feels like he's being pulled and tugged in between two places(specifically between me and his grandmother). I feel like my husband is going to be stuck there and live with his grandmother all his life. It seems like he has no intention on leaving his grandmother's anytime soon. His words and actions aren't matching up. He says he's going to leave but says he can't because he thinks he'll get cut off of SSI and put his grandmother in jeapordy. Ever since he's gotten to his grandmother's he been becoming more lazy and his grandmother doesn't say anything and encourages it and coddles him to death. My husband will stay up until almost three in the morning playing video games and wont wake up until almost 3 in the afternoon if I don't wake him up in the morning and tell him to wash up. I feel like me and my husband are on two different pages here. I'm tired of feeling like I'm in this relationship and marriage by myself! Doing EVERYTHING by myself. I get tired of motivating him I would like to be motivated sometimes too even though I'm very driven and focused. I have a two year old son from my previous relationship and my husband's grandmother hates it. My son is getting to the age where he's starting to say words and copy what people say. When my son calls his stepfather "daddy" my mother in law will tell my husband my son isn't his kid and has no right to allow or tell him to call him daddy. I don't know how this marriage is going to work if me and my husband can't come to an agreement on what to do.