relationship advice PLEASE help

Hey guys I need relationship advice I'm afraid to tell anyone because I will be judged. At the beginning of this 2015 year I talked to this guy who is 31 im 22 btw.I've known him since I was probably 18 when I graduated I started working at a pizza place and this guy would always tell my older sister who also worked there at the time to tell me he said hi. She'd tell me and of course I'd get really happy I thought he was very attractive, I'm attracted to older men just to clear things up. Well I never thought anything more than just the hi because he was known as a drunky and he has a horrible reputation . My sister said he would always be drunk and he would use drugs and I belive her she's my own sister. Well me and him have always had a thing for each other i could simply tell jusy by lookong at each othet but never pursued anything till at the beginning of this 2015 year. The only reason I ever contimplated in talking to him was because I didn't see him as this bad person people were panting him to be. I knew him as a very talkative respectful man sine the first day I ever even said hi. So we started talking and at first I didn't know if it was just physical attraction but when I started to get to know better I really thought he was such a sweet caring talkative funny attractive man, someone I could potentially spend my life with. As we continued to talk I explained to him my concerns being that he had criminal records ( he's gotten arrested for DUIs, possession of narcotics, he use to use drugs as well, and he would drink a lot too, I believe he had alcohol poisoning as well I'm not sure my sister told me someone told her, and he also has battery charge against his old wife) he said to the battery charge there is more to it. He said he grabbed her by het hands bevause she was teying to hit him they were arguing and yelling. The sister of the wife heard so she called the cops and that's how the got charged for that. I believe him in a sense because my brother go ot charged with the same thing simply because he smacked a phone away from his wife's hands. Well anyways that's how that happened. We talked for a bit but at the end I stopped talking to him because I was afraid to tell my family and them judging me and calling me a dumbass and telling me what am I thinking which I would completely understand why and where they are coming from. So when I stopped talking to him he went to jail he said he had to take care of an old charge he has, he Sai he didn't know he had a warrant after him till one day he got stopped and they told him, so he served about 5 months in jail for an old charge. During those 5 months I dated this other guy, things didn't work out. When this older guy got out we started talking again he keeps telling me the only thing he is asking from me is to let him prove himself to me to let hid actions show me that he is not what everyone claims him to be. That the stuff he had was from the past. I believe him and I want to give him a chance but all my family knows him to be with bad person. I talked to my brother and he said I was a dumbass that why would I want to associate myself with him. This older man also got shot when he was 20 he said he he best up this guy really bad and they went after him. Well anyways the only reason why I want to give him a chance I'd because I really trust him I'm good at seeing through people and I honestly believe he's not as bad as what his records make him look. He stopped drinking and he goes to church. He has an 8 year old daughter . I find me and him to have so much chemistry I really want to give him a chance but I'm terrified to tell my family since I know I will be judged sooooooo much and I would understand them. What would u guys do? should I stay away from him ? I just hate the fact that the first time I stopped talking to him was because I was afraid to let my family down and tell them and it's happening again. Idk what to do I'm afraid to give him a chance and be let down and my family throwing it in my face of what a dumbass I am but then I don't think I will be let down by him. I got to know him.more and I love the way he is. But I'm so torn and I have no idea what to do