I need some advice

So, I'm 22 years old. I've been seeing this guy for a while now I'll call him Mr.X. Before I met this guy I recently got out of a relationship and wasn't ready to date. When Mr.X met me he was hooking up with another girl and the moment he met me he just had to get to know me. He instantly fell in love with me. He stopped seeing that girl and started to pursue me. He went out of his way to get my number and to just to talk to me. He instantly fell in love with me and he didn't even know my name. He said ever since he met me he's been wanting to be a better man, that I made him want to wake up every morning, that he's been starting to live. I thought this guy was nuts until I started to hang out with him. I started to get to know him and he waited until I was ready to get into a relationship. I finally gave him a chance and I ended up falling in love with him more than ever. I've never fully accepted anyones past or fully trusted a guy since my heart has been broken once too many times. Mr. X changed that for me. I accepted him fully and I fully trusted him. A few weeks back I got scared for how much I fell for him and I ended it because I didn't know what I wanted. It hurt him so badly. it didnt even take me a day to realize how much I missed him and how much I truly love him.We would see eachother every day, cook dinner together every night and go on walks, take his dog to the park or to the beach, we were always together and we were always happy together. We both were happy, we made each other happy. We got back together but Mr.X never fully got over the pain. A week after I ended it he decided to end it. He didnt know what he wanted anymore. He went from being crazy over me and madly in love with me to just not knowing. This also lasted about 2-3 days. On the 3rd day we decided to work on our relationship and go back to being us. He seemed off.. as if he built his walls up. A week goes by and he said it doesn't feel right and broke it off. He said he tried to make it work but a week isnt really trying. He said he doesn't know i'f he's supposed to be with me and doesnt know if he's with me bc he's afraid of being alone or not. (He always had meaningless sex with girls and never wanted to date them, never wanted to date until he met me). so I don't believe it. I feel like he's just hurt and is confused. He said he's a broken man and he's dumb for doing this and may regret it but he just had to tell me. He knows how much I care for him. How can he go from being madly in love with me and crazy about me to this? He says he loves me but that we don't need to be together. He felt as if he forced me into this relationship and that he's forcing this. I reassured him that it's not the case. He says it killed what we shared when i dumped him a few weeks ago and he just cant regain it ever since then. he stopped having faith in it.He hates himself for doing this but he thinks it's the right thing to do, and believes if we were right we wouldnt be doing this again. I feel as if he's conflicted and has fear of me hurting him again. His words "I'll regret this, i'm regretting it now, but its hella better than hurting you even more down the road i never wanted us to end we need to end it just feels wrong, maybe it's my own pain but you deserved to know" I just don't know what to do anymore. I fought for him he knows how I feel, I Just think he's afraid I'll hurt him again when it wont ever happen! I need help!