Had a past with my bf brothers, I'm scared to lose him, help???

Okay so when i was younger i didn't really have the easiest childhood, i had a very unhealthy relationship with my dad and i was bullied for 3 and half years by an entire year group of ONLY GUYS. during this time they detierorated my self esteem and ability to communicate and deal with any males in my life. i was about 11 when this happened and it lasted till about 14. my current boyfriends brother came into my life right after i got bullied, he showed me attention and affection that I've never gotten from a guy and he made me feel like i worth something, this id never felt for quite sometime of my life, i never felt beautiful or anything worth considering a partner. his brother manipulated in very much, gave us a title as bf and gf and it made me feel so good, but i was naive and he was much older than me so he knew how to take advantage of me. he forced me into giving up a lot of sexual things to him that i had never experienced( btw i was 13 he was 15) he forced me to give him a blow job and would blackmail or guilt trip me about being a horrible girlfriend if i didn't do what he wanted. basically i give a glimpse of what he was like to me. then later on at around 15 years of age, around the time my bf and i really started getting to know each other as friends, i started hanging out with his oldest brother whom i never had any feelings for and only saw as an older friend. he would also make me feel good about myself, compliment me and tell me how beautiful i am and that he would date me if i was much older, in the end we got intimate and made out and he also fingered me. now his brother felt really guilty for what he had done and he quickly cut it off, i agreed cause i was so guilty for what i had done. i had been with 2 brothers. i liked my bf for sometime before we dated right from the age of 13, but he rejected me and in future got another gf, i moved on in hopes of finding someone else unfortunately is was his bros who came into my life and knew exactly what to tell naive girl. when i turned 16 i had forgotten about all the stuff i did with his bros and turned my life around, i earned some respect for myself and morals so i could never circum to the mistakes i did in the past. at this time i began developing really strong feelings for my bf like REALLY STRONG, we were so young before and i had puppy love on him, just a normal crush, but i always felt like i loved this guy. i honestly never in 3 billion years expected to fall in love with him and now that i fell so hard to knew i was fucked cause of what i had done. I'm 18 now, me and my bf have been dating for almost a year now a and i confessed all this to my bf about 3 0r 4 months ago. of course this news hasn't been easy to deal with, but he's slowly getting over it. it still bothers him at times and this upsets me so much cause i ******* regret it with all my life!! I'm so afraid to lose him i always have been thats why i kept this away, I've been embarrassed and ashamed for my actions. I've never even told anyone of the things that happened with his brothers cause i can honestly say those are the biggest failures i ever did in my life and i wish i could take it back but i can't. Look i love this guy so much you guys! please i need some answers that will help. I CANT LOSE HIM!