Who's in the wrong?

I have been friends with this guy for about 6 years now...He has always been a really good friend of mine. About three years ago we both went through difficult breakups with our bf/gf and we could relate with one another about how great/how much it sucked to be single again. This led into a friends with benefits situation that has lasted on and off about two years. I always made sure that we were on the same page by directly asking if he ever wanted to be more than jus friends with benefits because I did't. He told he he would be honest with me and always told me he never would want to date me. Perfect, we're on the same page. But hanging out as best friends + sex that long made me want a boyfriend that much more..but not with him. I could never see a life with him but I was beginning to see him like that so I told him about it and that we needed to stop cause I knew he didn't feel the same way. And I wanted to stop before I got too invested emotionally. Although he said he respected my wishes, he kept trying to come over for the benefits part...it made me so angry. Eventually he admitted he had developed feelings as well and it was hard for him to let go. So we took some time apart to maintain the friendship. A few weeks go by and I have friends over to watch GoT and he comes too. Fine, I missed my friend. But after everyone left, he stayed. He tried to initaite sex but I told him no. He got angry and told me I was a slut. (I went on two dates the month prior. Trying to move on from him...) I was angry that he called me a slut. It was disrespectful and only said to hurt me probably cause he felt hurt I denied him. But that's no excuse. I've never tried to hurt him and this isn't the first time he's done this to me. So I kicked him out immediately and haven't talked to him since June. The problem is we have the same friends so running into him is inevitable and I hate it... I wish he would just apologize to me cause I don't feel like I have anything to apologize for. He really hurt me and he knows it. Meanwhile, my friends don't care. They think its our problem since this is between us. They're right but I wish someone would say something to him...should I? Idk. At this point I almost want nothing to do with him anymore.