My man says I'm too selfish

So my man and I have been dating for 4 1/2 years, 2 of those years we have lived together. We have argued a lot and thought we resolved our issues, but it has now come to me being too selfish, which I can admit I have been, but I have been really focusing on where I get selfish and how to not be selfish, and every opportunity that is given to me to make it about him i shut it down due to my anxiety level of uncomfortablity. I just came to that realization a few days ago. I went to church Sunday and was invited to Wednesday bible study and Thursday woman's group but Wednesday is my busiest day running around with laundry and groceries, yet I went to church between doing those things, mean while I told my man my plans for the day and he did advice me to not go and to just go on Thur so I don't waste gas or my energy but I told him I'm not doing this every week just this week so I can get the vibe of the groups. I come home Wednesday night and he comes home from work and he is mad at me for going to bible study even though he advices me not to, which in his eyes he sees that as me not listening to his advice which in the past I havnt but I told him I was going anyway just this week, so he sees that as me being SELFISH ,
Another way he sees me as selfish with my anxiety is when we first met I mentioned I wanted a threesome, which I did but we kept talking about it and he felt like I was never going to go out and find a woman for us so one day he found someone and the second time he did and the third but now he is mad at me cause I never kept my end of the deal(which is another issue we have) with finding someone. So I have been putting my postove energy out and at the gym there was this girl who I have seen before and she has an eye for my man, great no problem, so I was leaving the locker room to leave and the girl was there on a machine and she jumps up and slaps her friend next to her and says "that's her! The girl with the boyfriend!" And I clearly heard it but I walked away cause I was already in motion of leavein and she was talking to her friend not directly to me, now. I know I should of stopped and been like oh are you talking to me? And what not, and it's obvious she was interested, and stupid me told my man this and now he is pissed cause I took an oprotunity that had to do with him directly and made it about ME and MY anxiety and how I DIDNT feel comfortable instead of standing up and being like oh your talking about my man? Well what would you like to know? Which I agree I did but he also is sick of my anxiety which I'm trying to control. So now he is pissed cause a situation that was directly about him I made about me and my anxiety and for a situation that is me not listening to his advice which are two huge issues in our reltionship..... HOW DO I NOT BE SO SELFISH? Or AM. I BEING SLEDISH AT ALL?