Can I trust my boyfriend?

In short, I'm with my boyfriend for about two years and we live together almost from the beginning. I think we are good couple, I can trust him that when I need him - he will be there for me. We care for each other and show our love not only in words, but in actions. However, lately there is a distance between us. On the surface, our relationship is good, because we don't argue and are affectionate to each other, but we don't talk, at least on some serious topics. Ealier we were open to each other about everything. I know that current situation is my fault and I'm afraid that he is preparing for a break-up and tries to distance himself. Lately we had a few big fights - all of them were caused in short by my lack of trust and I don't believe in what he is saying. I know that trust is basic thing in relationship, but I was really hurt in my previous relationship. I thought I'm over it, when I entered new relationship, but maybe I was wrong. My previous boyfriend dated someone else for a few months when he was with me and treated me like ATM, which takes him for expensive holidays. I know that my current boyfriend is not responsible for me being hurt and he is not my previous boyfriend, but unconsciously I search for hints that he might hurt me. My current boyfriend is outgoing guy, with lots of friends of both sexes, he can make everyone laugh and is really charming, I know that he is popular among girls. And I know that he probably is sick of me and my insecurities, and I want to overcome them. But can I really trust him? I will describe closer our one last fight, I want to know if I should be really worried or I'm just insecure. So, some time ago my boyfriend went to a party at a friends' house. He said that he will be really late, so I shouldn't wait for him, but I didn't worry, because he rarely goes out with his friends, not because I'm controlling him, but he is often busy at work or university and their also have their responsibilities. I fell asleep and woke up, when he came back - in the morning next day. He was in awful state, so I just let him get some rest. But I was little worried that he came back in the morning. Next day, he explained that he slept at his friends house, because he got really drunk and fell asleep in one of the rooms, while talking to someone. He claims that immidiately when he woke up, he came back home, when everyone else was sleeping at his friends' house, because he didn't want to worry me. I understood that, it was better at his condition to sleep at his friends' house, it was just safer. However, he has some holes in memory of that night - he said that he wasn't that drunk, but just really tired and he remembers everything until he, obviously, fell asleep and woke up in the morning. Nothing suspicious in all that, until we recently met his female friend, who was also there, at the party. According to her, she was in the room where he was sleeping in the middle of the night and tried to wake him up - she explained that she was drunk, looked for another friend or maybe some place to sleep and she only wanted to ask my boyfriend if it is ok if he will be sleeping not in our apartment. I don't really trust her that she tried to wake him up only to ask him about that - she wasn't fond of me from the beginning, few times I had a feeling that she is interested in my boyfriend (she tried to impress him, touched him from time to time) and she is known for creating drama, by sleeping with taken guys - she have to be in the centre of attention. The problem is that my boyfriend doesn't remember a thing about her waking him up, according to him it is because he was in the middle of the sleep, not because he was blackout drunk. However, I'm worried that something happenede between them back then, but he simply doesn't remember. He says that he woke up alone in his bed and he would remember if something would happen - he is really sensitive to touch when he sleeps and would immidiately react if she would try something with him. According to him, he probably confirmed that he can sleep there, but was not fully awaken and that's because he doesn't remember the fact that she tried to wake him up and then she left the room. He claims that he wouldn't even think that something would happen back then until I talked about it and she would mention it to him, if something actually did happen. But I'm really worried about it - that what I don't like about myself, because I don't know if I should worry, but my mind goes: What if he/she does remember what happened, but keep it a secret? What if he wants to use me? (I can't confirm if something happened between them, because he doesn't talk to her anymore and I'm not that close to her to ask such things, at least he says that he doesn't talk to her, but maybe he don't want to ask, because he is afraid that something happened?) I hate these thoughts, because I accuse my boyfriend of awful things, like the one, which I previously described. And my boyfriend is sick of it, because as he claims he is not doing anything wrong, he cares for me and I think of him really low, and he is not a cheater and a liar, but normal boyfriend, who loves his girlfriend.
Should I be worried? I don't know sometimes why I think so badly about my boyfriend, but it ruins our relationship. Maybe it is my gut, observing little, unconscious things he is doing? Or just my insecurities? I want to trust my boyfriend or let it go, because right now we both are unhappy in that relationship.