It doesn't matter if his story is true or not. What matters is simply whether you can be married to someone who did the following.
1. Went out drinking and supposedly got drunk.
2. Randomly ran into a woman who he had hooked up with in the past.
3. Invited this woman to come over to his place simply because she needed to be around someone. (Wasn't she around all kinds of people at the bar?)
4. Drunk or not, he obviously invited her to sleep in his bed with him. Why? No couch? If he did not want something to happen, why didn't he either sleep on the couch or insist that she do so?
Now if you want to continue the relationship with a man who would consent to all of the above happening, then great. Whether he consented or not is irrelavent. Forgive him and move on. Since this happened 3 years ago, if he has been faithful since then staying together may work.
You wanna bet that if the story was reversed and you were the one bringing home a man that he would find your story a bit tough to believe. Just saying.