@Raspelle,
You've got to enter this relationship with trust. If he lets you down so be it, move on and find someone more worthy.
A lot of people are friends with their ex's or have a brief relationship that doesn't work and remain friends.
The key to me of this, is that, you stated " he occasionally hangs out with guys he has slept with". That to me suggests they were just there at that time nothing serious in it.
However, having said this what do you mean by his past sexual encounters? You make it sound like there were "many". Perhaps as single there were, perhaps in a relationship he's faithful. All you can do is 1) ask him if he will be and 2) ask him to be or leave as you don't believe in cheating.