No, you're not paranoid - chances are he's seeing other women. You caught him seeing 2 girls at the same time, in addition to you ? Yes, I say his sex drive is not compatible with yours, clearly.
I have no idea why you don't do oral sex, it's part of love making and you mention that additionally, you don't have a very active sex life due to health reasons. I don't want to prey, but it seems to me that you should have your hormone levels checked.
I assume you're quite young and having a low sex drive at that age is kind of unusual, so there might be either a medical reason for it - or a psychological that you don't want to have sex with your current boyfriend. Maybe his infidelities hurt you more than you care to admit, or it is another reason, but it certainly is not a healthy situation and my advice to you would be to look for someone else who is trustworthy, honest and more compatible to you and your needs.