I love him but is it worth it?

So I have been with my boyfriend for about 18 months we haven't had any problems or even arguments. But there are a few things that bother me.
1. we have maybe actually had sex 3 times.. we do have oral and masterbate together a couple times a week. But I feel like I'm missing something am I? I have asked him why he doesn't want to and he says he's not a sex person and that it's to much work.

2. He works with his ex. No they dont have anything romantically together but they still are good friends text and every now and then we go eat with him and who ever he is with when they get off work. We started dating about 4 months after there break up. Am I a rebound even tho we have been together 18 months? When I bring it up he says his ex will always be part of his life and if I cant deal with that I need to go. Am I being selfish?

3. I really don't enjoy where I work and we live in a small town and I want to move. One day he does want to move the next he doesn't.