Sounds like you and your husband are not in the same place sexually and that you have different expectations of relationships.
He is looking for an exclusive relationship, which you are unable to provide for him.
He wants a wife who will be faithful to him.
While I understand that you want to stay married to him, it's not really fair to him. If you really love him, you should think about letting him know straight out that you can't be faithful and will give him a divorce so he can find a wife who will be faithful to him.
Do you have a job that you can do to support yourself after the marriage ends? you should probably do some serious planning for the future.