All affairs are based on fantasy; you fill in the rest.
I was in an affair with a married woman for 12 years; but it only survived b/c of a bubble called "fantasy". When reality hit us, and im' not talking about being found out or any of that drama... reality that is us... our relationship could not hold up.
Affairs are the escape from the reality that is our relationships. No matter when people say they're "happy" with their relationships, it's not really true. If they were truly "Happy" in their relationships, they wouldn't need to have affairs or be vulnerable to them.
Just b/c someone doesn't want to leave their marriage b/c everything but this one thing is missing... doesn't mean they're happy, and when they have an affair with you, it's b/c they're looking for an escape... sometimes temporarily, sometimes permanently, but even the ones who say they want to permanently, is only drawn to you b/c of how they feel inside their reality.
as for you, dear one... the sex starts in the mind, it's your biggest sex organ. If you're really curious why this man has given you the best sex of your life... other than that he might have good technique, which, btw, isn't that uncommon... plenty of manwhores can do the same... on a mental level, you need to seek therapy and uncover what is about this guy that triggers that feeling within you.
that is, if you really want to know... most people prefer to float around and stay blissfully unaware and ignorant of your true self and hence fall left and right depending on the circumstance, event or whatever blows their way.
good luck to you.