@Nik05,
There is a whole lot of things missing in your life screaming at you for that attention that you are seeking and so you are seeking, that.
There is more than one thing you are seeking, emotional connection, mental connection, sexual chemistry and experimentation of sensuality.
Can one man provide this? Possibly. Can a married man provide this? No.
Your husband can't provide it, a married man can't provide it so what are you going to do with the rest of your life? Jump to the next guy and stay with your husband? Finding that he also can't fulfil all that is missing in your life?
Or get real and realise that you are in a sexless marriage, emotional and mental and sensual "less" marriage that is a complete waste of your life, unless you can look deep inside as to when your relationship started and note that a lot of that was actually there at the beginning, just got lost, work on regaining it, seek professional help, or simply if that's not the case, walk away and find yourself and a partner that is single.
You know full well that he is giving you some glory, (the affair man), by being in a cushy position financially, some power therefore some power over you and in that, all his way, you are not his first, nor his last it's easy for him. He'll ditch you before you ditch him due to the neediness of emotional and mental stimulation, which means constant communication, he's not that great in bed so you know, there is a component missing that you are already missing.
Go it alone... Find a connection that you are after with someone single and enjoy the rest of your life and at the same time, his wife doesn't get cheated on and your husband gets to hopefully find a lovely lady that he wants to have sex with .. Win / Win.