Of course I don't know how bad it was. I was not involved. lol But, if it was as bad as you claim, you should have simply left your husband. Instead, you stayed in the marriage (so how bad could it really have been) and then began an affair at some point with a married man, who I am sure was also in a terrible marriage and treated very horribly by his wife, but like you, not so horribly that he got out.
Neither of you have any regard for your marriage vows, so it will only be a matter of time before one or the other is cheating again. Of course, all this is simply my opinion and thoughts. You can take them for what they are worth. Won't hurt my feelings any. lol
Do have a good day.
EDIT: I do give you credit for leaving your husband the second time when things started back up again instead of simply re-starting the affair. If this is what you want, then great. We all deserve to be loved. It is just the whole cheating aspect that I disagree with. And I do still think that when you get together with a cheater, I don't see how you could ever trust him. If someone is willing to cheat once and disregard their marriage vows, they will be willing to do so again. I do wish you luck.