I was the other woman and I'm trying to move on

So I was with a man who was married for 10 years with 2 kids.
He told me he lost interest in his wife and wanted to have a place where he could feel passion and excitements, ended up he found me.
We had a lot of laugh and jokes and also shared a lot of precious moments. It was happy and I fell for him.
I'm married too for 6 years with 2 kids, although my marriage is unhappiness, I wouldn't call it the reason of seeking something else out of marriage, an affair is never the solution for a marriage. And I truly feel sorry for my husband.
I broke up with him last week, our relationship only lasted for about 2 months.
I ended it up basically because I couldn't stand the loneliness.
But I have been thinking about him a lot and it's hurting me.
Somehow his wife got some information about our affair and now they are going through marriage counselor, I don't know details since we have cut off all the contacts. He told me he would take his responsibilities to make sure this doesn't get back to me. I'm thankful for him.
I know thing could get a million times worse if my husband found it out. I would be done.
I'm willing to stop and move on. There will be always a stain on my soul. I will live the rest of my life in shame and guilt, for the pain I have contributed that the people(mostly his wife) have been suffering now.
But the relationship between me and his was so wonderful and I can hardly bare to lost him.
Is it possible that he and I become friends in the future when his marriage is fixed (the surface at least) ? I know it's impossible me and his wife get to know each other but I would really love to have him in my life.
Please is there any suggestion ? Couple days ago, I was desperately hoping that he could come back to me to continue the affair. But I realized it would only deliver pain to everybody and I cannot hurt anybody anymore.
But for now, I still don't feel like banning him completely from my life.
Please help ! Any suggestion / advise is appreciated !!