Difficulty Coping with Idea of Divorce - Kids, Indecision, Infidelity?

I'll make this quick... my husband and I have been married since 2005. 3 beautiful kids together. When we first got together, I was coming off of a bad, abusive relationship. He had pursued me for years prior, was dependable, solid, steady, a great companion. In short, it was easy. I was never attracted to him, but knew he would be a good companion. He is a very good man.

Over the course of our marriage, we experienced our ups and downs. We never fought. We basically lived two separate lives, more as roommates. Focused on our kids.

About 1.5-2 years ago, he went through some major life changes. I did my best to stand by him but I felt there was so little left between us. Then I met another man and what I thought was a quick fling, developed into something more.

I fell head-over-heels for this new man. After 2 months, I told my husband I wanted to divorce. He said he didn't want to, but didn't protest. All he could say was "I'm really going to miss you." I moved out. We split custody of the kids. My relationship with the new partner takes off and it is the most amazing thing I've ever experienced on every level. However, I can't cope with only seeing my children 60% of the time. Child exchanges rip my heart in two. Christmas day and holidays were the worst ever.

Then guilt comes in... my husband makes MAJOR changes to his life. He doesn't want to divorce. He is patient and willing to work on anything and everything. We start to attend couples counseling. It really hasn't done much for us, but it's more symbolic than anything.

Recently, I moved back into the house because I feel it's the right thing to do. To give it one more try, for the kids. My husband is great. Patient, caring, a great dad. We are almost instantly back in our same patterns. Sharing a home, sharing kid duties, but not really "together." We made promises to try to have more date nights, to communicate more, etc. That hasn't happened yet. We are not intimate and I have no desire to be.

But I've never stopped being with the other man. I've tried to end it multiple times. We always find a way back. I love him madly. He wants to adopt my kids and to have more kids together. He is everything I ever wanted in a partner and can provide the life I always dreamed of for me and the children.

Only problem with the new man is... he is not their father. I just don't know if I can cope with being divorced. With the impact it would have on the kids, and my relationship with them. I want them all the time, and I know that can't happen. But how can I live the rest of my life bored and unfulfilled with their father?

Please help. Share your story with me and how you decided. Or at least tell me I'm not alone. Thank you!