married - affair with married man - my story and I need help!

Not looking for someone to tell me what I am doing wrong. I know what I am doing is WRONG. More so looking for people who have been in similar situations or are right now...looking for someone to talk to and hear others stories and how they ended up.
Ok - let me just first start off by saying that the affair that I am in is probably one of the worst possible scenarios possible for this!!!

My BF and I met at work. We have worked together for roughly 10 years. He is roughly 10 years older than me. Throughout the years at work, we have always been friends and have always been a little flirty but mostly because we both have flirty personalities. I don't even remember how it started about a year ago, we were texting (which we sometimes did, friend stuff) and somehow got on the subject of our significant others.

My husband and I have struggled since we have gotten married for some reason with lack of affection. I am not sure WHY this has happened because we when were dating and when I fell it love with him, it was like he was a completely different person than he is now. I am a very sexual/physical/emotional person and I feel like that is an extremely important part of a marriage. It's nothing though for us to go months without having sex and days without him even telling me he loves me. He has just turned into this guy who I feel I have no emotional connection with whatsoever. Honestly I feel like we are just roommates. I love him to death but I am just missing that little piece that should connect a husband and a wife. He knows that I have an issue with this. We discuss it all the time but he doesn't seem to try to fix anything. He just says "that's who he is"...not a very affectionate person.

Through our text conversations with one another it was somehow brought up that he was unhappy with his wife for the last couple of years.
Anyway once we started talking about our significant others we realized that we were missing the same things, so we had something we could relate to with each other. At first it was just innocent talking/complaining about the lack of love/affection/sex from our partners. I don't even know HOW it happened but one day after work we were both the only ones in the office and we were just standing there and he kissed me. It was like I felt fireworks!!! It felt like he knew exactly how to kiss me and it just felt so passionate.
Later that evening the more dirty texts began. Which was all it was for a while....we'd talk about how attracted we were to one another, send pictures here and there, and that was strictly it. We would sneak kisses in the office sometimes, and it felt great being so sneaky. It was fun!! We eventually decided to meet up one afternoon on a Sunday. It was AMAZING. This went on for several months. And in the beginning it was strictly pleasurable and we had both discussed and agreed upon that. We would meet up probably twice a week after work for about an hour or so and then most Sundays for a couple of hours. About five months into the affair, I began noticing that I was falling for him. And falling HARD. We didn't just have sex anymore. We had sex and laid in bed and talked for hours about everything. Most importantly to me...he made me laugh. He is one of the funniest guys I know and puts a smile on my face every time I see him. He made me feel BEAUTIFUL. He constantly tells me how pretty I am, how lucky he is, points out everything that he loves about me. And the kissing and sex with him is something that I have never experienced before. With my husband and the few others guys I had been with in the past, this was nothing like it. He was so passionate and it got more and more passionate as time went on.
Finally one day he told me he loved me. And I said it right back because I had been feeling it for a month or so. I fell completely in love with the man that I was just supposed to be having "fun" with and an escape from my boring marriage. NOW WHAT?! This is where I am so confused now. We tell each other we love each other every single day. We spend as much time together as possible. We are still falling in love with each other and we both want to be with each other badly. I text him all day every day and every time that I see his name on my phone I get so excited. When he walks into my office I get so happy. Most importantly in all of this, he makes me SO HAPPY. Something that I haven't had with my husband. I hate so bad to say that because it makes me feel like an awful wife. But I wonder, am I supposed to be with my husband? Or is this the man that I am supposed to be with? If we were together, would it always be this exciting and passionate? There is so much as risk here. Last week, I stayed with him one night when his wife and kids were out of town. He has told me several times that he wished we were together, wished we were married, wished we could be happy together. He tells me that I am the girl that he has been looking for all his life. But that morning he didn't want to let me go and he looked me straight in the eyes and told me that he wished I was his wife. This simple sentence sent cold chills down my body. This was what I wanted. I WANT to be with him. I miss him when I am not around him and I constantly think of him and how happy we are when we are together. I have told him recently that I want to be with him so bad but it would cause such a mess if we tried! Not to mention he has two kids and I do not want to split a children apart from their parents.
I am just so confused! Does he REALLY want to risk everything and be with me? I am nervous.....so nervous. what if I let my marriage to be with him and it was nothing like what I thought it would be? Would our age difference be a huge factor later on down the road??? sooooo many questions I have for myself and it drives me crazy! But I am so head over heels in love with that man.