How to let go 4 years after an affair

I had an affair with a married man. I know that this was probably one of my biggest personal let downs of my life so far. I don’t know exactly what I was thinking getting into and then staying in this situation for nearly a year. There is absolutely no way that in my sound, logical, mind that I would ever consider having an affair with a married man and the thought of others doing it repulses me. And yet this hangs over me like a ton of bricks with guilt and a significant amount of shame. I struggle to look back at this time and feel compassion or understanding of myself. I just look back now with so much anger towards myself that I let myself get in that position and took it for so long. Although there were reasons to continue to see him after he told me that he was married I can’t justify my actions with any of them. I want to move on from this pain so that it doesn’t continue to have an impact on my marriage. I have become so suspicious of my husband doing it to me that it’s consuming me. The irony is that the affair happened before I met him, while I was single, but because I still carry the burden of shame it’s seriously affecting our relationship and my health now. It’s like I am continuing to punish myself long after it’s happened and I want to stop. Any suggestions on moving forward would be appreciated.