@caligrl,
Clearly (now), he's emotionally stunted or emotionally unaware of the effect on others about what he's sorting (or sorted) through in his life. Some might call it signs of narcissism on his part. He was cheated on by her...and then ... in some direct or indirect way, ... he cheated on you with her.
He isn't the person you thought he was and that advertised himself to be. In a few short words, he has lied to you and manipulated the relationship. Don't participate in his life drama. Stop the dance.
Sad to say, but it's time to move on. I don't think it'll be easy, but heal yourself the best way you know how and get this person out of your life and any contact whatsoever. Delete and/or block any contact.