@mynewlife,
'Course not.
You're both separated, I take it (e. g. why would you be living together?).
So, two people who were basically free, started living together.
Then they weren't living together.
Yeah, the circumstances and the start of it were not the best (understatement of the year). But you were doing what you thought was best for the baby. And then the baby didn't make it.
I get the feeling the grief about the baby is turning on his feelings of guilt about the affair.
I recommend counseling, mainly because you recently lost a baby and that tends to be a good time to get counseling anyway. And talk about this, too. Frankly, I think he could use some counseling as well.