Fame does alot of things to people.
Makes them feel they are invincible. He married a "nice" girl, good looking and had kids... She wasn't stupid.. Maybe, he thought she was.
Loosing her, was nothing. The media attention was everything. Loosing his kids was something he didn't expect.
Stands to reason, no matter how many women Hawekye he "could" continue to screw, he's lost his respect from people. He's lost the right to see his children every morning, every night.
He's very egotistical.. So, that fall from grace, would be near impossible to get back up from.