It's not a good idea to threaten when you don't mean it. It can backfire on you. And it did!
His unwillingness to talk to you about the relationship, to assure you that, in spite of his job and family issues, you are the most important thing in his life, and his low sex drive all combine to be real issues for you.
The 30 year difference is huge and the gap - especially sexual- will only increase. Are you willing to accept that?
He's actually pushing you away perhaps in the hope you will leave first.
Lots to think sbout.