Do this instead.
Next time he contacts you, tell him you can't do this anymore.
Then cut the connection and block his sorry ass on all forms of social media and communication. Block his number on your phone. And don't answer the door if he comes a-knockin'.
He is seeing you as his go-to second whenever things are at all complex or unpleasant or boring with his wife.
You do not have to play his game.