You should see a lawyer right away.
The fact that he is not paying child support is ridiculous. Your son deserves to have money, and you deserve it too (since you are paying for everything).
I don't think there is much you can do to force him to spend time with his son, but you can get him to pay for childcare so you can have some much needed time for yourself. Tell the judge about the crazy schedule, and the solution may be for him to pay more money for a babysitter when you need time. That sounds reasonable to me.
What generally happens is the opposite of the situation you describe. Generally it is the non-custodial parent who is eager to have a visitation schedule because he wants to spend more time with his children. If the father in your case is being a jerk, there isn't much you can do about that.
But for goodness sake, go see a lawyer and get the child support you deserve. It is easy to do, and you are guaranteed to get money. There is no way that you should be stuck paying for everything.