Keep all the records you have -
Did you give a deposit? Is that what you are worried about not getting your deposit back?
If so, you can fight this if he refuses by telling him you will report all these things he did not do or provide.
If he looks for more money from you - say no - and state all the facts and records you have. Just be firm and polite. If he comes after you - you can always provide whatever records you do have.
He sounds shady - so it is highly unlikely he will pursue using legal means - he probably has more issues with the law on his side so he probably would want to avoid dealing with anything legally.