Please don't play "Stupid Human Games."
If you're done, you're done. It's that simple. No need to act as poorly and stupidly as him. Get a plan together and make yourself a better life.
Just tell him it's over. Research divorce in your State, draw up papers and have them ready. Lock down your credit. Make a fair and equitable settlement of debts & assets. Make copies of important documents. Consult an attorney.
Don't throw out his personal belongings, don't behave badly and certainly don't seek "revenge," It only hurts you in the long run.
Do be proactive, calm, thoughtful of your own self care.
Yes, it hurts. Yes, you're mad. Yes, you've been betrayed.
Yes, you can get over it. Yes, you can learn to love again. And yes, you are worth your own time, energy and money.
Just be smart, calm down and get to work.