Ow in affair After affects - advice please

I was having an affair with a married man that he ended. I know this is morally wrong, I do not need to be told that. It was a huge mistake
A few weeks after he contacted me to say his wife had found out and we can't have any more contact (we weren't at this point anyway)
We work for the same company and two weeks ago he got a colleague of his to lie over something that had happened in the office. I went to confront him and he walked out the office and said he can't have contact with me and 2 colleagues told me I wasn't allowed in the office and to leave. I then sent an email to the guy and one to his wife apologising for everything. I wanted them to know I was sorry and also for selfish reasons I don't want this to affect my career.
That night I started getting anonymous threatening messages. This went on for 2 weeks so I went to confront the guy again. In front of others he said he didn't know anything about it and then separately he admitted it and said he would get them stopped. I reported these to the police but they cannot trace the number or find out who was sending them and they did not use my name in any. I have one ally in the office who has told me he has everyone fooled and he's told everyone (including his wife) that I'm crazy and nothing has ever happened beteeen us and that he has had to change his mobile number to stop me contacting him (I wasn't) He is hugely protective if his reputation and works hard to be seen as the loyal family man.
I do not want to break up his fanily, what I did was wrong. I am however amazed that he is trying to hurt and threaten me in this way when 1) he was the one that ended it 2) everything in his life is rosy and normal. He is making me out to be crazy and I do realise by confronting him I have urged that thought process on. I cannot defend myself. I've changed my mobile number as well.
I know some people will think this is karma or what I did but I just don't know what to do anymore. My reputation is in ruins at work. Any advice much appreciated. Thanks