@Laura1986,
Put his **** on the curb and call him to tell him to pick it up. Then block him on all forms of social media and block his number. Stop associating with him at work and, in all seriousness, consider looking for another job.
Why?
Because as you have seen, affairs are a mess at the end. And work affairs? Even more so. You are in a teetering position, particularly if your affair partner has more seniority or a higher position than you.
Please learn from this error in judgment and don't **** where you eat again.