@coworker,
Hi Coworker
You are still making the mistake of being concerned about
his feelings - worrying that he's behaving coldly towards you after agreeing to be pleasant.
It sounds as if he's still managing to manipulate you! Whilst you continue worrying, he's winning!
It isn't easy, I know, but you are doing well in that you realised that there was a problem in the first place, and sought help from people on this forum - please take heed of them, and ignore his machinations. That's the only way you are going to feel comfortable working there!
Don't let your resolve weaken further.