Dating/Married/Attraction Advice - Hiding signs of Attraction

So I think I screwed up... here’s my story.

About a year ago I was trying out some new styles and wore this cute little dress to work. It wasn’t until I got to work that I realized it was a bit too revealing/ tight etc. the reason is due to a male coworker that was stationed a couple desks over was literally squirming / shifting , grunting and staring at me all day. TBH, I hadn’t really noticed him much before that day. I don’t know if he was laying low before then or he was new or what, but I sure as hell noticed him then and since then.

So... I have not reworn “the dress” to work, but we have continued to have an awkward dynamic. About 6 months later we ended up both being on a work trip overseas. That was a nice escape from the office where it was a little easier to try and connect on a more casual level, and I did end up going out with him two times (once in Shanghai and another time in Vienna), but there was always 1 other girl. I really recognize that I am crushing on this guy, like actually wanted to go out with him more and want his attention for various reasons (hot, nice, smart, funny, charming, sensitive, loaded, motivated), but I’m not too keen on the competition and there’s apparently a lot of competition

In one city, me and another girl meet him in his room. She is literally throwing herself at him, while we a readying to go out. I am hating myself for feeling like I’m in high school and wanting him. In the next city, it totally baffles me that we sleep in separate rooms that are located right next to each other. He makes no direct moves. I chalk it all up to he is a MASSIVE flirt, possibly good for a fun time.... maybe someone I should tease....

So back to life, it turns out that I am married and very unhappy for a long time with an emotionally abusive husband who really couldn’t care less except about his own needs. Word gets out about me being married (not the unhappy part), and this randomly comes up in conversation and he has a disappointed reaction. (Me too, but I try to keep a straight face). He then takes off for a wild vacation with god-knows who (why am I getting jealous?)

He comes back briefly and then leaves for like 3 or 4 months to go work on a project in Switzerland. During point I worry about all the Swedish girls and how much fun he’s having without me (yet again). At this point I realize, I’ve really got it bad. I resign myself to strictly deny myself. Cut it off (in my mind) and No contact ever.

My nerves eventfully calm down with him away and out of the picture. But then he comes back about a month ago and it’s been torture. I can’t look at him without being dumb, so I avoid all potential interaction even eye contact. I can feel him staring at me and see it in the periphery. He sits diagonal across the room from me now but we still position our bodies facing each other. There is intense level of energy exchange going on that it’s highly distracting. We both have headphones on most of the time, but the vibe is just too strong. He watches me when I get up and walk about, or if I move around even at the desk, and watches/casually notices if any other males come around me. He also seems confused/hurt that I no longer acknowledge him when he comes/goes or leaves for the day. I mean he’s getting every else’s attention, why should he need mine? Although I briefly welcomed him back (I think), I’ve really been trying to block it out bc the attraction is so strong and I don’t like feeling out of control. But then the other day on my birthday I got an office bday card which he signed “I hope you’re birthday was as much fun as the time we got lost in Shanghai”. That made me smile, but it’s also driving me crazy because I had been trying to forget it all.

I really can’t take this much longer, something has got to give. I hope I’m able to move offices, or he steps up, or he leaves for good, or something else.

Any advice? Have you experienced this before?