Guilt and anxiety

Long story short I've struggled with feeling guilty for things in a relationship. One day my wife and i were talking and i found out she doesn't like if i was to have a crush on another woman. At the time I did. I wouldn't ever do anything about it. Time has passed and I am always worrying about it. With my boss who is not attractive. I'm pretty sure it's just that need t make your boss happy. The other I thought one time when I made I contact with her sister and that gave me the conclusion I had one on her. But i dont i just cant get the what if out of my head. And I've been having panic attacks about it. Obviously I cant talk to my wife about that. Just wondering if anyone else has delt with something like this? How do you deal with it?