@Kingqueen123,
You're having problems in your marriage (don't deny it - you wouldn't be acting on these fantasies 2x if you weren't). He's probably bored in his. Bingo, time to make out.
You are the architect of your own misery here. Work on your marriage, or end it if it's not working out. Leave him alone, and watch your drinking. And for God's sake, don't get involved with someone at your office. The endgame is rarely pretty, even when both parties are unmarried.