So your his side poon. He goes back to your thrauft for his occasional drink. Knowing it could blow at any moment. Point taken is you actually giving him a better time in life. Congratulation your making a guy feel he is worth more then he really is.
Point taken is that you could pull a Monoica Lewinski ( that is basically where your at ). Next is to save the garments, and "barack-male" him with a lawsuit and retire for that job as far as it would take you. Maybe you could have another career if the pay-out is big enough.
Then again maybe he lets one go and get you pregnant, and gives you the molly-coddling treatment allowing you to stay at home from work, and send you to school or something along those lines just to keep it yourself and not come out with "Who is the fajar"
Point is that
I knew a guy not in my family. Had a thing with two of these employees. He made sex with them. Then he got married to his sweet-heart ( who looks like his sister ). He tried to call it off but they took him to court for harassment charges. I don't know how that went down but again there is no proof of anything occurring. Reality it is a big no when it comes to doing it with co-workers and especially those above and or under you.