@Eliusa,
You don't have to TELL anyone, but you should ASK yourself: am I happy ?
You have lover #2 who covers for lover#1 and both cover for your husband.
What's missing, you cannot achieve with 3 men? Once you have that answer, you'll be on your way to bliss.