@birdgreg,
You are in love with someone who has no morals. Don't you think you would want someone in your life with morals?
There are two things you can do, if you are "ships passing in the night" and "friends". 1) Work it out and work at it to change it, or 2) walk. There is no in-between.
I suggest you stop contact and slap yourself for seeing a married woman and get out there and find someone of your own that you can call yours, in togetherness off course.