@Cianna,
We train people how to treat us. You have allowed your husband to believe that it crosses no personal boundaries of yours to have this woman calling and texting him at all hours .
You are at a disadvantage because you didn't put your foot down early, so he doesn't think he needs to respect you, and has no problem with hurting your feelings as long as he gets to carry on this outside "friendship". Personally, he sounds like a jerk and maybe not worth keeping.
If you have any self-respect, you need to stand up for yourself and decide what you are willing to live with.
If this man and this marriage is something you really think you need to fight for, you're going to have to either demand he respect your feelings, or accept that he is going to continue to walk all over you.