This smells an awful lot like the other man is abusive.
What the hell are you doing with him?
Someone who calls you names can possibly graduate to hitting you. This is not a good situation.
As for the man you live with, why are you staying if you're unhappy? Break up with him and move out. I bet he'll be as relieved as you will. And do so for your own purposes and on your own terms, not because of gas station guy.
Can't afford your own place? Then check the local paper or Craigslist and get a roommate situation going on. Save your pennies and strive for independence.
Also, I am not a doctor, but your self-esteem is in the sewer if you think gas station guy who calls you names is anyone but someone to walk away from. Please go to counseling and get some tools to shore up yourself self-esteem and recognize that you don't need to be verbally abused in order to feel like anything is happening.