I can see why the abuse of the word 'ex' could get irritating.
It irritates me too sometimes.
An X should refer to someone there was something to X in the first place.
I have had lots of 'relationships', but only refer to those lasting, long intimate relationships as X's. It'd be great if everyone would do the same:)
I love the French vocab when it comes to relationships.
That is where fiance and fiancee came from; and they have a whole list of names for all kinds of relationships. They even have a name for obsessive, crazy chemical 'love'. They've got it down, baby.