@Ragman,
I've been direct with them saying that I knew they speaking ill of me. I admit, I handled it bad and it turned into an argument more times than I'd like to admit. Usually counselors tell me that it might be me just hearing things that aren't being said. I am not schizophrenic, but I am bipolar.
Most of the time, I'm told that I'm "ill" or "need help", but I've heard them say things about me very clearly because they don't go far. Every time I've told that that I don't like that they're talking about me, they'll say things like "I never said that" or "we're not doing that" or "nobody talks about you". I know they do though. These are things I've confronted my family about.
My sisters have always spoken ill of me. I was very direct once and asked them if they were talking about me and they said no. When I left they scoffed and said "NOW we are!" And proceeded to laugh at me.