It means they're not wanted - yourself and your conversations. See point 3. below:
im·pose [im-pohz] verb, im·posed, im·pos·ing.
verb (used with object)
1. to lay on or set as something to be borne, endured, obeyed, fulfilled, paid, etc.: to impose taxes.
2. to put or set by or as if by authority: to impose one's personal preference on others.
3. to obtrude or thrust (oneself, one's company, etc.) upon others.
4. to pass or palm off fraudulently or deceptively