Why do stalkers stalk you, and get annoyed with you?

They're crazy people. That's all you need to know. There is no logic there.

Some are raised very badly by their parents and those peoples' parents, if even known. They have desire, which can get to be mortal for someone else, a big need, fixation. I'm not a psychologist, but I'm virtually sure this has been studied.

So, pay attention.

https://en.wikipedia.org/wiki/Stalking

Studied a lot. To take just one example, in "A Study of Stalkers" Mullen et al* identified five types of stalkers:

Rejected stalkers follow their victims in order to reverse, correct, or avenge a rejection (e.g. divorce, separation, termination).

Resentful stalkers make a vendetta because of a sense of grievance against the victims – motivated mainly by the desire to frighten and distress the victim.

Intimacy seekers seek to establish an intimate, loving relationship with their victim. Such stalkers often believe that the victim is a long-sought-after soul mate, and they were 'meant' to be together.

Incompetent suitors, despite poor social or courting skills, have a fixation, or in some cases, a sense of entitlement to an intimate relationship with those who have attracted their amorous interest. Their victims are most often already in a dating relationship with someone else.

Predatory stalkers spy on the victim in order to prepare and plan an attack – often sexual – on the victim.

* Paul E. Mullen, Michele Pathé, Rosemary Purcell, and Geoffrey W. Stuart."A Study of Stalkers", American Journal of Psychiatry 156:1244–1249, August 1999.

Who is your stalker complaining to? You? Mutual friends?

Have you met your stalker before he started following you?