Am I getting bullied?

I am now in my last year of school, its been now 13 years I've been in that school, and I still never had a friend. Sometimes I did went to talk to the "nerds", but I never stayed very long. I'm a loner but I really don't look like one. When I try to talk to someone, or they look at me like I'm a bully, or they ignore me, or they make fun of me. What usually happens is when I talk to a single person, they look scared, their hand even starts shaking, but then when they are in group, they bitch behind my back and make fun of me. They think I cant hear but I can. I've heard "shes a bitch", "i want to slap her", or them just making fun of the way I look. Most of the time, I see them laughing but I don't know about what. What usually happens is the whole class turn against me. I have never ever said or done anything that could offense them, I wouldn't even dare because I would be to scared or shy to do it. When they are isolated, they are scared of me, and look at me like i'm very mean or something. When they are in group, I get scared of them. When I have talked to people of my school about the bullying, they tell me its because i'm aggressive. This is bullshit, I've never even said "****" to someone. I would just like to understand why they hate me so much, and in particurlarly why are they scared of me, and is this linked?
I think the worse in all this is that all the blame goes on me.