@Quietguy,
Why don't you try talking to him and explain your situation. You should be happy to have such caring brother btw
Maybe he doesn't have lot of friends, so he prefers to stick out with you, or perhaps you like a role model to him. Try to talk to him and set him on track, be a big brother and he will stop intruding in your life.
Good Luck.