"If I have a problem with having crushes on people it probably stems from past self confidence issues as I've always been this way, . . ."
What? That you need and crave attention from men?
A truly "self confident" woman would put this in its place - in and then out of the mind. Then, that "self-confident" woman would feel assured that there's so much better stuff at home that she wouldn't need to spend time and energy "mind screwing" another man.
Work on your self confidence and self esteem so you are not so distracted by another man who shows attention. .
PLUS - this ALL could be in your head and your have misread his friendship.