Hi - I was a bit like you.
So my junior year - I had no interest in going to prom - I did not - never regretted it.
In my senior year - I thought the same thing. My mom kept telling me you will regret it if you don't go. Well, it was toward the deadline of buying tickets and I was dating a boy that was 2 years older. He heard about the prom and asked me to go so I said sure. I got his best friend a date so he wasn't the old guy at the prom (he was only 20) - we went it was fine, fun and all that, but to be honest my graduation (even though a bit boring and long) was so much more a highlight for me.
I think, if I did not go to the senior prom, I would not care and not have a regret. I do not even know where my prom pictures are.
For some people this is a huge deal, we are all different - for some it is a highlight and so be it - everyone has different things that the like more, care about and so forth.
It will only bother you if you let it. If someone asks about your prom just be honest - that wasn't my thing at the time.
Remember it is just one night - you get a nice (not really that nice) dinner, you dance get pictures and go home. Isn't there something else you spent hundreds of dollars at that you probably enjoyed more?