@mirtschi08,
In a situation like that, sounds "ok" to me. I mean it's a bus, you have to hold onto something when it's moving and someone is "moving" past another person.
I'm not sure why you even mention age " about 50" if he was 80 would it have bothered you?
I'm not sure what there is to shout about, or quietly say something in a situation like this.