But it wont hurt imparting a "dont let 'em mess with you" attitude to him..
Eg, when you do have to intervene, let him know how he can deal with such a situation himself next time, probably.
I think my mom raised me, not with a "dont let 'em mess with you" mentality, but with a "be a good person" mentality. Which is fine and fair and nice, but boy did I land into trouble in primary school. Bullying is an evil thing.
The trouble starts right at the beginning too, because once you're identified as victim, the process is very hard to reverse - the initial bullying will make the kid more insecure, which'll attract more bullying, et cetera. Its just different for boys. They face a tougher deal than girls - "cuteness" never offers an out for looking vulnerable, if you look insecure you're done. (I dont really know where your kid stands on this - you said he reacts rather passively but I dont know whether he's insecure or emotionally troubled or the like).
Preventing all that is a tough dilemma though, I imagine, because you dont want to put any obstacles in the way of a kid's more dreamy or kind-hearted or trusting sides, that would be a shame too, no reason to instill cynicism in a little kid. And there's an open-hearted kind of natural kindness that's not the same as insecurity. But yeah, I sure could've done with a dad who'd have told me, f*ck that, give 'em hell! - and in combination, sent me to a karate class yeah

- and that would have had to have started at age, say, 5, 6...