Actually, more than that, I think. I thought it came from an old movie starring Bob Hope and Bing Cosby...? One of the "On the Road..." movies?
My dad used to say something very similar: "In a village of idiots, he'd BE the village idiot." He got his best lines from Bob Hope movies.
But he also said some other things that certainly didn't come from Bob Hope, that I really can't repeat here...