There is definitely some self-serving aspects of the Vinaya, but I think you're over-generalizing about the majority of people having been on the brink of starvation during the Buddha's lifetime. That's not borne out by the stories in the Pali Canon, nor by archaeological evidence. Gautama lived when there was a population shift from rural agrarian life to city life, and when there was a shift from bartering to coinage. It was mostly a time of prosperity, though there a couple of droughts and a resulting famine or two are known.
Also, it's not like the monks were taking food out of the mouths of starving people. There were lots of mendicants of various sects (or no sect at all) at the time and it was a part of the culture to support those who were seeking after the truth by taking the homeless and possessionless route. That existed before Gautama and continues to exist today.
In my experience as a monk in Thailand, I can tell you from direct experience that the monks on their morning alms rounds do not pose a burden on the poor. The very poor people whom we encountered would usually just give us a little plastic bag of clean water. According to their superstition about making merit, that's just as worthy as giving a full meal. People give according to their ability, if at all, and there's no penalty for not giving.
Also, our monastery served as a refuge for people who were stressed out from daily life. They would take a few days off, hang out and meditate in the peace and quiet, talk with a senior monk about their problems, etc. Even if they didn't get counseling from a monk, the monks maintain the refuge, and thereby provide a service. But I did experience many occasions, quite routine, when monks were giving counseling to people with problems. Buddhism is all about the alleviation of mental suffering, after all. It's not a cosmology or even a soteriology in the common sense. Gautama taught that whatever gods or heavens may exist, they're not important. Instead, you've got to work things out for yourself, and, he said, that's easier if you follow some of his advice. Overcoming anger, greed and delusion is greatly aided by the meditation practices he taught, in my experience.
Now, I'm all for exposing the bullshit in Buddhism. I could go into a good deal of detail about what I experienced. But I'm not interested in blindly throwing a blanket over the whole of anything and claiming without thorough examination that the whole thing is a scam. There are even good parts about xtianity and Islam, despite how fucked up they are otherwise. I'd rather study and discern the details instead of lazily and sloppily throwing the baby out with the bath water.