@WBYeats,
Not every person no, there's always diehard loyalists. It means the powerful enemies will stir up the country until most people (or the ones that matter anyway) will want to kick him out of England.
As has been pointed out context helps. This is from a Sherlock Holmes story, not one I've read, so I can only assume he's talking about members of 'polite society,' the moneyed, landed Aristocracy and lesser gentry who don't need to work for a living.