Another possibility is breed-specific rescue. Say you want a Boston terrier (they're small; my avatar is one), you can go to
http://www.bostonrescue.net/ or the like (just put
"Boston terrier" rescue into Google) and go from there.
Breed rescues are like the pound, but they are for purebred dogs of one particular breed. See, people give back dogs for all sorts of reasons, and most if not all responsible breed groups make an effort to take in rejected or returned members of their own breed. The advantage isn't in getting a dog with papers (personally, I find that sorta thing kinda silly, unless you're raising show dogs), but in making sure that you get a dog of the size and type that you want. The pound is catch-as-catch-can and often the dogs in there are large.
So if you want a smaller dog, your odds are considerably better if you go to breed rescue. This doesn't, of course, mean that all of the dogs at the pound are terrible or anything of the sort. I've had wonderful and also not-so-wonderful dogs from the pound. I also had a dog from breed rescue (retired racing greyhound).
As for which breed to get, Dog Fancy Magazine (they're online) has information that may be of help to you.