"Deleting Stuff", as most users understand and do it, more or less amounts to removing the "Old Stuff's" address from the directory, essentially freeing the space formerly occupied by the "Old Stuff", allowing that space to be occupied by "New Stuff" as it comes along.
All too often, uninstalling a program, whether through its own uninstall routine or through the Add/Remove Software feature, results in icons, shortcuts, folders, files, and registry entries being left behind, orphaned, so to speak. Typically, these "orphans" take up very little space, and do no harm, but it isn't a bad idea to hunt these down and delete them as well. Often, after performing an uninstall, you'll be presented with an information box telling you some things were not removed, listing the name and path of the remaining items for manual deletion.
Its a bit trickier to clean out obsolete rgistry entries, and prolly not somethig the casual user really oughtta tackle. There are a number of apps, free and for-fee, which can do some registry cleaning safely and reliably. For XP users,
CCleaner is a donation-ware (free to use whether or not you donate) has a feature which does a fair job of finding and deleting no longer necessary registry entries,
RegSrubXP is a donation-ware registry cleaner for XP users which is safe, reliable, and relatively easy-to-use, and
Spybot S&D (also donation-ware), with its "System Internals" feature offers some registry cleaning functionality to users of any Windows platform. If you use any of these, read and understant the app's documentation before you jump in. Beware as well of the many heavilly advertised rogue apps out there that offer a "Click For Free Scan" come-on, then report all sortsa stuff needs to be cleaned, and promise to clean things up just as soon as you send money. Much if not most of what they claim to find will not actually be present on your system, and these apps frequently install other unwanted stuff, such as adware and spyware, as part of the bargain.
Deleting old emails, pictures, media files, and stored games (not necessarilly the game app itself, but games that have been played and stored for later resumption of play), along with old personal stuff - no longer needed correspondence and other documents, old news articles and web clips, no longer useful spreadsheets and caledars, and that sorta stuff, can free up an awful lot of space on most folks' machines. Do that and regularly defrag, and your machine should be happy, and you along with it.