I'd just as soon use filtered lists but I need to know what's being filtered out in order to visit those forums and add them back in.
It's more of a recommendation list than a filter list right now. It will change further but right now it takes the your forums (added by vote totals automatically, so if you participate they tend to add, if you vote down then tend to remove, but also can be added/removed manually) and shows you those topics. It takes a "what to add" more than a "what to remove" approach right now.
Anyway, we are going to make more changes to this. It was initially intended to be a "my forums" filter, but that signal alone isn't good enough in all usecases, so now we are going for a general "noise filter". It will get better but the intended usecase isn't going to be those who want to view all, it is for those who can't keep up with the firehose and want a curated stream.
So right now I think an idea would be to have the all topics view collapse the removed forums (giving you the all-but control) and continue to iterate the noise filter to make it more useful (but still a bit of a black box as it's not going to be a straightforward algorithm).