@boomerang,
I went through a period of doing a lot of visiting, it varies a lot!
We have a good friend who is a Lutheran minister, in her church you definitely could just plain show up, in whatever you happen to be wearing, and she'd be thrilled to see you.
I've gone a few times, she doesn't expect any participation, you can but you don't need to.
At my Grandma-in-law's (I guess, after 20 years I consider her my own grandma) Catholic Church, they would definitely find it disrespectful if you just showed up in any old thing -- you should dress up a bit, and you should probably wait in the back and observe at first. Someone would probably approach you and take you under their wing if you are nicely dressed and quiet/ respectful.
I haven't participated, and that's been fine.
I went to one church (Baptist, I think) that definitely would frown on anyone just stopping by. I was brought by someone and that was a bit of an issue, she was beloved so they let it slide.
Probably calling ahead, while not necessary at all places, could smooth the way if you're not sure.
What kind of places are you thinking of going?