I would not recommend Anthony's Pier 4 at all. I've been their twice in the past few years - the service is terrible and the food so-so. It used to be one of THE places to go - but since the owner Anthony passed away, it isn't run the way it used to be. The son stinks in service.
Jimmy's is no longer in existence. The new Legal's coincedently is there along with a whole new waterfront area - beautiful walkway, etc. I work right next door so I am very familar with the area - I agree Legal's is a bit more expensive, but you always get very fresh seafood and I've never had bad service there. And the new location is fantastic - so for an overall experience it is great. Probably why I recommend it.
There are some other good seafood places that you get cheaper - like No-Names (right nearby) but for overall atmosphere Legal is better. No-names also used to be better, not as good any more. Another thought is the Fresh Catch - there is one in the North End and one not too far from this area - near the new courthouse.