Well for me the issue would be moot. If you saw me in the water with seals it wouldn't be me swimming with the seals but me drowning nearby some seals. I can't swim or even doggy paddle to save my life.
So, if you see me in the water with a couple of seals, I would like to ask in advance for a life preserver ring, s'il vous plait
But this whole scenario of people being potentially compelled to swim out to see the seals sounds like its a situation ripe for future Darwin Award winners.
There are bound to be more then several yahoos who will openly ignore the official warnings and lament that fact after the fact if they are victims of a shark attack and are lucky enough to survive.