Can I just wear a Sports Bra in Public?

You mean: Not top less - but buttom less.
Not exactly appropriate in my opinion.

Instead of a shirt? Sure. It's been done before; Google Brandi Chastain.

Be prepared to get catcalled.

It's just as hard for men.

no you cant.

What sports bras (as opposed to tank tops) mostly reveal is the mid-section of the body. If, as you say, you have a "sleek body". I think you could wear one in public in very hot weather, particularly if your choice clearly looked more like sportswear than underwear.

Something like this would seem appropriate for someone your age.

Even if this Sports Bra looks nice - it is a bra.
Nicer for other people is if you were a light linen or cotton blouse on top of the bra.
For your skin when it comes to sunshine. also the blouse is better.