Is it harmful for a 4 yo boy to occasionally dress in female clothing

No. Why would it be harmful? And what does you being straight have to do with anything?

What's wrong with dressing him in boy's clothing?

What is a straight family?

Whose business is it except the child's what he wears?

There are just too many variable here: when is he doing this? I only ask because I wonder if this is during play time with lots of choices or is he wanting to wear these clothes to school? Does he have older sisters who play with him?

It also is important that he not be shamed in any way for the choices he makes. Sounds like the adults around him need counseling.

My son was brought up with a strong willed older sister and her best friend who used to make him into any kind of role they wanted him to be when they were playing, many times with costumes. It usually was playing house or theater. I remember him storming into the house and yelling, "That's it. I'm not going to be the grandma." He was about 4 at that time.

If you're that bothered with it, let the child play dress up at home and not out in public. Either way there's nothing wrong with playing dress up and don't read anything into it. He's only four years old for Pete's sake. Plus keep the proverbial haters away from this child as their undereducated/uneducated opinion need not poison the lovely and imaginative nature of this playful kid.

What should be wrong doing that?
Girls dress up as boys and boys as girls when playing.
If some female family member in jeans/slacks hate it - tell them they dress as men and should stop nagging your little boy.
(PS Privately I do not mind women in jeans)

No, it's not harmful.