We agree in the case about principle. We disagree on the case about "simple remarks and careless jokes". Here are the reasons I disagree with you.
1. I don't accept the idea that anyone has an obligation
to apologize. If apologies are obligatory (even for minor offenses) then they are meaningless.
2. I don't like the idea that someone can be pressured
to apologize. In this story, the offended person didn't just ask for a personal apology, she went to a professional organization to apply pressure.
3. There is risk in confusing of offense with harassment.
There is a legitimate need to prevent harassment and abuse of power in the workplace. We recognize special protections for people in a workplace (which I agree with). When I am in my workplace, I can't just leave when I am offended (without real economic consequences). I am impacted by the social organization and a hierarchy that gives people the ability to impact my career.
Arguing about off-hand jokes mentioning lingerie being made in a public elevator as "harassment" is a distraction from important issues around creating decent working environments.
These protections in a workplace are special. They limit expression and regulate behavior that are perfectly acceptable elsewhere. I can tell jokes in a bar that I can't tell at work.
The importance of creating a fair workplace warrant these limits. But expanding them to other public spaces is dangerous to the freedom of expression.
4. In the past, freedom of expression was a core part of being a liberal. The defenders of the First Amendment used to be liberals. Now it is liberals who are saying ridiculous things about "hate speech not being free speech".
Freedom of speech remains one the principles that is most important to me.