1. Be very careful restricting free speech. Some day you might want it again. Free speech was used by abolitionist, LGBT activists and anti-war activists throughout history. For you to think that this is just protecting far right "nazis" is misguided. Free speech protects you too.
2. I am speaking out against extremism. I speak out against extremism from the right. I also speak out against extremism from the left. The two sides feed off of each other, extremists from one side use the extremism from the other side to justify their own extreme views (for example taking away free speech).
3. Free speech has nothing to do with violence. I support your free speech. I don't believe that you have ever "taken things to the next level and started shooting".
4. I support the free speech of everyone. This includes nazis and the women of #MeToo. I am not making an equivalence (in fact you brought them up). The point is that if free speech only applies to people you like, then it isn't free speech.
5, You don't want to give human rights to Nazis
. I kind of understand that. However there is a problem. Someone, or some political group, or some religion has to decide who gets human rights and who doesn't.
The question of who is a Nazi is rather fuzzy, particularly among political ideologues (on both the right and the left). I know for a fact that you yourself have been called a Nazi.