Guys, I was treated rude and unfair by my professor this morning! He was kind of threatening my fina

I'm a college student in Houston TX( I will reveal the professor's name and the college if I really get an unfair grade in the end). Few days before the final exam our professor didn't come to class instead he asked a lady to give us a quiz and gave our previous exam papers back. The lady informed us to return the exam papers after we read them. I was not paying attention that I didn't realize it was necessary for me to turn them back. So I left the class with those previous exam papers in my bag. (The lady did not know)
Later I was getting ready for the final and somehow I realized that not many professors would let students keep their previous exam papers and why would this professor be special. Then I text my friend (in the same class with me) asking he if we needed to turn them back and he said yes. So I brought them back on the day of final and gave them back to the professor as soon as I finished the final.
The professor looked at me said u had a problem and asked me to meet him in the office so I did. He asked the lady if she told us to hand in those and the lady said yes. And he asked me why didn't I hear it I said I was paying attention and I am sorry for that.
The next thing he said to me was OH WELL I DIDN'T SEE YOUR FINAL PAPER EITHER! (which i just turned in with previous papers) I said I just gave them to you in the class he replied "well so what I didn't see it and your final can be zero" I asked him why would that matter so much that I forgot to turn them back in on time(those previous exams have nothing to do with the final, no question from them is the same with anything in the final). He said "it matters for me and we will see how it matters when your final grade comes out." His attitude was disgusting and he didn't give me any explanation of the problem and then he said he got something to do and left.
Things he said the most was I didn't see your give me your final papers which I did in front of the whole class and he said you would see what happened when the final came out.
So what should I do if he really gives me an unfair grade later and how can I find a right way to complain against him?