@Thavam,
I don't know what backfire you are worried about. Certainly any backfire from making an accusation of racism is less severe than a backfire of removing one.
The best way to deal with problems is to talk them out to find an agreement that everyone is happy with.
If you feel like you are in a situation you can be treated fairly, and your "superior" is requesting that you remove the charge of racism, then why not? It sounds like he or she is negotiating a reasonable situation. Negotiation is generally a good thing.
And, if your charge of racism was a hasty, there is nothing wrong with apologizing. There is nothing wrong with saying, "I was overly upset and said something out of emotion that I didn't really mean". If this is true, then not only is apologizing the decent thing to do, it is probably the best way for you to end up with a good work situation. And, many of us say things overly strong when we are a stressful situation.
Figure out how to resolve the problem in a way that is best for everyone.