Yes - you are wrong.
You are making a mountain out of a molehill.
I see why your husband said
We will talk about it later.
simply by how you are handling this. It seems your husband was trying to spare you from feeling hurt by his brother's childish comments.
You say he has a close relationship with his brother - well let me tell you --- in these sorts of relationships boys will be boys. They probably have an inside joke that your mom is not the best cook. Now this may or may not be true. These two brothers are resorting back to their childhood when they acted like kids and this is some perhaps in bad taste joke. It is not meant to be hurtful they are just being a bit childish, but it in a weird sense keeps them bonded which is actually (in an odd sort of way) really nice. Let them have this boys will be boys relationship and realize what it is. It makes your husband happy and it really is kind of sweet to be so close to someone.
Honestly it was a stupid thing to argue about. Save your arguments for something important. And just consider that your husband is kind enough to try to spare your feelings because he knows this comment from his brother is simply that a silly childish joke.
Everyone is different - some people have thick skin and others do not - seems your husband probably grew up where you say things like --- Your mother wears army boots -- all meant in good fun. And you likely grew up completely different where maybe you never said anything mean at all as a joke. I really do not see at all how he disrespected you - but then again I grew up with all brothers so had I been in your situation I would have probably laughed with them.