@Mou,
You're better off that this happened before you got married, before your finances were tangled up with his, before you moved or quit your job, etc.
The guy's an idiot if that's his deal breaker, but I imagine he was clutching out for some way to end it and that's all he could come up with.
I'm sorry this happened, and I know it hurts, but you don't want to spend the next several decades putting up with that kind of shallow, childish behavior.