You're certainly right about the foolishness of expecting perfection from anyone but that is definitely not part of what I meant by being infatuated. Nor does it have much to do with simple sexual attraction.
People in the state I'm talking about are more likely to overlook any flaw, accept any cruelty and forgive any injustice from the one they have put on a pedestal.
As bad as it is, 'the pedestal' is celebrated by society in innumerable songs, books, movies, even wedding vows. "I only have eyes for you", "You are my everything", "Loving this one and forsaking all others", etc. Society also embraces the other side of it too. They thought it was perfectly normal for the Beatles to sing - "I'd rather see you dead little girl than to be with another man". Sorry for the age of these examples, I don't listen to pop songs much anymore but I doubt they've changed.
There are two ways to look at what you said about not really being attracted to other women. The negative way is when people are obsessed or infatuated with someone. The positive way is when they have an appreciation for physical and emotional intimacy combined. Hopefully that is what you and your wife have.
Not sure why I developed that appreciation early on but I did. Even way back when I subscribed to Playboy, the centerfolds had no sexual appeal to me unless I fantasized a full relationship with the women in the pictures. That still did not keep me from getting infatuated with the real women I met. I fantasized that emotional intimacy with them too, even when it wasn't there. I did that for 50 years.
Don't be sorry that's over with; I'm so much happier now that I am open to real relationships without society's unrealistic ideals.