I was thinking about what we mean when say that music or an album "grows" on us, and I found this interesting article.
http://www.bigtakeover.com/essays/music-that-grows-on-you
Unfamiliar music in general takes time to appreciate, but when one becomes accustomed to the particular genre or style of music, one tends to appreciate it easier. With progressive bands who change their sound every release however, and thus become slightly unfamiliar with each release, tend to divide fans of their previous releases.
We enjoy familiarity or at least we're comfortable with it, like being at home, and I think it could be a similar case with music. People who like familiarity will generally listen to a small range of music, whereas people who enjoy being taken out of their comfort zone will be better able to appreciate the unfamiliar, alien sounds of new and different music.