Why do we always hear the number six million for the Holocaust?
Because the Jews more than any other group was the focus of Hitler's wrath and he shaped them into a tool to whip up anti-Semitism against them. Down through the centuries Jews were viewed as evil and responsible for the crucifixion of Jesus Christ. These were negative sentiments easily compounded and cultivated among non-thinking Europeans. Non-Jewish victims, was a secondary issue, not as important and would not have been worth what Hitler wanted to do; non-Jewish victims were just swept up because of their undesirability, like Gypsies, Gays, and the mentally challenged. Another thing, like the West all over, there were not that many out-of-the-closet Gays, and Gypsies were a minor minority.
Hitler also wanted the Jews' material wealth, their bank accounts, their artwork, their businesses, the homes of the wealthy Jews. All this wealth would add to the coffers of the Third Reich and allow them to to have a well endowed treasury.