Do atheists see what Wittgenstein was pointing out ?

I can't tell the provenance of this; I have heard it was Anscombe(!) whom he addressed with his killer reply but I don't know.

SO here it is in one form:

Meeting a friend in a corridor, Wittgenstein said: “Tell me, why do people always say that it was natural for men to assume that the sun went around the earth rather than that the earth was rotating?” His friend said, “Well, obviously, because it just looks as if the sun is going around the earth.” To which the philosopher replied, “Well, what would it have looked like if it had looked as if the earth was rotating?”