@gollum,
It means, in part, you don't have to swear your oath to God, or put your hand upon a Bible, or such.
I believe the courts puts judges, and department heads as said judicial and executive officers, usually they are elected or need legislative confirmation to get their position.
As police officers and others who hold positions of public trust, they too must swear allegiance to support the Constitution. (That they had to say "..so help me God." is an Urban legend, added for dramatic purposes or proof that they weren't a godless communist.)