@roger,
Dana Ste. Claire, a Florida historian and anthropologist who studies, er, crackers...It was in the late 1800s when writers from the North started referring to the hayseed faction of Southern homesteaders as crackers. "[Those writers] decided that they were called that because of the cracking of the whip when they drove slaves," Ste. Claire said.