What are rural American versions of "what the heck" that aren't mild oaths or expetives?

You must be writing for a VERY conservative audience if you're worried about offending people with phrases like those.

All of the examples you cited—including "What the hay?"—are euphemisms for "What the hell?" None of them are any more offensive than the others. In particular, "What the heck?" is very common, and completely acceptable, in all forms of literature.

Ya gotta wonder if someone who offends Gosh will be darned to Heck for all of eternity.