Never too early. I grew up on a farm driving tractors, farm implements, ranch trucks, dirt bikes, 4-wheelers, and all kinds of powered things... of course with proper supervision.
When I was 5 I pestered my dad so much to teach me how to drive, here is what he did. He backed his 69 chevy pickup with three-on-the-tree down to the bank beside the river, put me in the driver's seat and said, "get the truck back to the house." I tore up the yard, but he had taught me enough about cars that I knew what to do, and he was right beside me giving pointers and protection.
Nothing is more dangerous than a new driver who has been protected from learning until the last minute.
As a result of my early education I can proudly say that after 20 years of driving (including over 1 million miles on a CDL in school buses, motocoaches, dump trucks, and 80,000-lb semis,) I have never had a single accident that was my fault. And I have been able to maneuver all of those vehicles in such a manner that I was able to avoid all but three fender benders caused by others.
Teach them early, teach them well, supervise and educate. States will give a license to anyone with a pulse and a signature, so its up to the parent/guardian to improve the safety on the roads by educating.
Of course, this is all off the road for legal purposes. As far as the government assigning a legal age at which all teens are capable of driving... that's like assigning 18 as the age of consent. I know 8 year olds who are more competent at choosing their destiny than many 18 year olds. By the same notion, I know 12 year olds who could drive better than 30 year olds.
As far as legally driving on the road, follow the law if you want to avoid prosecution. As far as teaching your child to drive, start them when they show interest. You may have a child like my Mother was... she got her license at 27 because she had to for gas rationing. She had no interest in driving. She is now 60 and has frightened me since I was 4 in a car. Or you may have a child like I was... ready to hit the road at 13 and be safer than most.