People are often confused about discriminating between stress and distress. What most people have and (actually benefit from) daily stress in their every day jobs/ and/or routine, whether it's exercise or activity.
Those that complain about job stress may have legit complaints...but are generally dealing with stress - not actual distress. Those that are actually dealing with distress, may have the wrong job but are locked into the job due to bad economy or the inability to make a change. Furthermore, constant distress yield eventually can lead to DIS-ease
However, what is destructive behavior or activity is the prolonged exposure to distress. TIMHO, that's what can contribute to elevated levels of damage, subtle or not so subtle. We've evolved as a humans over the millennium to handling stress. Our grandparents generations handled the increased physical activity (no cars, hunting/harvesting fields, chopping wood etc.) with tougher skeletons and muscles and diets that had far less insults from environmental issues (for the most part). It seems we need stress as it toughens us and adds to our ability to survive and prosper.
Many people I know from their 2-s to 70s are engaged in activities such as brisk walking to mntn biking or bike touring and health clubs. I'd say it's the majority that do.