often to our detriment and to that of people whom we care about?

Does "often to our detriment and to that of people whom we care about" mean "often to our detriment and to the detriment of people whom we care about"?

Context:

Since these imperatives are such frequent intruders into our consciousness and play so important a role in shaping our emotions and behavior - often to our detriment and to that of people whom we care about.