You are right that exhaust fumes are not good for a person. Also, you might be more sensitive to them than other people. That being said, normally, this type of exposure isn't considered too dangerous. I would be okay with it, if it were me. However, if you are bothered by it, you might want to look into a different way home or invest in disposable masks. Don't feel paranoid. Everyone is different.