Two possibilities. I have a dye allergy where my throat closes when I get it on my skin. I can tell you that dye is such a common ingredient that you could easily react to many soaps; I have to really look for soap I can use. Same with shampoo and conditioner. You might be reacting after gardening if you wash your hands thoroughly afterwards, leaving the soap on your skin for longer. Food dye allergy does NOT show up on an allergy test, but your doctor can still help you diagnose your trigger.
I've also read that some potting soil contains ingredients from peanuts, tree nuts, and coconut (you may also react to mold). These allergies, including mold, will show up on an allergy test.
Lastly, try dye-free Benadryl. If you are allergic to dye, you will feel beyond miserable ingesting dye and antihistamine at the same time (regular Benadryl makes me much more severely tired than the dye-free version). Good luck! New allergies are bewildering.
Lastly, Benadryl is the right choice for acute allergic reactions. Sudafed and Claritin will not stop a rapid reaction nor keep you safe. They are best for mild, around-the-clock allergies.