@JoeSmittle,
As far as I can tell (not an expert), any drug consumption (and caffeine is technically a drug) creates a ressistance.
The more caffeine you consume, the more your body stops producing the chemicals that are replaced by caffeine, because it can predict you will take caffeine so you will have a new normal.
Yes, I think you should test for normality.
Perhaps, increased caffeine consumption - increased reaction time.
But you should also be wary of other variables such as the participants daily diet as whole. Probably should ask them what they eat today in the questionnare as well, just to make sure there aren't any other variables at play.