Alpha level (aka: p-value) & what it might indicate in research

Hello,
The significance level alpha is the probability of rejecting the null hypothesis, given that the null hypothesis is true (Type I error). That is, the probability of falsely rejecting the null hypothesis. In general, this probability (alpha) is set to 0.05. In this way, we know that in the long run (over many tests), we will only falsely reject the null hypothesis 5% of the time.

Of course, we may want this probability to be even smaller, like 0.001. Decreasing alpha to 0.001 ensures that we will falsely reject the null hypothesis only 0.1% of the time. That is, we will make a Type I error less often. However, other things being equal, this will increase the probability of a Type II error, like you said.

Thus far I have only talked about the situation where we perform a single test, and we have to decide upon the level of alpha we want to use. Now imagine that we want to perform a lot of tests, for whatever reason (e.g., because of the reason your professor mentioned). And suppose we use an alpha of 0.05 for each of these tests (i.e., per test, the probability of making a Type I error is 0.05). Then the probability of making a Type I error in at least one of these tests (overall alpha) will become much larger than 0.05. Hence, it may be a good idea to lower the alpha per test, in order to keep the overall alpha at an acceptable level.