Before we landed at the point of having a rule for when we reject or accept a hypothesis, we have been thinking about this more holistically. The rule says that if the chance of observing the sample when the null is true is less than one in twenty, we judge that the hypothesis is not likley to be correct, or at least we are fairly sure it is not correct. Our holistic approach was more along the lines of thinking about what this probability was (this is the p-value approach) and thinking for that problem is the chance of such a sample or a more extreme sample too big to discount the null or too small to believe it. For example if the probability of a more extreme example was about $7\%$, we would think about the problem and if this was too low to believe the theory (we now call our theory the null hypothesis).

One reason it is better not to have a strict cutoff is that we can think for different problems we might want to have a different cutoff. For example if our theory is that a new drug cures a particular problem, we might consider different cutoffs depending on the nature of the disease. For example if the cure is for the common cold and the drug does not have side effects, allowing the sale of such a drug when the effects might actually not be real results in costs to the buyer in having a treatment that does not work, but these costs could be small. So in that case we might think that rejecting the null of no effect with less evidence that it is wrong (so using a test size of $15\%$) might be fine. We make a bigger type one error but this probably does not cause too many problems. If the drug is for a disease that, if it does not work, would leave the patient untreated (because they are taking this drug instead of another) for something that might kill them, we might have a higher threshold to be sure that the drug really works. In this case we might choose a much smaller size, say $1\%$.