Mat. statistics: what criterion of reliability to choose for an experiment with one subject?

Question

Analytics

There is an experiment with one subject. The bottom line is this: the subject passes a certain test A five times, receives the results, then is exposed to B, passes the test A again five times, receives the results:

(the questions in the results are because the data has not yet been received, they will eventually be replaced by real values)
Question: what criterion should be used to determine the statistical significance of the effect of B on the subject? In other words, how do you know if the resulting changes were statistically significant?

Answer 1

There is very little data to draw conclusions about the statistical significance of the influence of the factor on the result. We need a sample, and a representative one, preferably from 50 people. Each subject is subjected to tests->influence of the factor->tests.

Answer 2

There is no statistically significant effect of B on the subject.
The phase of the moon has changed, your subject began to give different readings.
Now, if you did test A 5 times - impact B - test A - then there would be significance.
I'm not a statistician so I can't be more specific.

Answer 3

The fact that the subject is alone does not affect the assessment of the statistical significance of the results, unless taking the test once does not improve his average scores a second time (without any influence), for example, due to remembering the correct answers.

Answer 4

In fact, this is a classic task described in any textbook on hypothesis analysis.
To "work" in this case, the standard is to use a two-sample Wilcoxon-Mann-Whitney test. (Just don't confuse it with the one-sample Wilcoxon test - it's meant for something else).
Please note that the completely "classical" Student's t-test (checking for equality of mathematical expectations) will not work here, because in order to use it, it is necessary to prove the normality of the distribution law, for which there is clearly not enough data.