Statistical validity of the results obtained in small sample size experiments. A parametric test simulation [Validità statistica dei risultati in esperimenti a bassa numerosità campionaria. Simulazione di un test parametrico]

Many authors have criticized small sample size experiments because of the lack of statistical reliability, on the basis of the statistical power and considering a subjective evaluation of prior probability of the null Hypothesis H0 (Cohen, 1994; Chertow, Palevsky e Green, 2006). The aim of the present study is to test the reliability of significant results obtained from the small sample size experiments in comparison with larger ones. The different samples (10, 20, 40, 80, 160) are obtained by monte Carlo simulation, representing two conditions: H0 true and H0 false. As parametric procedure, the linear regression analysis has been used. Thus, the frequency of the type i error and the false positive rate probability (FPRP) have been evaluated. The frequency of the i type error is around 5%, independently on the sample size. Moreover, the FPRP values obtained in the small size samples are comparable to the values obtained in the larger samples. In conclusion the significant results obtained in small size samples are reliable and have statistical validity as well as those obtained in larger samples. This is true even from a Bayesian point of view when a non-informative a priori probability of H0 is taken.