To compute the t-statistic in a 5x2 CV test, the numerator is based on a single CV fold. Usually, the numerator is the performance difference between two estimators on the first fold of the first iteration. Why not compute the numerator as the average performance difference between the two estimators across all folds of all iterations?
Sources: Kaggle, USF slide 5.
2.1m questions
2.1m answers
60 comments
57.0k users