Compute metrics of the submissions in the history.

Usage

compute_metrics(hist_dir = "history", metrics, y_test, ind_quiz, read_fun)

Arguments

hist_dir
string. directory where the history of the submissions are stored. contains one subdirectory per team.
metrics
named list of functions. Each function in the list computes a performance criterion and is defined as: function(y_pred, y_test)
y_test
character or numeric vector. the test set output.
ind_quiz
logical vector with the same length as y_test. ind_quiz[i]=TRUE if y_test[i] in the quiz subset.
read_fun
function that reads a submission file and returns a vector of predictions.

Value

compute_metrics returns a named list with one named member per team. Each member is a data.frame where the rows are the submission files sorted by date and the columns are:
date
the date of the submission

file
the file name of the submission

.quiz
the score obtained on the quiz subset

.test
the score obtained on the test set

Description

Compute metrics of the submissions in the history.