topk_evals#

scio.eval.topk_evals(evals, k=1, *, baseline=None)[source]#

Identify best performing scores in at least one scenario.

Parameters:

evals (NDArray[np.floating]) – Input evaluations, usually from a compute_metrics() call. Shape must be (n, *scenarios_shape).
k (int) – Parameter defining the top \(k\) for every scenario. If not 0 < k <= n, every row is selected — even if full of nan (see Note below). Defaults to 1.
baseline (int, optional) – If provided, the corresponding row is considered separately and always included in the final result.

Returns:

idxs (NDArray[np.integer]) – Indexes of the rows of evals with at least one value in top k, across rows. Additionally, see baseline if provided. It is a sorted \(1\)D array.

Note

Conventionally, nan values are never considered amongst top k.

Tip

Using k=len(evals) can be useful to filter out only rows full of nan.