ROC#

class scio.eval.ROC(labels, scores)[source]#

Bases: object

ROC utility for Discriminative Power and visualization.

We recall that a Discriminative Power only depends on the Pareto front of all the \((FP, TP)\) tuples when thresholding with every possible threshold. Per convention:

The thresholding test is score <= threshold.
Positive (i.e. OoD) samples should verify this and thus have a low score.
Scores must not be nan.

Parameters:

labels (ArrayLike) – The label of samples, interpreted as bool. Shape (n_samples,).
scores (ArrayLike) – The score of samples. Shape (n_samples,).

Raises:

AssertionError – If there is no positive (resp. negative) labels.
AssertionError – If there is at least one nan score.

Note

In cases where the ROC curve would theoretically begin with a nonzero FPR, we artificially add the point \((0, 0)\) — corresponding to the trivial False classifier —, for consistency in Discriminative Power definitions. This situation arises when a negative (i.e. InD) sample is assigned a score of \(-\infty\).

Useful methods defined here

plot(*[, legend, ax])

Plot the ROC curve.

Useful attributes defined here

`FN`	False Negative points.
`FNR`	False Negative Rate points.
`FNRch`	Same as `FNR`, reduced to convex hull front.
`FNch`	Same as `FN`, reduced to convex hull front.
`FP`	False Positive points.
`FPR`	False Positive Rate points.
`FPRch`	Same as `FPR`, reduced to convex hull front.
`FPch`	Same as `FP`, reduced to convex hull front.
`N`	Total number of negative samples.
`P`	Total number of positive samples.
`TN`	True Negative points.
`TNR`	True Negative Rate points.
`TNRch`	Same as `TNR`, reduced to convex hull front.
`TNch`	Same as `TN`, reduced to convex hull front.
`TP`	True Positive points.
`TPR`	True Positive Rate points.
`TPRch`	Same as `TPR`, reduced to convex hull front.
`TPch`	Same as `TP`, reduced to convex hull front.
`pareto`	Ordered \((FP, TP)\) tuples defining the Pareto front.
`pareto_ch`	Same as `pareto`, reduced to convex hull front.
`thresholds`	The threshold intervals associated with Pareto points.
`thresholds_ch`	Same as `thresholds`, reduced to convex hull front.
`tot`	Total population size.

plot(*, legend=False, ax=None)[source]#

Plot the ROC curve.

Parameters:

legend (bool) – Whether to display the legend. Defaults to False.
ax (plt.Axes, optional) – If provided, where the curve is plotted.