GradNorm#

class scio.scores.GradNorm(*, act_norm=None, mode='raw', temperature=1.0, grad_norm=1.0, discard_functional_forward=False)[source]#

Bases: BaseScoreClassif

GradNorm for classification.

Personal interpretation: The computed gradient norm mimicks brain elasticity estimation. The network should be more elastic in In-Distribution regions.

Parameters:

  • temperature (float) – Temperature scaling factor. Defaults to 1.0.

  • grad_norm (float) – Order of the vector norm used for gradnorm computation. Defaults to 1.0.

  • discard_functional_forward (bool) – Whether to compute output from vanilla forward, if the implemented functional call is not satisfactory. Setting this to True requires an additional forward pass through the network. Defaults to False.

  • mode – See BaseScoreClassif.

  • act_norm – See BaseScore.

References

[HGL21]

Rui Huang, Andrew Geng, and Yixuan Li. On the importance of gradients for detecting distributional shifts in the wild. In Advances in Neural Information Processing Systems, volume 34, 677–689. 2021. URL: https://proceedings.nips.cc/paper_files/paper/2021/file/063e26c670d07bb7c4d30e6fc69fe056-Paper.pdf.

Hint

Below this point, the documentation is meant for development purposes only. Manual use of any listed member is highly discouraged. For usage, see Inferring with Confidence.

Useful methods defined here

params_gradients_with_logits(params, inputs)

In batch: computes gradients, flattens and concatenates.

to_derive(params, sample)

Quantity to derive as a function of selected parameters.
params_gradients_with_logits(params, inputs)[source]#

In batch: computes gradients, flattens and concatenates.

to_derive(params, sample)[source]#

Quantity to derive as a function of selected parameters.