normalize_samples#

scio.scores.utils.normalize_samples(tensor, *, ord, same_inf=False, sample_start_dim=1)[source]#

Vector-normalizes all samples if ord is provided.

Parameters:

  • tensor (Tensor) – The samples to normalize. Floating point tensor of shape (*batch_shape, *sample_shape). The norm is computed by interpreting every sample as a \(1\)D vector.

  • ord (float | None) – If None, the function returns tensor as is. Otherwise, the norm to use. Must be nonzero. Regarding negative ord, the following is always true element-wise: 1 / normalize_samples(tensor, ord=ord, **kw) == normalize_samples(1 / tensor, ord=-ord, **kw).

  • same_inf (bool) – For simplicity, the following description assumes that ord > 0 and that tensor has positive coordinates. A sample vector norm may be infinite because of a single coordinate or several being infinite. In the former case, the normalized sample is well-defined as [..., 0, 1, 0, ...]. In the latter case, it remains undefined (resulting in nan), unless the infinites are given the same importance. This is what this option allows, resulting in, for example with ord=1 and exactly two infinite coordinates, [..., 0, 0.5, 0,..., 0, 0.5, 0, ...]. It ensures that the output has unit norm even in (nonzero) degenerate cases. Defaults to False.

  • sample_start_dim (int) – Defines sample_shape = tensor.shape[sample_start_dim:]. Defaults to 1.

Returns:

out (Tensor) – Normalized samples. Samples with zero (resp. infinite when ord < 0) vector-norm are unchanged. If not same_inf, samples may be changed to nan in some degenerate cases (see same_inf).

Raises:

  • TypeError – If tensor is not of floating point dtype.

  • ValueError – If ord == 0.

Examples

Z, I = 0, torch.inf
t = torch.tensor([
    [1, 2, 3, 4],
    [Z, 2, 3, 4],
    [Z, Z, 3, 4],
    [Z, Z, I, 4],
    [Z, Z, I, I],
    [1, Z, I, I],
    [1, 2, I, I],
    [1, 2, 3, I],
    [1, Z, I, 4],
    [Z, Z, Z, Z],
    [I, I, I, I],
])

With ord > 0:

>>> normalize_samples(t, ord=1)
tensor([[0.1000, 0.2000, 0.3000, 0.4000],
        [0.0000, 0.2222, 0.3333, 0.4444],
        [0.0000, 0.0000, 0.4286, 0.5714],
        [0.0000, 0.0000, 1.0000, 0.0000],
        [   nan,    nan,    nan,    nan],
        [   nan,    nan,    nan,    nan],
        [   nan,    nan,    nan,    nan],
        [0.0000, 0.0000, 0.0000, 1.0000],
        [0.0000, 0.0000, 1.0000, 0.0000],
        [0.0000, 0.0000, 0.0000, 0.0000],
        [   nan,    nan,    nan,    nan]])
>>> normalize_samples(t, ord=1, same_inf=True)
tensor([[0.1000, 0.2000, 0.3000, 0.4000],
        [0.0000, 0.2222, 0.3333, 0.4444],
        [0.0000, 0.0000, 0.4286, 0.5714],
        [0.0000, 0.0000, 1.0000, 0.0000],
        [0.0000, 0.0000, 0.5000, 0.5000],
        [0.0000, 0.0000, 0.5000, 0.5000],
        [0.0000, 0.0000, 0.5000, 0.5000],
        [0.0000, 0.0000, 0.0000, 1.0000],
        [0.0000, 0.0000, 1.0000, 0.0000],
        [0.0000, 0.0000, 0.0000, 0.0000],
        [0.2500, 0.2500, 0.2500, 0.2500]])

And with ord < 0:

>>> normalize_samples(t, ord=-1)
tensor([[2.0833, 4.1667, 6.2500, 8.3333],
        [1.0000,    inf,    inf,    inf],
        [   nan,    nan,    nan,    nan],
        [   nan,    nan,    nan,    nan],
        [   nan,    nan,    nan,    nan],
        [   inf, 1.0000,    inf,    inf],
        [1.5000, 3.0000,    inf,    inf],
        [1.8333, 3.6667, 5.5000,    inf],
        [   inf, 1.0000,    inf,    inf],
        [   nan,    nan,    nan,    nan],
        [   inf,    inf,    inf,    inf]])
>>> normalize_samples(t, ord=-1, same_inf=True)
tensor([[2.0833, 4.1667, 6.2500, 8.3333],
        [1.0000,    inf,    inf,    inf],
        [2.0000, 2.0000,    inf,    inf],
        [2.0000, 2.0000,    inf,    inf],
        [2.0000, 2.0000,    inf,    inf],
        [   inf, 1.0000,    inf,    inf],
        [1.5000, 3.0000,    inf,    inf],
        [1.8333, 3.6667, 5.5000,    inf],
        [   inf, 1.0000,    inf,    inf],
        [4.0000, 4.0000, 4.0000, 4.0000],
        [   inf,    inf,    inf,    inf]])

Note

Finite but extreme values of ord might result in computation under/overflow, leading to unexpected inf or nan values. For 32 bits inputs in \([0.5, 2]\), we roughly identified 0.15 < |ord| < 115 as a pretty safe range.

Note

This function meaningfully differentiates between 0.0 and -0.0:

>>> normalize_samples(torch.tensor([[1.0, -0.0]]), ord=-1)
tensor([[inf, -1.]])
On this page