Regularizers

TV(; <keyword arguments>)

This function returns a function to calculate the Total Variation regularizer of a n-dimensional array.

Arguments

• num_dims=nothing: Dimension of the array that should be regularized. When nothing, it is inferred from the array upon use.
• sum_dims=nothing: A array containing the dimensions we want to sum over. Defaults to all dimensions (1:N).
• weights=nothing: A array containing weights to weight the contribution of different dimensions. If weights=nothing all dimensions are weighted equally.
• step=1: A integer indicating the step width for the array indexing
• mode="forward": Either "central" or "forward" accounting for different modes of the spatial gradient. Default is "forward".
• ϵ=1f-8: A constant which is added to the squared spatial gradient before taking the square root, so that the Total Variation is smoothly differentiable: L = sqrt.(grad² + ϵ). In the limit ϵ → 0 this reduces to the pure gradient magnitude sqrt.(grad²) (i.e. abs.(grad)), while a larger ϵ smoothly interpolates toward grad-squared (compare with Tikhonov(mode="spatial_grad_square")).

Examples

To create a regularizer for a 3D dataset where the third dimension has different contribution. For the derivative we use forward mode.

julia> reg = TV(num_dims=2, sum_dims=[1, 2], weights=[1, 1], mode="forward");

julia> reg([1 2 3; 4 5 6; 7 8 9])
12.649111f0

Tikhonov(; <keyword arguments>)

This function returns a function to calculate the Tikhonov regularizer of a n-dimensional array.

Arguments

• num_dims=nothing: Dimension of the array that should be regularized. When nothing, it is inferred from the array upon use.
• sum_dims=nothing: A array containing the dimensions we want to sum over. Defaults to all dimensions (1:N).
• weights=nothing: A array containing weights to weight the contribution of different dimensions. If weights=nothing all dimensions are weighted equally.
• step=1: A integer indicating the step width for the array indexing
• mode="laplace": Either "laplace", "spatial_grad_square", "identity" accounting for different modes of the Tikhonov regularizer. Default is "laplace".

Examples

To create a regularizer for a 3D dataset where the third dimension has different contribution.

julia> reg = Tikhonov(num_dims=2, sum_dims=[1, 2], weights=[1, 1], mode="identity");

julia> reg([1 2 3; 4 5 6; 7 8 9])
285

GR(; <keyword arguments>)

This function returns a function to calculate the Good's roughness regularizer of a n-dimensional array.

Arguments

• num_dims=nothing: Dimension of the array that should be regularized. When nothing, it is inferred from the array upon use.
• sum_dims=nothing: A array containing the dimensions we want to sum over. Defaults to all dimensions (1:N).
• weights=nothing: A array containing weights to weight the contribution of different dimensions. If weights=nothing a weight of 1 is assumed for each dimension.
• step=1: A integer indicating the step width for the array indexing
• mode="forward": Either "central" or "forward" accounting for different modes of the spatial gradient. Default is "forward".
• ϵ=1f-8: A constant which is added to the array before evaluating the spatial gradient, so that the Good's roughness sqrt.(x + ϵ) is smoothly differentiable everywhere. In the limit ϵ → 0 the expression reduces to the pure sqrt.(x), while a larger ϵ smooths the transition.

Examples

To create a regularizer for a 3D dataset where the third dimension has different contribution. For the derivative we use forward mode.

julia> reg = GR(num_dims=2, sum_dims=[1, 2], weights=[1, 1], mode="forward");

julia> reg([1 2 3; 4 5 6; 7 8 9])
-26.36561871738898

TH(; <keyword arguments>)

This function returns a function to calculate the Total Hessian norm of a n-dimensional array.

Arguments

• num_dims=nothing: Number of spatial dimensions (1, 2 or 3) of the array. When nothing, it is inferred from the array upon use.
• sum_dims=nothing: An array containing the dimensions over which the Hessian is computed. Defaults to all dimensions (1:num_dims). Dimensions not listed here only contribute to the summation over the array (the @tullio reduction), but not to the Hessian itself.
• weights=nothing: An array containing weights to weight the contribution of different dimensions. If weights=nothing all dimensions are weighted equally. The weights are matched positionally to sum_dims, i.e. the weight weights[k] belongs to the dimension sum_dims[k]. A diagonal term of dimension i enters with weights[i]^2 and the cross term between dimensions i and j enters with 2 * weights[i] * weights[j].
• ϵ=1f-8: A constant which is added to the squared Hessian entries before taking the square root, so that the Total Hessian norm is smoothly differentiable at zero. In the limit ϵ → 0 the expression reduces to the pure Hessian norm sqrt.(H²), while a larger ϵ smooths the transition (compare with TV, where ϵ plays the same role).

HS(; p=1, sum_dims=nothing, weights=nothing)

Hessian Schatten norm. p determines which Schatten norm is used.

The Hessian is evaluated over exactly two array dimensions, which are given by sum_dims (defaulting to (1, 2)). The selected dimensions may be a subset of the array dimensions, so e.g. sum_dims=(1, 2) works for both 2D and 3D (or higher dimensional) arrays – the remaining dimensions only take part in the final summation. Because the two rows and columns of the 2×2 pixel-wise Hessian are inverted in closed form for speed, sum_dims is limited to exactly two dimensions; passing more throws an ArgumentError.

weights[k] pair positionally with sum_dims[k]: the diagonal Hessian entry of dimension d enters with weights[k]^2 and the cross entry between the two dimensions enters with 2 * weights[k] * weights[l].

The pixel-wise Hessian uses centered (symmetric) second-order stencils, so the norm has no preferred direction. The computation is regularized by an internal smoothing constant 1f-8 (see ϵ in TV/TH), so that the norm is differentiable.

Arguments

• p=1: The order p of the Schatten norm. p=1 uses a cheap fast path.
• sum_dims=nothing: The two array dimensions over which the Hessian is evaluated. Defaults to (1, 2). Exactly two dimensions are supported.
• weights=nothing: Weights for the different dimensions, matched positionally to sum_dims. If nothing all dimensions are weighted equally.

TV_cuda(; num_dims=nothing, sum_dims=nothing, weights=nothing, step=1, mode="forward", ϵ=1f-8)

This function returns a function to calculate the Total Variation regularizer of a n-dimensional array.

Arguments

• num_dims=nothing: Dimension of the array that should be regularized. When nothing, it is inferred from the array upon use.
• sum_dims=nothing: A array containing the dimensions we want consider in the TV calculation. Defaults to all dimensions (1:N).
• weights=nothing: A array containing weights to weight the contribution of different dimensions. If weights=nothing all dimensions are weighted equally.
• step=1: A integer indicating the step width for the array indexing
• mode="forward": Either "central" or "forward" accounting for different modes of the spatial gradient. Default is "forward".
• ϵ=1f-8: A constant which allows to smoothly vary between TV and grad^2 regularization: L = sqrt.(grad^2+ϵ).

julia> using CUDA

julia> reg = TV_cuda(num_dims=2);

julia> reg(CuArray([1 2 3; 4 5 6; 7 8 9]))
12.649111f0

GR_cuda(; num_dims=nothing, sum_dims=nothing, weights=nothing, step=1, mode="forward", ϵ=1f-8)

This function returns a function to calculate the Good's roughness regularizer of a n-dimensional array on CUDA (or CPU) arrays.

Differentiable with Zygote on CuArrays because it avoids Tullio and only uses view/broadcast/sum operations. The math is identical to GR (see GR for a description of the arguments).

If num_dims is nothing, the number of dimensions is inferred from the array upon use. The default weights then assume 1 for each of the num_dims dimensions.

TH_cuda(; num_dims=nothing, sum_dims=nothing, weights=nothing, ϵ=1f-8)

This function returns a function to calculate the Total Hessian norm of a n-dimensional array on CUDA (or CPU) arrays.

Differentiable with Zygote on CuArrays. The math is identical to TH (see TH for the arguments).

When sum_dims is given, the Hessian is only computed over the listed dimensions (the remaining dimensions are summed over); the generic TH_view fallback is used then.

Tikhonov_cuda(; num_dims=nothing, sum_dims=nothing, weights=nothing, step=1, mode="laplace")

This function returns a function to calculate the Tikhonov regularizer of an n-dimensional array on CUDA (or CPU) arrays.

Differentiable with Zygote on CuArrays because it avoids Tullio and only uses view/broadcast/sum operations. The math is identical to Tikhonov (see Tikhonov for a description of the arguments).

HS_cuda(; p=1, sum_dims=nothing, weights=nothing)

This function returns a function to calculate the Hessian Schatten norm of an n-dimensional array on CUDA (or CPU) arrays.

Differentiable with Zygote on CuArrays because it avoids Tullio and only uses view/broadcast/sum operations. The math is identical to HS (see HS for a description of the arguments).