Synthesis

class Synthesis

Instantiate Cool-chic convolution-based synthesis transform. It performs the following operation.

\[\hat{\mathbf{x}} = f_{\theta}(\hat{\mathbf{z}}).\]

Where \(\hat{\mathbf{x}}\) is the \([B, C_{out}, H, W]\) synthesis output, \(\hat{\mathbf{z}}\) is the \([B, C_{in}, H, W]\) synthesis input (i.e. the upsampled latent variable) and \(\theta\) the synthesis parameters.

The synthesis is composed of one or more convolution layers, instantiated using the class SynthesisConv2d. The parameter layers_dim set the synthesis architecture. Each layer is described as follows: <output_dim>-<kernel_size>-<type>-<non_linearity>

• output_dim: number of output features \(C_{out}\).

• kernel_size: spatial dimension of the kernel. Use 1 to mimic an MLP.

• type: either linear or residual i.e.

  \[\begin{split}\mathbf{y} = \begin{cases} \mathrm{conv}(\mathbf{x}) + \mathbf{x} & \text{if residual,} \\ \mathrm{conv}(\mathbf{x}) & \text{otherwise.} \\ \end{cases}\end{split}\]

• non_linearity: either none (no non-linearity) or relu.

  The non-linearity is applied after the residual connexion if any.

Example of a convolution layer with 40 input features, 3 output features, a residual connexion followed with a relu: 40-3-residual-relu

__init__(input_ft: int, layers_dim: List[str])

Parameters:

• input_ft (int) – Number of input features \(C_{in}\). This corresponds to the number of latent features.

• layers_dim (List[str]) – Description of each synthesis layer as a list of strings following the notation detailed above.

forward(x: Tensor) → Tensor

Perform the synthesis forward pass \(\hat{\mathbf{x}} = f_{\theta}(\hat{\mathbf{z}})\), where \(\hat{\mathbf{x}}\) is the \([B, C_{out}, H, W]\) synthesis output, \(\hat{\mathbf{z}}\) is the \([B, C_{in}, H, W]\) synthesis input (i.e. the upsampled latent variable) and \(\theta\) the synthesis parameters.

Parameters:

x (Tensor) – Dense latent representation \([B, C_{in}, H, W]\).

Returns:

Raw output features \([B, C_{out}, H, W]\).

Return type:

Tensor

get_param() → OrderedDict[str, Tensor]

Return a copy of the weights and biases inside the module.

Returns:

A copy of all weights & biases in the layers.

Return type:

OrderedDict[str, Tensor]

set_param(param: OrderedDict[str, Tensor])

Replace the current parameters of the module with param.

Parameters:

param (OrderedDict[str, Tensor]) – Parameters to be set.

reinitialize_parameters() → None

Re-initialize in place the params of all the SynthesisConv2d layers.

Return type:

None

class SynthesisConv2d

Instantiate a synthesis layer applying the following operation to an input tensor \(\mathbf{x}\) with shape \([B, C_{in}, H, W]\), producing an output tensor \(\mathbf{y}\) with shape \([B, C_{out}, H, W]\).

\[\begin{split}\mathbf{y} = \begin{cases} \mathrm{conv}(\mathbf{x}) + \mathbf{x} & \text{if residual,} \\ \mathrm{conv}(\mathbf{x}) & \text{otherwise.} \\ \end{cases}\end{split}\]

__init__(

in_channels: int,

out_channels: int,

kernel_size: int,

residual: bool = False,

)

Parameters:

• in_channels (int) – Number of input channels \(C_{in}\).

• out_channels (int) – Number of output channels \(C_{out}\).

• kernel_size (int) – Kernel size (height and width are identical)

• residual (bool) – True to add a residual connexion to the layer. Default to False.

forward(x: Tensor) → Tensor

Perform the forward pass of this layer.

Parameters:

x (Tensor) – Input tensor of shape \([B, C_{in}, H, W]\).

Returns:

Output tensor of shape \([B, C_{out}, H, W]\).

Return type:

Tensor

initialize_parameters() → None

Initialize in place the weights and biases of the SynthesisConv2d layer.

• Biases are always set to zero.

• Weights are set to zero if residual is True. Otherwise, they follow a Uniform distribution: \(\mathbf{W} \sim \mathcal{U}(-a, a)\), where \(a = \frac{1}{C_{out}^2\sqrt{C_{in}k^2}}\) with \(k\) the kernel size.

Return type:

None