Synthesis

class Synthesis

Instantiate Cool-chic convolution-based synthesis transform. It performs the following operation.

\[\hat{\mathbf{x}} = f_{\theta}(\hat{\mathbf{z}}).\]

Where \(\hat{\mathbf{x}}\) is the \([B, C_{out}, H, W]\) synthesis output, \(\hat{\mathbf{z}}\) is the \([B, C_{in}, H, W]\) synthesis input (i.e. the upsampled latent variable) and \(\theta\) the synthesis parameters.

The synthesis is composed of one or more convolution layers, instantiated using the class SynthesisConv2d. The parameter layers set the synthesis architecture. Each layer is described as follows: <output_dim>-<kernel_size>-<type>-<non_linearity>

• output_dim: number of output features \(C_{out}\).

• kernel_size: spatial dimension of the kernel. Use 1 to mimic an MLP.

• type: either linear or residual i.e.

  \[\begin{split}\mathbf{y} = \begin{cases} \mathrm{conv}(\mathbf{x}) + \mathbf{x} & \text{if residual,} \\ \mathrm{conv}(\mathbf{x}) & \text{otherwise.} \\ \end{cases}\end{split}\]

• non_linearity: either none (no non-linearity) or relu.

  The non-linearity is applied after the residual connection if any.

Example of a convolution layer with 40 input features, 3 output features, a residual connection followed with a relu: 40-3-residual-relu

__init__(input_ft, layers, flag_linear_stabiliser=True, flag_common_randomness=False)

Parameters:

• input_ft (int) – Number of input features \(C_{in}\). This corresponds to the number of latent features.

• layers (List[str]) – Description of each synthesis layer as a list of strings following the notation detailed above.

• flag_linear_stabiliser (bool) – True to add a linear stabiliser running parallel to the main trunk layers, as presented in the diagram below.

• flag_common_randomness (bool) – Set to true if half of the input features are common randomness features. In this case, the stabiliser layer does not take the common randomness features and as thus \(\frac{C_{in}}{2}\) input features.

       ┌──────┐   ┌──────┐    ┌──────┐   ┌──────┐ trunk ┌─────┐
x ──►──┤ Conv ├─►─┤ ReLU ├─►──┤ Conv ├─►─┤ ReLU ├───────┤  +  ├─► (mu, logscale)
│      └──────┘   └──────┘    └──────┘   └──────┘       └─────┘
▼                                                          ▲
│                      ┌─────┐                 stabiliser  │
└──►───────────────────┤ Lin ├─────────────────────────────┘
                       └─────┘

forward(x)

Perform the synthesis forward pass \(\hat{\mathbf{x}} = f_{\theta}(\hat{\mathbf{z}})\), where \(\hat{\mathbf{x}}\) is the \((B, C_{out}, H, W)\) synthesis output, \(\hat{\mathbf{z}}\) is the \((B, C_{in}, H, W)\) synthesis input (i.e. the upsampled latent variable) and \(\theta\) the synthesis parameters.

Parameters:

x (Tensor) – Dense latent representation \((B, C_{in}, H, W)\).

Returns:

Raw output features \((B, C_{out}, H, W)\).

Return type:

Tensor

get_param(which=None)

Return a copy of the weights and biases inside the module.

Parameters:

which (Optional[Literal[``”weight”, ``"bias"]]) – Wether to return only the weights or the biases. If None, return everything. Defaults to None.

Returns:

A copy of all weights & biases in the layers.

Return type:

OrderedDict[str, Tensor]

set_param(param)

Replace the current parameters of the module with param.

Parameters:

param (OrderedDict[str, Tensor]) – Parameters to be set.

reinitialize_parameters()

Re-initialize in place the params of all the SynthesisConv2d layers.

Return type:

None