When dealing with point clouds (reordered and reshaped to a H × W × 3 size and using c = C / 2) we approximate, with operations in blue, global features in coupling layers while still being invertible. To the best of our knowledge there is no previous work using such preprocessing for point clouds.įigure 4: Approximating global features in point clouds. Notice that not only we can establish a neighborhood relationship but also a semantically-stable ordering ( e.g. For each training sample, we project its points into a 3D Hilbert curve and reorder them based on their ordering along the curve (Fig. In this paper we therefore consider using the approximation provided by the Hilbert’s space-filling curve algorithm. As discussed in, for high dimensional spaces it is not possible to produce a perfect ordering stable to point perturbations. To fulfill this condition on unordered point clouds, we propose to sort them based on proximity. C-Flow is based on convolutional layers which require input data with a local neigboorhood consistent across samples. (i) Approximate Sorting with Space-Filling Curves. By sequentially concatenating these bijective operations we can transform data points x into their latent representation y (forward propagation) and vice versa (backward propagation). The first halves of all tensors are not updated. The second half of x B conditioned on the first halves of x A and x B. Given two input tensors x A and x B, the proposed conditional coupling layer transforms (a) Forward Propagationįigure 2: Conditional coupling layer for forward and backward propagation. For a detailed description on functions in grey refer to. This scheme allows sampling x B conditioned on x A. 4 Conditional Flow-Based Generative Modelįigure 1: The C-Flow model consists of two parallel flow branches mutually interconnected with conditional coupling layers. This is one of the major virtues of flow-based approaches. Note that optimizing over the actual log-likelihood of the observations is more stable and informative than doing it over a lower-bound of the log-likelihood for VAEs, or minimizing the adversarial loss in GANs. The negative log-likelihood over the observations D. Since we can now compute the exact log-likelihood, the training criterion of flow-based generative model is Where ∂ h i / ∂ h i − 1 is the Jacobian matrix of g − 1 i at h i − 1 and the Jacobian determinant measures the change of log-density made by g − 1 i when transforming h i − 1 to h i. = log p ϑ ( z ) + K ∑ i = 1 log | det ( ∂ h i / ∂ h i − 1 ) | = log p ϑ ( z ) + log | det ( ∂ z / ∂ x ) | This scheme allows guiding a source domain towards a target domain guaranteeing the satisfaction of the aforementioned properties of flow-based models.Ĭonditional inference is then implemented in a simple manner, by (exactly) embedding the source sample into its latent space, sampling a point from a Gaussian prior, and then propagating them through the learned normalizing flow.įor example, for the application of synthesizing multiple plausible photos given a semantic segmentation mask, each image is generated by jointly propagating the segmentation embedding and a random point drawn from a prior distribution across the learned flow. For this purpose, we introduce C-Flow, a framework consisting of two parallel flow branches, interconnected across their reversible functions using conditional coupling layers and trained with an invertible cycle consistency. The main contribution of this paper is a novel approach to condition normalizing flows, making it possible to perform multi-modality transfer tasks which have so far not been explored under the umbrella of flow-based generative models. Images, segmentation maps, and edge masks. Multi-modal image-to-image mapping in a diversity of domains, including RGB We demonstrate our conditioning method to be veryĪdaptable, being also applicable to image manipulation, style transfer and Reconstruction from a single image and its inverse problem of rendering an WeĪlso devise a new strategy to model unordered 3D point clouds that, inĬombination with the conditioning scheme, makes it possible to address 3D Sequence of invertible mappings in which a source flow guides the target flowĪt every step, enabling fine-grained control over the generation process. Possibilities for multi-modal data modeling. Scheme that brings normalizing flows to an entirely new scenario with great In this paper, we introduce C-Flow, a novel conditioning Still in their infancy and have not received as much attention as alternative Log-likelihood evaluation and exact latent-variable inference, however they are Flow-based generative models have highly desirable properties like exact

Modelio conditional flow