Autoregressive Label-Conditioned Autoencoder for Controllable Image-To-Video Generation

Abstract

Generating videos from a single image with user-controlled attributes is a complex challenge in the field of computer vision, despite the significant advancements recently made in the field. This paper presents a novel approach to tackle this issue, leveraging a convolutional autoencoder with supervised principal component analysis and autoregressive inference step. The efficacy of the proposed method is evaluated on two datasets – MNIST handwritten-digits and time-lapse photos of the sky. Results from both quantitative and qualitative analyses show that the proposed approach produces high-quality videos of variable duration with user-defined attributes, while preserving the integrity of original image contents.