Datasets and Evaluation – Computer Vision and Robotics Laboratory

Low-Level Multiscale Image Segmentation and a Benchmark for its Evaluation

Posted on August 25, 2021August 25, 2021 by 1qs9y

We present a segmentation algorithm to detect low-level structure present in images. The algorithm is designed to partition a given image into regions, corresponding to image structures, regardless of their shapes, sizes, and levels of interior homogeneity. We model a region as a connected set of pixels that is surrounded by ramp edge discontinuities where the magnitude of these discontinuities is large compared to the variation inside the region.

Sound2Sight: Generating Visual Dynamics from Sound and Context

Posted on August 25, 2021August 25, 2021 by 1qs9y

Learning associations across modalities is critical for robust multimodal reasoning, especially when a modality may be missing during inference. In this paper, we study this problem in the context of audio-conditioned visual synthesis – a task that is important, for example, in occlusion reasoning. Specifically, our goal is to generate future video frames and their motion dynamics conditioned on audio and a few past frames.

Unsupervised 3D Pose Estimation for Hierarchical Dance Video

Posted on August 25, 2021August 25, 2021 by 1qs9y

Dance experts often view dance as a hierarchy of information, spanning low-level (raw images, image sequences), mid-levels (human poses and bodypart movements), and high-level (dance genre). We propose a Hierarchical Dance Video Recognition framework (HDVR). HDVR estimates 2D pose sequences, tracks dancers, and then simultaneously estimates corresponding 3D poses and 3D-to-2D imaging parameters, without requiring ground truth for 3D poses.

Visual Scene Graphs for Audio Source Separation

Posted on August 25, 2021August 25, 2021 by 1qs9y

State-of-the-art approaches for visually-guided audio source separation typically assume sources that have characteristic sounds, such as musical instruments. These approaches often ignore the visual context of these sound sources or avoid modeling object interactions that may be useful to characterize the sources better, especially when the same object class may produce varied sounds from distinct interactions.

A Hierarchical Variational Neural Uncertainty Model for Stochastic Video Prediction

Posted on August 25, 2021August 25, 2021 by 1qs9y

Predicting the future frames of a video is a challenging task, in part due to the underlying stochastic real-world phenomena. Prior approaches to solve this task typically estimate a latent prior characterizing this stochasticity, however do not account for the predictive uncertainty of the (deep learning) model. Such approaches often derive the training signal from the mean-squared error (MSE) between the generated frame and the ground truth, which can lead to sub-optimal training, especially when the predictive uncertainty is high.

Compression of Image-based Rendering Data

Posted on November 9, 2003July 28, 2021 by 1qs9y

Video Compression using Wyner-Ziv Codes

Predictive coding is posed as a variant of the Wyner-Ziv coding, and problems in source and channel coding of video are addressed in this framework.

Compression of Image-based Rendering Data

The design of compression techniques for streaming of image-based rendering data to remote viewers. A compression algorithm based on the use of Wyner-Ziv codes is proposed, which satisfies the key constraints for IBR streaming, namely those of random access for interactivity, and pre-compression.