Auto-Directed Video Stabilization with Robust L1 Optimal Camera Paths

Five stills from our video stabilization with saliency constraints using a face detector. Original frames on top, our face-directed final result at the bottom. The resulting optimal path is essentially static in y (the up and down motion of camera is completely eliminated) and composed of linear and parabolic segments in x. Our path centers the object of interest (jumping girl) in the middle of the crop window (bottom row) without sacrificing smoothness of the path. Please see accompanying video.
Authors
Matthias Grundmann
Vivek Kwatra
Irfan Essa
Abstract
We present a novel algorithm for automatically applying constrainable, L1-optimal camera paths to generate stabilized videos by removing undesired motions. Our goal is to compute camera paths that are composed of constant, linear and parabolic segments mimicking the camera motions employed by professional cinematographers. To this end, our algorithm is based on a linear programming framework to minimize the first, second, and third derivatives of the resulting camera path. Our method allows for video stabilization beyond the conventional filtering of camera paths that only suppresses high frequency jitter. We incorporate additional constraints on the path of the camera directly in our algorithm, allowing for stabilized and retargeted videos. Our approach accomplishes this without the need of user interaction or costly 3D reconstruction of the scene, and works as a post-process for videos from any camera or from an online source.
Paper

Download PDF
Video
Slides
Auto-Directed Video Stabilization slides.

Download slides (key-note file, ~350 MB)
Note: This file is password protected (under OSX use unzip command to extract).
Please request password by via e-mail



General introduction to LinearProgramming.
Uses linked mathematica scripts, that can be viewed with the free Wolfram CDF Player
Download slides (key-note file, ~1 MB)
Press coverage
Citation
@INPROCEEDINGS{GrundmannKwatra2011,
author = {M. Grundmann and V. Kwatra and I. Essa },
title = {Auto-Directed Video Stabilization with Robust L1 Optimal Camera Paths},
booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2011},
}  
Dataset
You can download the original videos used in our video and in the paper.
stabilization.zip (465 MB) [passwd protected, see below]
Distribution is permitted under the fair-use clause for academic purposes. Please request the password for the zip file via e-mail
.
Copyright

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without explicit permission of the copyright holder.