Code and Data
Cars in the Wild: Ground truth landmarks with labeled landmark indices for different viewpoints.
Alignment of 3D objects from 2D images is one of the most important and well studied problems in computer vision. A typical object alignment system consists of a landmark appearance model which is used to obtain an initial shape and a shape model which refines this initial shape by correcting the initialization errors. Since errors in landmark initialization from the appearance model propagate through the shape model, it is critical to have a robust landmark appearance model. While there has been much progress in designing sophisticated and robust shape models, there has been relatively less progress in designing robust landmark detection models. In this paper we present an efficient and robust landmark detection model which is designed specifically to minimize localization errors thereby leading to state-of-the-art object alignment performance. We demonstrate the efficacy and speed of the proposed approach on the challenging task of multi-view car alignment.
- A new appearance model that is explicitly designed for accurate localization of parts.
Vector Correlation Filter: The outputs of each feature channel are aggregated to compute the final correlation output which would have a sharp peak at the target location.
The filters for the feature channels are jointly learned.
- A shape model that is explicitly designed to account for occluded landmarks.
We use the Bayesian Partial Shape Inference shape model to regularize the landmarks detected by the appearance model.
A comparison between two different appearance models used with the same shape model for the task of car alignment. Top Row: car alignment with a random forest based landmark appearance model. Bottom Row: car alignment with the proposed landmark detector.
Comparison of the sorted RMSE for each pose for different appearance models along with example alignment results with the VCF appearance model corresponding to small, medium and large landmark RMSE.
Yan Li, Leon Gu, Takeo Kanade, Robustly Aligning a Shape Model and Its Application to Car Alignment of Unknown Pose, IEEE Transactions on Pattern Analysis and Maching Intelligence, 2011
Yan Li, Leon Gu, and Takeo Kanade, A Robust Shape Model for Multi-View Car Alignment, CVPR 2009