RECONSTRUCTING PART-LEVEL 3D MODELS FROM A SINGLE IMAGE

Dingfeng Shi1
Yifan Zhao1
Jia Li1,2*

1State Key Laboratory of Virtual Reality Technology and Systems, SCSE, Beihang University

2Beijing Advanced Innovation Center for Big Data and Brain Computing, Beihang University

ICME 2020

The overview of framework

Abstract

Understanding an image with 3D representations has been an increasingly attractive topic in computer vision. The state-ofthe-art 3D reconstruction methods usually focus on the reconstruction of the holistic object, while missing important part information, which is crucial in robotic interaction and virtual reality applications. To solve this issue, we make the first attempt to reconstruct the 3D models with part-level representations in a unified framework. With the input of the singleview images, we first develop a feature enhancement encoder to incorporate discriminative local features into the feature representation. The local features are selected adaptively by a learnable local awareness module. Then the enhanced local features are fused with the global branch to form the 3D representations. We then develop a 3D part generator to decode the image priors to 3D parts with a 3D focal loss, which enables the representations of small parts. Experimental results indicate that our model generates reliable part-level structures while achieving state-of-the-art performance in object-level recovering.

The Local Awareness Module

Visualization

Benchmark

BibTex Citation

@inproceedings{shi2020reconstructing,
title={Reconstructing Part-Level 3D Models From a Single Image},
author={Shi, Dingfeng and Zhao, Yifan and Li, Jia},
booktitle={2020 IEEE International Conference on Multimedia and Expo (ICME)},
pages={1--6},
year={2020},
organization={IEEE}
}