# Jingwei Huang

Ph.D Candidate

Geometric Computing Lab
Dept. of Computer Science
Stanford University, USA

Office: S296 James H. Clark Center
Stanford, CA 94305
Email: jingweih@stanford.edu

github / google scholar / publication

Research Philosophy

TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes

Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation

Spherical CNNs on Unstructured Grids

Convolutional Neural Networks on non-uniform geometrical signals using Euclidean spectral transformation

QuadriFlow: A Scalable and Robust Method for Quadrangulation

3DLite: Towards Commodity 3D Scanning for Content Creation

6-DOF VR Videos with a Single 360-Camera

Unsupervised Texture Transfer from Images to Model Collections

Real-time Interactive Tree Animation

Automatic Thumbnail Generation Based on Visual Representativeness and Foreground Recognizability

A surface approximation method for image and video correspondences

Stereo Panorama Video Stitching

RGB-D SLAM and Real-time Reconstruction

### Gallery (Click on the Images!)

#### Stereo Panorama | #### Indoor Reconstruction | #### Object Reconstruction | #### QuadMesh Conversion
--- | --- | --- | ---

### Research Statement

My research interests are computer graphics, computer vision, virtual reality and geometry processing. My goal is to provide ready-to-use scanning geometry and offer high-level understanding to the 3D data.

### Skills

Proficient with C/C++ Programming

Good at Python/Matlab/CUDA/GLSL/HLSL GPU Programming

Familiar with Pytorch/Tensorflow

Familiar with Java/C# Programming

Familiar with PHP/Javascript/Unity/Android programming

### My Mission

To my knowledge, there are creative ideas and awesome applications emerging every year, and the demos are very fancy. However, most of these potential applications can hardly be used in common days, mostly due to the problem of robustness in graphics or poor accuracy in vision. I view my mission as to solve problems in computer graphics and vision, particularly in environment reconstruction from images and video.

Recently, my proposed solution is to reconstruct and quantize the 3D data as CAD models and textures on top of that. I have solved the problem of surface parametrization to a industry useable level (quadrangulation, see my QuadriFlow project), and will use this as a foundation to solve the 3D texture generation. Hopefully, I will find practical solutions and bring them to market in the future.

### PhD Thesis

Surface Texture Processing
Jingwei Huang
Stanford University
PDF Video Slides

### Publications

#### Deep Learning

Adversarial Texture Optimization from RGB-D Scans
Jingwei Huang, Justus Thies, Angela Dai, Abhijit Kudu, Chiyu "Max" Jiang, Leonidas Guibas, Matthias Niessner, Thomas Funkhouser
CVPR 2020
PDF Website Code Video
Joint texture optimization with a learned deep metric which is robust to complex scanning errors.
Learning Local Implicit Grid Representation for 3D Scenes
Chiyu "Max" Jiang, Avneesh Sud, Ameesh Makadia, Jingwei Huang, Matthias Niessner, Thomas Funkhouser
CVPR 2020
PDF Code Video
Learning a chunk-based deep SDF for scene reconstruction.
NeurVPS: Neural Vanishing Point Scanning via Conic Convolution
Yichao Zhou, Haozhi Qi, Jingwei Huang, Yi Ma
NeurIPS 2019
PDF Website Code Slides
Deep learning-based vanishing point detection method: Warp the image into conic space and use CNN to evaluate corresponding potential vanishing point.
FrameNet: Learning Local Canonical Frames of 3D Surfaces from a Single RGB Image
Jingwei Huang, Yichao Zhou, Thomas Funkhouser, Leonidas Guibas
ICCV 2019
PDF Website Code
We introduce the novel problem of identifying dense canonical 3D coordinate frames from a single RGB image. We observe that each pixel in an image corresponds to a surface in the underlying 3D geometry, where a canonical frame can be identified. We propose an algorithm to predict it from RGB.
TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes
Jingwei Huang, Haotian Zhang, Li Yi, Thomas Funkhouser, Matthias Niessner, Leonidas Guibas
CVPR 2019 (Oral Presentation)
PDF Website Code
We introduce, TextureNet, a neural network architecture designed to extract features from high-resolution signals associated with 3D surface meshes (e.g., color texture maps). The key idea is to utilize a 4-rotational symmetric (4-RoSy) field to define a domain for convolution on a surface.
Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation
He Wang, Srinath Sridhar, Jingwei Huang, Julien Valentin, Shuran Song, Leonidas Guibas
CVPR 2019 (Oral Presentation)
PDF Code
Estimate 6DOF 3D object pose with unknown categories from 2D images. We create the synthetic dataset by rendering canonical object coordinates to images and learn it from neural networks.
Spherical CNNs on Unstructured Grids
Chiyu 'Max' Jiang, Jingwei Huang, Karthik Kashinath, Prabhat, Philip Marcus, Matthias Niessner
ICLR 2019
PDF Website Code
We present an efficient convolution kernel for Convolutional Neural Networks (CNNs) on unstructured grids using parameterized differential operators while focusing on spherical signals such as panorama images or planetary signals.
Convolutional Neural Networks on non-uniform geometrical signals using Euclidean spectral transformation
Chiyu 'Max' Jiang, Dequan Wang, Jingwei Huang, Philip Marcus, Matthias Niessner
ICLR 2019
PDF Website Code
We develop mathematical formulations for Non-Uniform Fourier Transforms (NUFT) to directly sample nonuniform data signals of different topologies defined on a simplex mesh into the spectral domain with no spatial sampling error.

#### Geometry, VR and 3D Reconstruction

QuadriFlow: A Scalable and Robust Method for Quadrangulation
Jingwei Huang, Yichao Zhou, Matthias Niessner, Jonathan Shewchuk, Leonidas Guibas
Symposium on Geometry Processing, 2018 (Best Paper Award)
PDF Website Code
Robustly and Efficiently Convert Triangle Manifold Meshes to High-quality Quadrilateral Manifold Meshes. It approximates the mixed-integer optimization problem with a maximum flow problem.
3DLite: Towards Commodity 3D Scanning for Content Creation
Jingwei Huang, Angela Dai, Leonidas Guibas, Matthias Niessner
SIGGRAPH Asia, 2017
PDF Website
Complete the geometry and color of 3D scanning model. Detect planar structure of the geometry, extrapolate to complete the scene. Texture optimization by fixing misalignment, motion blur and auto-exposure artifacts.
6-DOF VR Videos with a Single 360-Camera
Jingwei Huang, Zhili Chen, Duygu Ceylan, Hailin Jin
IEEE VR, 2017
PDF Video
Build a VR application that offer 6-DOF freedom with translation and rotation with input as a monoscopic 360 video.

#### Other Computer Graphics Publications

Unsupervised Texture Transfer from Images to Model Collections
Tuanfeng Y. Wang, Hao Su, Qixing Huang, Jingwei Huang, Leonidas Guibas, Niloy J. Mitra
SIGGRAPH Asia, 2016
PDF Website
Transfer textures from product images to 3D shapes. The increased texture variation in ShapeNet is validated to be effective for RenderForCNN. The technique can future be used in improving texture of scanning data.
Real-time Interactive Tree Animation
Ed Quigley, Yue Yu, Jingwei Huang, Winnie Lin, Rod Fedkiw
IEEE TVCG, 2017
PDF Video
Posing and Animating botanical tree models interactively in real time. We use an articulated rigid body model with as-stiff-as-desired rotational springs in conjunction with our newly proposed simulation technique.
Automatic Thumbnail Generation Based on Visual Representativeness and Foreground Recognizability
Jingwei Huang, Huarong Chen, Bin Wang, Stephen Lin
ICCV, 2015
PDF
Generating image thumbnail balancing between visual representativeness and foreground recognizability. We propose a combination of various features and use SVM model to learn good thumbnail.
A Surface Estimation Method for Space-time Correspondences
Jingwei Huang, Bin Wang, Pradeep Sen, Wenping Wang
IEEE Transactions on Image Processing, 2015
PDF
Using B-Spline Approximation and Extension method to refine dense non-rigid correspondences in images or video.

### Other Selected Projects

Real-time RGB-D SLAM and 3D Reconstruction
Jingwei Huang, Matthias Niesser, Leonidas Guibas
In progress
Video
We use plane matching, structural information together with SIFT feature to do real-time RGBD SLAM. The system can get globally consistent structure in a large space with a long scan. Currently, I am trying to use planes to abstract the scenes, and using texture synthesis to achieve a ready-to-use 3D reconstruction.
Stereo Panorama Stitching with 14 GOPROs
Jingwei Huang
Qingxian Tech.
Video
This is what I do as a startup during 3/2015 and 8/2015. The company was purchased by Wei-Shi-Zai-Xian in 3/2016.
PhysBowling: Real-Time Interactive Simulation of Rigid Bodies
Jingwei Huang, Bin Wang
Tsinghua
[Code]
Self-implemented Rigid Body Physics Engine from Scratch.
GPU Acceleration of Incompressible Smoothed Particle Hydrodynamics and Applications
Jingwei Huang, Yun Fei, Bin Wang
Tsinghua
[Code] Video
Using DirectX Parallel Computing (HLSL) to implement a real-time SPH Solver for Water Simulation.
GPU Ray Tracing
Jingwei Huang, Bin Wang
Tsinghua
[Code]
Using OpenGL Parallel Computing (GLSL) to implement a fast GPU ray tracer.

### Working Experience

- Research Intern in Google (Machine Perception Team), 6/2019-9/2019
- Research Intern in Technical University of Munich, 6/2018-9/2018
- Google Software Engineer Intern (PhD, Daydream), 6/2017-9/2017
- Research Intern in Adobe Research, 6/2016 - 9/2016
- CTO and Co-Founder in Qingxian Tech., 3/2015 - 8/2015
- Research Intern in Microsoft Research Asia, 9/2014 - 1/2015
- Research Intern in Stanford Graphics Lab, 6/2014 - 8/2014
- Research Intern in University of Hong Kong, 7/2013 - 9/2013