Jan Kautz
Senior Director of Visual Computing and Machine Learning Research @ NVIDIA

I lead the Visual Computing Research Group at NVIDIA, working predominantly on computer vision problems — from low-level vision (denoising, super-resolution, computational photography) and geometric vision (structure from motion, SLAM, optical flow) to visual perception (detection, recognition, classification), as well as machine learning problems (deep learning, reinforcement learning, generative models). I am actively hiring for my group.

News
sep
2017
NIPS
We have four papers accepted at NIPS 2017, on semi-supervised optical flow, universal style transfer, learning affinity, and unsupervised image-to-image translation (spotlight).
sep
2017
3DV
We have two papers accepted at 3DV 2017, one on cascaded scene flow (oral) and one on multi-frame scene flow (spotlight).
jul
2017
PRESS
The Verge, Engadget, DPReview, PetaPixel, and New Atlas have written about our Computational Zoom work, which was just presented at ACM SIGGRAPH 2017.
jul
2017
ICCV
We have four papers accepted at ICCV 2017, on 3D reconstruction, on DL-based reflectance acquisition (oral), on video CNNs (oral), and on super-resolution.
apr
2017
SIGGRAPH
We have two papers conditionally accepted at ACM SIGGRAPH 2017, one on computational zoom and one on computational displays.
mar
2017
PRESS
A New York Times article on driver monitoring mentions our work on gaze and head pose tracking, which are part of NVIDIA's AI Co-Pilot.
mar
2017
CVPR
We have three papers accepted at CVPR 2017, as well as a tutorial on Generative Adversarial Networks, and an invited talk at the NITRE workshop.
feb
2017
ICLR
My group has two papers accepted at ICLR 2017, one on pruning neural networks and one on GPU-based reinforcement learning.
jan
2017
CES
My group's work on head tracking and gaze estimation was shown during Jen-Hsun Huang's keynote at CES 2017. See Techcrunch, Engadget, The Verge, and many others.
dec
2016
NIPS
We presented two papers at the NIPS Workshop on Efficient Methods for Deep Neural Networks: on GPU-based reinforcement learning and pruning neural networks.
oct
2016
Technology
Our exposure fusion method from many years ago is used in Google's mobile camera pipeline, see here.
sep
2016
SIGGRAPH
Our paper on automatic bounce flash illumination for portrait photography was accepted at ACM SIGGRAPH Asia 2016.
aug
2016
ACM MM
Our paper on action recognition using deep learning was accepted at ACM Multimedia 2016.
jun
2016
News
Some more news articles on our work with DARPA on Virtual Eye have appeared: Engadget, Digital Trends, Daily Mail, NVIDIA's blog.
may
2016
News
Some news articles on our work with DARPA on Virtual Eye have started to appear: Yahoo, Tech Insider, Business Insider, C4ISR & Networks, Tech.Mic, China Topix.
may
2016
Talk
I am speaking at the LDV Vision Summit 2016 in New York City. The program is very interesting, come join us.
mar
2016
CVPR
We have two papers accepted at CVPR 2016 — one on gesture recognition and one on representing point clouds with GMMs.
oct
2015
3DV
We presented our Maximum Likelihood Mixture Decoupling technique at 3DV (oral presentation). It enables very fast and accurate point cloud registration.
jun
2015
Award
Our paper on hand gesture recognition using 3D convolutional neural networks won the VIVA hand gesture recognition challenge.
jun
2015
HPG
Our paper on accelerated screen-space ray tracing was accepted at High-Per­for­mance Graphics 2015.
may
2015
EGSR
Our paper on physically-based rendering for mixed reality was accepted at the Eurographics Symposium on Rendering 2015.
may
2015
Jobs
My team at NVIDIA Research is continuing to hire in computational photography and computer vision. In addition, we are now hiring in machine learning. See here.
apr
2015
CVPR Workshops
We have one paper accepted at the IEEE Embedded Vision Workshop and one at the IEEE Workshop on Observing and Understanding Hands in Action.
apr
2015
Optics
Our paper on slim near eye displays using pinholes was published in Applied Optics.
mar
2015
CVPR
Our paper on modeling object appearance through context-conditioned component analysis was accepted to CVPR 2015.
mar
2015
Highlight
Our Local Laplacian Filtering method is a research highlight in this month's Communi­cations of the ACM and has made it to the cover page.
oct
2014
Jobs
NVIDIA Research is hiring in computational photography / optics, computer vision, and computational sensing / UI. See here for more details.
oct
2014
NVIDIA
I am now leading the Mobile Visual Computing Research Group at NVIDIA Research.
aug
2014
SIGGRAPH
Our paper on high-qualilty camera image processing was accepted at ACM SIGGRAPH Asia 2014.
Research Areas & Highlights
Low-Level Vision
We work on a variety of low-level vision problems, from image processing problems, such as denoising, demosaicking and super-resolution. Furthermore, we pursue research in computational photography.
Geometric Vision
My team and I are investigating novel approaches for simultaneous localization and mapping (SLAM), efficient 3D reconstruction methods, 3D processing, as well as stereo and optical flow.
Visual Perception
Detection, recognition, and classification are important aspects of our work. In particular, we have focused on gesture recognition and action recognition using machine learning approaches.
Machine Learning
Recently, we started research in machine learning and are building up a group in this area. We are currently conducting research in deep learning, reinforcement learning, and generative models.
Publications
2018
AAAI
Learning Binary Residual Representations for Domain-specific Video Streaming
Y.-H. Tsai, M.-Y. Liu, D. Sun, M.-H. Yang, J. Kautz
AAAI Conference on Artificial Intelligence (AAAI)
December 2017 (spotlight)
2017
NIPS
Unsupervised Image-to-Image Translation
M.-Y. Liu, T. Breuel, J. Kautz
Neural Information Processing Systems (NIPS)
December 2017 (spotlight)
NIPS
Learning Affinity via Spatial Propagation Networks
S. Liu, S. De Mello, J. Gu, G. Zhong, M.-S. Yang, J. Kautz
Neural Information Processing Systems (NIPS)
December 2017
ICCV
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting
R. Maier, K. Kim, D. Cremers, J. Kautz, M. Niessner
IEEE International Conference on Computer Vision (ICCV)
October 2017
ICCV
A Lightweight Approach for On-The-Fly Reflectance Estimation
K. Kim, J. Gu, S. Tyree, P. Molchanov, M. Niessner, J. Kautz
IEEE International Conference on Computer Vision (ICCV)
October 2017 (oral)
3DV
Cascaded Scene Flow Prediction using Semantic Segmentation
Z. Ren, D. Sun, J. Kautz, E. Sudderth
International Conference on 3D Vision
October 2017 (oral)
3DV
Multiframe Scene Flow with Piecewise Rigid Motion
V. Golyanik, K. Kim, R. Maier, M. Niessner, D. Stricker, J. Kautz
International Conference on 3D Vision
October 2017 (spotlight)
SIGGRAPH
Computational Zoom: A Framework for Post-Capture Image Composition
A. Badki, O. Gallo, J. Kautz, P. Sen
ACM Transactions on Graphics (Proceedings SIGGRAPH 2017)
July 2017
SIGGRAPH
Mixed-primary Factorization for Dual-frame Computational Displays
F.-C. Huang, D. Pajak, J. Kim, J. Kautz, D. Luebke
ACM Transactions on Graphics (Proceedings SIGGRAPH 2017)
July 2017
PDF
CVPRW
Reconstructing Intensity Images from Binary Spatial Gradient Cameras
S. Jayasuriya, O. Gallo, J. Gu, T. Aila, J. Kautz
IEEE CVPR Embedded Vision Workshop
July 2017
PDF
CVPR
Dynamic Facial Analysis: From Bayesian Filtering to Recurrent Neural Network
J. Gu, S. De Mello, X. Yang, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
July 2017
CVPR
Polarimetric Multi-View Stereo
Z. Cui, J. Gu, B. Shi, P. Tan, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
July 2017
ICLR
GA3C: GPU-based A3C for Deep Reinforcement Learning
M. Babaeizadeh, I. Frosio, S. Tyree, J. Clemons, J. Kautz
International Conference on Learning Representations
April 2017
ICLR
Pruning Convolutional Neural Networks for Resource Efficient Transfer Learning
P. Molchanov, S. Tyree, T. Aila, T. Karras, J. Kautz
International Conference on Learning Representations
April 2017
PDF
ARXIV
Unsupervised Image-to-Image Translation Networks
M.-Y. Liu, T. Breuel, J. Kautz
ArXiv
March 2017
IEEE TCI
Loss Functions for Neural Networks for Image Processing
H. Zhao, O. Gallo, I. Frosio, J. Kautz
IEEE Transactions on Computational Imaging
3(1), March 2017, pages 47-57
2016
SIGGRAPH ASIA
Computational Bounce Flash for Indoor Portraits
L. Murmann, A. Davis, J. Kautz, F. Durand
ACM Transactions on Graphics (Proceedings SIGGRAPH Asia 2016)
December 2016
NIPSW
GA3C: GPU-based A3C for Deep Reinforcement Learning
M. Babaeizadeh, I. Frosio, S. Tyree, J. Clemons, J. Kautz
NIPS Workshop on Efficient Methods for Deep Neural Networks
December 2016
NIPSW
Pruning Convolutional Neural Networks for Resource Efficient Transfer Learning
P. Molchanov, S. Tyree, T. Aila, T. Karras, J. Kautz
NIPS Workshop on Efficient Methods for Deep Neural Networks
December 2016
PDF
ARXIV
Deep Learning with Energy-efficient Binary Gradient Cameras
S. Jayasuriya, O. Gallo, J. Gu, J. Kautz
arXiv
December 2016
PDF
ARXIV
Learning Adaptive Parameter Tuning for Image Processing
J. Dong, I. Frosio, J. Kautz
arXiv
November 2016
PDF
ACM MM
Multilayer and Multimodal Fusion of Deep Neural Networks for Video Classification
X. Yang, P. Molchanov, J. Kautz
ACM Multimedia
October 2016, pages 978-987, oral
PDF
CVPR
Accelerated Generative Models for 3D Point Cloud Data
B. Eckart, K. Kim, A. Troccoli, A. Kelly, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2016, pages 5497-5505, spotlight oral
CVPR
Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks
P. Molchanov, X. Yang, S. Gupta, K. Kim, S. Tyree, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2016, pages 4207-4215
IEEE IVS
Towards Selecting Robust Hand Gestures for Automotive Interfaces
S. Gupta, P. Molchanov, X. Yang, K. Kim, S. Tyree, J. Kautz
IEEE Intelligent Vehicles Symposium 2016
June 2016
PDF
2015
ICCV
Robust Model-based 3D Head Pose Estimation
G. P. Meyer, S. Gupta, I. Frosio, D. Reddy, J. Kautz
IEEE International Conference on Computer Vision (ICCV)
December 2015, pages 3649-3657
CGF
Interactive Sketch-Driven Image Synthesis
D. Turmukhambetov, N. Campbell, D. Goldman, J. Kautz
Computer Graphics Forum
34(8), December 2015, pages 130-142
UIST
Joint 5D Pen Input for Light Field Displays
J. Tompkin, S. Muff, J. McCann, H. Pfister, J. Kautz, M. Alexa, W. Matusik
ACM User Interface Software and Technology (UIST)
November 2015
3DV
MLMD: Maximum Likelihood Mixture Decoupling for Fast and Accurate Point Cloud Registration
B. Eckart, K. Kim, A. Troccoli, A. Kelly, J. Kautz
International Conference on 3D Vision
October 2015, pages 241-249
HPG
An Adaptive Acceleration Structure for Screen-space Ray Tracing
S. Widmer, D. Pajak, A. Schulz, K. Pulli, J. Kautz, M. Goesele, D. Luebke
High-Performance Graphics 2015
August 2015, pages 67-76
EGSR
Filtering Environment Illumination for Interactive Physically-Based Rendering in Mixed Reality
S. Mehta, K. Kim, D. Pajak, K. Pulli, J. Kautz, R. Ramamoorthi
Eurographics Symposium on Rendering 2015
June 2015
CVPR
Modeling Object Appearance using Context-Conditioned Component Analysis
D. Turmukhambetov, N. Campbell, S. Prince, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2015, pages 4156-4164
PDF
CVPRW
Hand Gesture Recognition with 3D Convolutional Neural Networks
P. Molchanov, S. Gupta, K. Kim, J. Kautz
IEEE CVPR Workshop on Observing and Understanding Hands in Action
June 2015, winner of the VIVA hand gesture challenge
PDF
CVPRW
Locally Non-rigid Registration for Mobile HDR Photography
O. Gallo, A. Troccoli, J. Hu, K. Pulli, J. Kautz
IEEE CVPR Embedded Vision Workshop
June 2015
PDF
APP. OPTICS
Slim Near Eye Display Using Pinhole Aperture Arrays
K. Aksit, J. Kautz, D. Luebke
Applied Optics
54(11), April 2015, pages 3422-3427
CACM
Local Laplacian Filters: Edge-aware Image Processing with a Laplacian Pyramid
S. Paris, S. Hasinoff, J. Kautz
Communications of the ACM – Research Highlight
58(3), March 2015, pages 81-91
ACM MM
Speaker-Following Video Subtitles
Y. Hu, J. Kautz, Y. Yu, and W. Wang
ACM Transactions on Multimedia Computing, Communications, and Applications
11(2), January 2015, pages 32:1-32:17
CV
apr 2017

Senior Director of Visual Computing and Machine Learning Research, NVIDIA, USA

oct 2015

Director of Visual Computing Research, NVIDIA, USA

oct 2014

Senior Research Manager, NVIDIA, USA

Head of the visual computing research group at NVIDIA.

sep 2013

Senior Research Scientist, NVIDIA, USA

Research in comp. photography and computer vision.

oct 2012

Professor of Visual Computing, University College London, UK

oct 2011

Associate Professor (Reader), University College London, UK

oct 2009

Associate Professor (Senior Lecturer), University College London, UK

mar 2006

Assistant Professor (Lecturer), University College London, UK

Research in visual computing, teaching and supervision of students (BSc, MSc, PhD).

jul 2003

Post-Doctoral Researcher, Massachusetts Institute of Technology, USA

Working on appearance editing and realistic, real-time rendering.

sep 1999

PhD Student, Max-Planck-Institut fur Informatik, Germany

Received PhD (summa cum laude).

may 1998

Graduate Student, University of Waterloo, Canada

Received MMath.

oct 1993

Student, University Erlangen-Nurnberg, Germany

Received Diplom-Informatiker (MSc in Computer Science).

Contact
Jan Kautz
NVIDIA | Visual Computing Research Group
2 Technology Park Drive | Westford, MA 01886