Jan Kautz
Vice President of Learning and Perception Research @ NVIDIA

I lead the Learning and Perception Research Team at NVIDIA, working predominantly on computer vision problems — from low-level vision (denoising, super-resolution, computational photography) and geometric vision (structure from motion, SLAM, optical flow) to visual perception (detection, recognition, classification), as well as machine learning problems (deep learning, reinforcement learning, generative models).

News
aug
2023
SIGGRAPH
Honored to see our work on Precomputed Radiance Transfer make the SIGGRAPH Seminal Graphics papers list.
jun
2023
CVPR
My team has over 10 papers accepted to CVPR 2023. Come and join us in Vancouver.
may
2023
ICLR
We have 3 papers accepted at ICLR 2023. We hope to see you there!
may
2023
ICRA
My team has 4 papers at ICRA 2023. See you in London!
dec
2022
NeurIPS
My team has over 5 papers at NeurIPS 2022. Please stop by!
oct
2022
ECCV
My team has 7 papers accepted to ECCV 2022. Please stop by our posters!
mar
2022
CVPR
My team has 10 papers accepted to CVPR 2022. Hope to see you there!
dec
2021
NeurIPS
My team has 3 papers at NeurIPS 2021, with a focus on advanced generative modeling.
jun
2021
NVIDIA
NVIDIA Canvas, based on our GauGAN work, is now out as a native Windows app. It's being covered by Engadget, PetaPixel, TechCrunch and others.
mar
2021
CVPR
My team has over 10 papers (including five orals) accepted at CVPR 2021. Please “stop” by our posters and talks.
mar
2021
ICLR
We have 3 papers accepted (including a spotlight) at ICLR 2021. We hope to see you online!
dec
2020
NeurIPS
My team has 8 papers at NeurIPS 2020. We hope to see you at our presentations!
jul
2020
ECCV
My team has 9 papers (including an oral and several spotlights) at ECCV 2020. We hope to see you online!
feb
2020
CVPR
My team has 9 papers accepted (including two orals) at CVPR 2020. Please stop by our posters and talks.
Research Areas
Efficient AI
We are interested in efficient AI, for both training and inferencing, which includes methods such as pruning, neural architecture search, and so forth.
Visual Perception
Solving perception tasks is an important aspect of our work. In particular, we work on a variety of 2D and 3D perception tasks using deep learning.
Foundation Models
My team is investigating foundation models in the area of computer vision, multi-modal LLMs, as well as robotics.
Generative AI
We investigate learning-based methods that can synthesize multi-dimensional data for creative as well as scientific applications.
Publications
2024
ICLR
FasterViT: Fast Vision Transformers with Hierarchical Attention
A. Hatamizadeh, G. Heinrich, H. Yin, A. Tao, J. M. Alvarez, J. Kautz, P. Molchanov
International Conference on Learning Representations (ICLR)
May 2024
ICLR
3D Reconstruction with Generalizable Neural Fields using Scene Priors
Y. Fu, S. D. Mello, X. Li, A. Kulkarni, J. Kautz, X. Wang, S. Liu
International Conference on Learning Representations (ICLR)
May 2024
ICLR
Learning to Jointly Understand Visual and Tactile Signals
Y. Li, Y. Du, C. Liu, F. Williams, M. Foshey, B. Eckart, J. Kautz, J. B. Tenenbaum, A. Torralba, W. Matusik
International Conference on Learning Representations (ICLR)
May 2024
PDF
ICLR
A Variational Perspective on Solving Inverse Problems with Diffusion Models
M. Mardani, J. Song, J. Kautz, A. Vahdat
International Conference on Learning Representations (ICLR)
May 2024
3DV
Field-of-View Agnostic Depth Estimation for Cross-Dataset Generalization
D. Lichy, H. Su, A. Badki, J. Kautz, O. Gallo
International Conference on 3D Vision
March 2024 (oral)
3DV
PACE: Human and Camera Motion Estimation from in-the-wild Videos
M. Kocabas, Y. Yuan, P. Molchanov, Y. Guo, M. Black, O. Hilliges, J. Kautz, U. Iqbal
International Conference on 3D Vision
March 2024
2023
NeurIPS
Generalizable One-shot Neural Head Avatar
X. Li, S. D. Mello, S. Liu, K. Nagano, U. Iqbal, J. Kautz
Advances in Neural Information Processing Systems (NeurIPS)
December 2023
NeurIPS
Convolutional State Space Models for Long-Range Spatiotemporal Modeling
J. T. Smith, S. D. Mello, J. Kautz, S. Linderman, W. Byeon
Advances in Neural Information Processing Systems (NeurIPS)
December 2023
MICCAI
SMRD: SURE-based Robust MRI Reconstruction with Diffusion Models
B. Ozturkler, C. Liu, B. Eckart, M. Mardani, J. Song, J. Kautz
International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)
October 2023
ICCV
RANA: Relightable and Articulated Neural Avatars
U. Iqbal, A. Caliskan, K. Nagano, S. Khamis, P. Molchanov, J. Kautz
IEEE International Conference on Computer Vision (ICCV)
October 2023
ICCV
PhysDiff: Physics-Guided Human Motion Diffusion Model
Y. Yuan, J. Song, U. Iqbal, A. Vahdat, J. Kautz
IEEE International Conference on Computer Vision (ICCV)
October 2023 (oral)
SIGGRAPH
Single-Shot Implicit Morphable Faces with Consistent Texture Parameterization
C. Lin, K. Nagano, J. Kautz, E. Chan, U. Iqbal, L. Guibas, G. Wetzstein, S. Khamis
ACM SIGGRAPH
August 2023
ICML
Global Context Vision Transformers
A. Hatamizadeh, H. Yin, J. Kautz, P. Molchanov
International Conference on Machine Learning (ICML)
July 2023
ICML
Loss-Guided Diffusion Models for Plug-and-Play Controllable Generation
J. Song, Q. Zhang, H. Yin, M. Mardani, M.-Y. Liu, J. Kautz, Y. Chen, A. Vahdat
International Conference on Machine Learning (ICML)
July 2023
PDF
CVPR
Heterogeneous Continual Learning
D. Madaan, H. Yin, W. Byeon, J. Kautz, P. Molchanov
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
June 2023 (highlight)
PDF
CVPR
Zero-shot Pose Transfer for Unrigged Stylized 3D Characters
J. Wang, X. Li, S. Liu, S. D. Mello, O. Gallo, X. Wang, J. Kautz
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
June 2023
PDF
CVPR
The Best Defense is a Good Offense: Adversarial Augmentation Against Adversarial Attacks
I. Frosio, J. Kautz
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
June 2023
CVPR
Global Vision Transformer Pruning with Hessian-Aware Saliency
H. Yang, H. Yin, M. Shen, P. Molchanov, H. Li, J. Kautz
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
June 2023
PDF
CVPR
Recurrence without Recurrence: Stable Video Landmark Detection with Deep Equilibrium Models
P. Micaelli, P. Molchanov, A. Vahdat, H. Yin, J. Kautz
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
June 2023
CVPR
BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects
B. Wen, J. Tremblay, V. Blukis, S. Tyree, T. Müller, A. Evans, D. Fox, J. Kautz, S. Birchfield
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
June 2023
ICLR
Pseudoinverse-Guided Diffusion Models for Inverse Problems
J. Song, A. Vahdat, M. Mardani, J. Kautz
International Conference on Learning Representations (ICLR), 2023
May 2023
PDF
ICRA
Online Consistent Video Depth using Continuous Geometric Representations
C. Liu, B. Eckart, J. Kautz
IEEE International Conference on Robotics and Automation (ICRA)
May 2023
PDF
IEEE TMI
Do Gradient Inversion Attacks Make Federated Learning Unsafe?
H. Roth, A. Hatamizadeh, H. Yin, P. Molchanov, A. Myronenko, W. Li, P. Dogra, A. Feng, M. Flores, J. Kautz, D. Xu
IEEE Transactions on Medical Imaging
?(?), January 2023
2022
SIGGRAPH ASIA
Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation
Y.-Y. Yeh, K. Nagano, S. Khamis, J. Kautz, M.-Y. Liu, T.-C. Wang
ACM Transactions on Graphics (Proceedings SIGGRAPH Asia 2022)
41(6), December 2022
MIA
Towards Annotation-efficient Segmentation via Image-to-image Translation
E. Vorontsov, P. Molchanov, M. Gazda, C. Beckham, J. Kautz, S. Kadoury
Medical Image Analysis
82, November 2022
ECCV
LANA: Latency Aware Network Acceleration
P. Molchanov, J. Hall, H. Yin, N. Fusi, J. Kautz, A. Vahdat
European Conference on Computer Vision (ECCV)
October 2022
ECCV
Neural Light Field Estimation for Outdoor Scenes with Differentiable Virtual Object Insertion
Z. Wang, W. Chen, D. Acuna, J. Kautz, S. Fidler
European Conference on Computer Vision (ECCV)
October 2022
CVPR
GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras
Y. Yuan, U. Iqbal, P. Molchanov, K. Kitani, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2022 (oral)
CVPR
A-ViT: Adaptive Tokens for Efficient Vision Transformer
H. Yin, A. Vahdat, J. M. Alvarez, A. Mallya, J. Kautz, P. Molchanov
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2022 (oral)
CVPR
GradViT: Gradient Inversion of Vision Transformers
A. Hatamizadeh, H. Yin, H. Roth, W. Li, J. Kautz, D. Xu, P. Molchanov
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2022
CVPR
GroupViT: Zero-Shot Transfer to Semantic Segmentation with Text Supervision
J. Xu, S. D. Mello, S. Liu, W. Byeon, T. Breuel, J. Kautz, X. Wang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2022
CVPR
CoordGAN: Self-Supervised Dense Correspondences Emerge from GANs
J. Mu, S. Liu, S. D. Mello, Z. Yu, N. Vasconcelos, X. Wang, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2022
CVPR
FreeSOLO: Learning to Segment Objects without Annotations
X. Wang, Z. Yu, S. D. Mello, J. Kautz, A. Anandkumar, C. Shen, J. M. Alvarez
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2022
PDF
ICLR
Learning Continuous Environment Fields via Implicit Functions
X. Li, S. D. Mello, X. Wang, M.-H. Yang, J. Kautz, S. Liu
International Conference on Learning Representations (ICLR)
April 2022
PDF
IJCV
Learning Contrastive Representation for Semantic Correspondence
T. Xiao, S. Liu, S. D. Mello, Z. Yu, J. Kautz, M.-H. Yang
International Journal on Computer Vision (IJCV)
March 2022
PDF
IJCV
Displacement-Invariant Cost Computation for Stereo Matching
Y. Zhong, C. Loop, W. Byeon, S. Birchfield, Y. Dai, K. Zhang, A. Kamenev, T. Breuel, H. Li, J. Kautz
International Journal on Computer Vision (IJCV)
March 2022
PDF
AAAI
Neural Interferometry: Image Reconstruction from Astronomical Interferometers using Implicit Neural Representations
B. Wu, B. Eckart, C. Liu, J. Kautz
AAAI Conference on Artificial Intelligence (AAAI)
February 2022
PDF
2021
NeurIPS
Coupled Segmentation and Edge Learning Using Dynamic Graph Propagation
Z. Yu, R. Huang, W. Byeon, S. Liu, G. Liu, T. Breuel, A. Anandkumar, J. Kautz
Neural Information Processing Systems (NeurIPS)
December 2021
PDF
NeurIPS
A Contrastive Learning Approach for Training Variational Autoencoder Priors
J. Aneja, A. Schwing, J. Kautz, A. Vahdat
Neural Information Processing Systems (NeurIPS)
December 2021
PDF
NeurIPS
Score-based Generative Modeling in Latent Space
A. Vahdat, K. Kreis, J. Kautz
Neural Information Processing Systems (NeurIPS)
December 2021
3DV
KAMA: 3D Keypoint Aware Body Mesh Articulation
U. Iqbal, K. Xie, Y. Guo, J. Kautz, P. Molchanov
International Conference on 3D Vision (3DV)
December 2021
BMVC
Hierarchical Contrastive Motion Learning for Video Action Recognition
X. Yang, X. Yang, S. Liu, D. Sun, L. Davis, J. Kautz
British Machine Vision Conference (BMVC)
November 2021
ICCV
Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting
Z. Wang, J. Philion, S. Fidler, J. Kautz
International Conference on Computer Vision (ICCV)
October 2021 (oral)
ICCV
Self-Supervised Object Detection via Generative Image Synthesis
S. K. Mustikovela, S. De Mello, A. Prakash, U. Iqbal, S. Liu, T. Nguyen-Phuoc, C. Rother, J. Kautz
International Conference on Computer Vision (ICCV)
October 2021
TPAMI
Domain Stylization: A Fast Covariance Matching Framework towards Domain Adaptation
A. Dundar, M.-Y. Liu, Z. Yu, T.-C. Wang, J. Zedlewski, J. Kautz
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
43(7), July 2021
CVPR
Binary TTC: A Temporal Geofence for Autonomous Navigation
A. Badki, O. Gallo, J. Kautz, P. Sen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2021 (Best Student Paper Honorable Mention & oral)
CVPR
Weakly-Supervised Physically Unconstrained Gaze Estimation
R. Kothari, S. De Mello, U. Iqbal, W. Byeon, S. Park, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2021 (oral)
CVPR
Learning to Track Instances without Video Annotations
Y. Fu, S. Liu, U. Iqbal, S. De Mello, H. Shi, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2021 (oral)
CVPR
See Through Gradients: Image Batch Recovery via GradInversion
H. Yin, A. Mallya, A. Vahdat, J. M. Alvarez, J. Kautz, P. Molchanov
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2021
CVPR
Self-Supervised Learning on 3D Point Clouds by Learning Latent Generative Models
B. Eckart, W. Yuan, C. Liu, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2021
CVPR
DexYCB: A Benchmark for Capturing Hand Grasping of Objects
Y.-W. Chao, W. Yang, A. Handa, Y. Xiang, Y. Narang, K. V. Wyk, U. Iqbal, P. Molchanov, J. Tremblay, S. Birchfield, J. Kautz, D. Fox
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2021
ICLR
VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Models
Z. Xiao, K. Kreis, J. Kautz, A. Vahdat
International Conference on Learning Representations (ICLR)
May 2021 (spotlight)
PDF
ICLR
Parameter Efficient Multimodal Transformers for Video Representation Learning
S. Lee, Y. Yu, G. Kim, T. Breuel, J. Kautz, Y. Song
International Conference on Learning Representations (ICLR)
May 2021
PDF
2020
NeurIPS
NVAE: A Deep Hierarchical Variational Autoencoder
A. Vahdat, J. Kautz
Neural Information Processing Systems (NeurIPS)
December 2020 (spotlight)
PDF
NeurIPS
Online Adaptation for Consistent Mesh Reconstruction in the Wild
X. Li, S. Liu, S. De Mello, K. Kim, X. Wang, M.-H. Yang, J. Kautz
Neural Information Processing Systems (NeurIPS)
December 2020
PDF
NeurIPS
Convolutional Tensor-Train LSTM for Spatio-Temporal Learning
J. Su, W. Byeon, J. Kossaifi, F. Huang, J. Kautz, A. Anandkumar
Neural Information Processing Systems (NeurIPS)
December 2020
PDF
ISMAR
Optical Gaze Tracking with Spatially-Sparse Single-Pixel Detectors
R. Li, E. Whitmire, M. Stengel, B. Boudaoud, J. Kautz, D. Luebke, S. Patel, K. Aksit
IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
November 2020
PDF
ECCV
Contrastive Learning for Weakly Supervised Phrase Grounding
T. Gupta, A. Vahdat, G. Chechik, X. Yang, J. Kautz, D. Hoiem
European Conference on Computer Vision (ECCV)
August 2020 (spotlight)
PDF
ECCV
DeepGMR: Learning Latent Gaussian Mixture Models for Registration
W. Yuan, B. Eckart, K. Kim, V. Jampani, D. Fox, J. Kautz
European Conference on Computer Vision (ECCV)
August 2020 (spotlight)
PDF
ECCV
Joint Disentangling and Adaptation for Cross-Domain Person Re-Identification
Y. Zou, X. Yang, Z. Yu, B. V. K. V. Kumar, J. Kautz
European Conference on Computer Vision (ECCV)
August 2020 (oral)
PDF
ECCV
Self-supervised Single-view 3D Reconstruction via Semantic Consistency
X. Li, S. Liu, K. Kim, S. De Mello, V. Jampani, M.-H. Yang, J. Kautz
European Conference on Computer Vision (ECCV)
August 2020
PDF
ECCV
Weakly Supervised 3D Hand Pose Estimation via Biomechanical Constraints
A. Spurr, P. Molchanov, U. Iqbal, O. Hilliges, J. Kautz
European Conference on Computer Vision (ECCV)
August 2020
PDF
ECCV
UFO2: A Unified Framework towards Omni-supervised Object Detection
Z. Ren, Z. Yu, X. Yang, M.-Y. Liu, A. Schwing, J. Kautz
European Conference on Computer Vision (ECCV)
August 2020
PDF
ICML
Angular Visual Hardness
B. Chen, W. Liu, A. Garg, Z. Yu, A. Shrivastava, J. Kautz, A. Anandkumar
International Conference on Machine Learning (ICML)
July 2020
PDF
CVPR
Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion
H. Yin, P. Molchanov, J. M. Alvarez, Z. Li, A. Mallya, D. Hoiem, N. Jha, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2020 (oral)
CVPR
UNAS: Differentiable Architecture Search Meets Reinforcement Learning
A. Vahdat, A. Mallya, M.-Y. Liu, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2020 (oral)
CVPR
Self-Supervised Viewpoint Learning from Image Collections
S. K. Mustikovela, V. Jampani, S. De Mello, U. Iqbal, S. Liu, C. Rother, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2020
CVPR
Bi3D: Stereo Depth Estimation via Binary Classifications
A. Badki, O. Gallo, A. Troccoli, K. Kim, P. Sen, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2020
CVPR
Meshlet Priors for 3D Mesh Reconstruction
A. Badki, O. Gallo, P. Sen, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2020
CVPR
Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild
U. Iqbal, P. Molchanov, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2020
CVPR
Two-shot Spatially-varying BRDF and Shape Estimation
M. Boss, V. Jampani, K. Kim, H. Lensch, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2020
CVPR
Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Camera
J. S. Yoon, K. Kim, O. Gallo, H. S. Park, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2020
CVPR
Instance-aware, Context-focused, and Memory-efficient Weakly-Supervised Object Detection
Z. Ren, Z. Yu, X. Yang, M.-Y. Liu, Y. J. Lee, A. Schwing, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2020
IJCV
Exploiting Semantics for Face Image Deblurring
Z. Shen, W.-S. Lai, T. Xu, J. Kautz, M.-H. Yang
International Journal on Computer Vision (IJCV)
March 2020
WACV
NRMVS: Non-Rigid Multi-View Stereo
M. Innmann, K. Kim, J. Gu, M. Niessner, C. Loop, M. Stamminger, J. Kautz
IEEE Winter Conference on Applications of Computer Vision (WACV)
March 2020, pages 2754-2763
2019
NeurIPS
Joint-task Self-supervised Learning for Temporal Correspondence
X. Li, S. Liu, S. De Mello, X. Wang, M.-H. Yang, J. Kautz
Neural Information Processing Systems (NeurIPS)
December 2019
NeurIPS
Dancing to Music
H.-Y. Lee, X. Yang, M.-Y. Liu, T.-C. Wang, Y.-D. Lu, M.-H. Yang, J. Kautz
Neural Information Processing Systems (NeurIPS)
December 2019
NeurIPS
Few-shot Video-to-Video Synthesis
T.-C. Wang, M.-Y. Liu, A. Tao, G. Liu, J. Kautz, B. Catanzaro
Neural Information Processing Systems (NeurIPS)
December 2019
ICCV
Extreme View Synthesis
I. Choi, O. Gallo, A. Troccoli, M. H. Kim, J. Kautz
IEEE International Conference on Computer Vision (ICCV)
October 2019 (oral)
ICCV
SENSE: A Shared Encoder Network for Scene-flow Estimation
H. Jiang, D. Sun, V. Jampani, Z. Lv, E. Learned-Miller, J. Kautz
IEEE International Conference on Computer Vision (ICCV)
October 2019 (oral)
ICCV
Few-shot Adaptive Gaze Estimation
S. Park, S. De Mello, P. Molchanov, U. Iqbal, O. Hilliges, J. Kautz
IEEE International Conference on Computer Vision (ICCV)
October 2019 (oral)
ICCV
Learning Propagation for Arbitrarily-structured Data
S. Liu, X. Li, V. Jampani, S. De Mello, J. Kautz
IEEE International Conference on Computer Vision (ICCV)
October 2019
ICCV
Few-shot Unsupervised Image-to-Image Translation
M.-Y. Liu, X. Huang, A. Mallya, T. Karras, T. Aila, J. Lehtinen, J. Kautz
IEEE International Conference on Computer Vision (ICCV)
October 2019
ICCV
Neural Inverse Rendering of an Indoor Scene from a Single Image
S. Sengupta, J. Gu, K. Kim, G. Liu, D. Jacobs, J. Kautz
IEEE International Conference on Computer Vision (ICCV)
October 2019
PDF
ICCV
Unsupervised Video Interpolation Using Cycle Consistency
F. Reda, D. Sun, A. Dundar, M. Shoeybi, G. Liu, K. Shih, A. Tao, J. Kautz, B. Catanzaro
IEEE International Conference on Computer Vision (ICCV)
October 2019
BMVC
Few-Shot Viewpoint Estimation
H.-Y. Tseng, S. De Mello, J. Tremblay, S. Liu, S. Birchfield, M.-H. Yang, J. Kautz
British Machine Vision Conference (BMVC)
September 2019
PDF
BMVC
Video Stitching for Linear Camera Arrays
W.-S. Lai, O. Gallo, J. Gu, D. Sun, M.-H. Yang, J. Kautz
British Machine Vision Conference (BMVC)
September 2019
CVPR
Joint Discriminative and Generative Learning for Person Re-identification
Z. Zheng, X. Yang, Z. Yu, L. Zheng, Y. Yang, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2019 (oral)
CVPR
STEP: Spatio-Temporal Progressive Learning for Video Action Detection
X. Yang, X. Yang, M.-Y. Liu, F. Xiao, L. Davis, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2019 (oral)
CVPR
PlaneRCNN: 3D Plane Detection and Reconstruction from a Single View
C. Liu, K. Kim, J. Gu, Y. Furukawa, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2019 (oral)
CVPR
Neural RGB → D Sensing: Depth and Uncertainty from a Video Camera
C. Liu, J. Gu, K. Kim, S. Narasimhan, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2019 (Best Paper Finalist & oral)
CVPR
SCOPS: Self-Supervised Co-Part Segmentation
W.-C. Hung, V. Jampani, S. Liu, P. Molchanov, M.-H. Yang, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2019
CVPR
Pixel Adaptive Convolutional Neural Networks
H. Su, V. Jampani, D. Sun, O. Gallo, E. Learned-Miller, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2019
CVPR
Learning Linear Transformations for Fast Image and Video Style Transfer
X. Li, S. Liu, J. Kautz, M.-H. Yang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2019
CVPR
Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments
X. Li, S. Liu, K. Kim, M.-H. Yang, X. Wang, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2019
CVPR
Importance Estimation for Neural Network Pruning
P. Molchanov, A. Mallya, S. Tyree, I. Frosio, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2019
TPAMI
Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation
D. Sun, X. Yang, M.-Y. Liu, J. Kautz
IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
?(?), ? 2019
TIP
Statistical Nearest Neighbors for Image Denoising
I. Frosio, J. Kautz
IEEE Transactions on Image Processing
28(2), February 2019, pages 723-728
WACV
A Fusion Approach for Multi-Frame Optical Flow Estimation
Z. Ren, O. Gallo, D. Sun, M.-H. Yang, E. Sudderth, J. Kautz
IEEE Winter Conference on Applications of Computer Vision (WACV)
January 2019
PDF
2018
NeurIPS
Context-aware Synthesis and Placement of Object Instances
D. Lee, S. Liu, J. Gu, M. Liu, M.-H. Yang, J. Kautz
Neural Information Processing Systems (NeurIPS)
December 2018
NeurIPS
Video-to-Video Synthesis
T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, G. Liu, A. Tao, J. Kautz, B. Catanzaro
Neural Information Processing Systems (NeurIPS)
December 2018
ECCV
Separating Reflection and Transmission Images in the Wild
P. Wieschollek, O. Gallo, J. Gu, J. Kautz
European Conference on Computer Vision (ECCV)
September 2018
PDF
ECCV
Hand Pose Estimation via Latent 2.5D Heatmap Regression
U. Iqbal, P. Molchanov, T. Breuel, J. Gall, J. Kautz
European Conference on Computer Vision (ECCV)
September 2018
PDF
ECCV
Multimodal Unsupervised Image-to-image Translation
X. Huang, M.-Y. Liu, S. Belongie, J. Kautz
European Conference on Computer Vision (ECCV)
September 2018
ECCV
Switchable Temporal Propagation Network
S. Liu, J. Gu, S. De Mello, V. Jampani, G. Zhong, M.-H. Yang, J. Kautz
European Conference on Computer Vision (ECCV)
September 2018
PDF
ECCV
Tackling 3D ToF Artifacts Through Learning and the FLAT Dataset
Q. Guo, I. Frosio, O. Gallo, T. Zickler, J. Kautz
European Conference on Computer Vision (ECCV)
September 2018
PDF
ECCV
SEAL: A Framework Towards Simultaneous Edge Alignment and Learning
Z. Yu, W. Liu, Y. Zou, C. Feng, S. Ramalingam, B. V. K. V. Kumar, J. Kautz
European Conference on Computer Vision (ECCV)
September 2018
ECCV
Superpixel Sampling Networks
V. Jampani, D. Sun, M.-Y. Liu, M.-H. Yang, J. Kautz
European Conference on Computer Vision (ECCV)
September 2018
ECCV
A Closed-form Solution to Photorealistic Image Stylization
Y. Li, M.-Y. Liu, X. Li, M.-H. Yang, J. Kautz
European Conference on Computer Vision (ECCV)
September 2018
ECCV
Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimation
Z. Lv, K. Kim, A. Troccoli, D. Sun, J. Rehg, J. Kautz
European Conference on Computer Vision (ECCV)
September 2018
ECCV
HGMR: Hierarchical Gaussian Mixtures for Adaptive 3D Registration
B. Eckart, K. Kim, J. Kautz
European Conference on Computer Vision (ECCV)
September 2018
PDF
BMVC
EOE: Expected Overlap Estimation Over Unstructured Point Cloud Data
B. Eckart, K. Kim, J. Kautz
International Conference on 3D Vision (3DV)
September 2018
PDF
BMVC
Budget-Aware Activity Detection with A Recurrent Policy Network
B. Mahesseni, X. Yang, P. Molchanov, J. Kautz
British Machine Vision Conference (BMVC)
September 2018
CVPR
PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume
D. Sun, X. Yang, M.-Y. Liu, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2018 (oral)
CVPR
SPLATNet: Sparse Lattice Networks for Point Cloud Processing
H. Su, V. Jampani, D. Sun, S. Maji, E. Kalogerakis, M.-H. Yang, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2018 (Best Paper Honorable Mention Award & oral)
CVPR
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
T.-C. Wang, M.-Y. Liu, J.-Y. Zhu, A. Tao, J. Kautz, B. Catanzaro
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2018 (oral)
CVPR
Geometry-Aware Learning of Maps for Camera Localization
S. Brahmbhatt, J. Gu, K. Kim, J. Hays, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2018 (spotlight)
CVPR
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation
H. Jiang, D. Sun, V. Jampani, M.-H. Yang, E. Learned-Miller, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2018 (spotlight)
CVPR
Depth-Based 3D Hand Pose Estimation: From Current Achievements to Future Goals
S. Yuan, G. Garcia-Hernando, B. Stenger, G. Moon, J. Y. Chang, K. M. Lee, P. Molchanov, J. Kautz, S. Honari, L. Ge, J. Yuan, X. Chen, G. Wang, F. Yang, K. Akiyama, Y. Wu, Q. Wan, M. Madadi, S. Escalera, S. Li, D. Lee, I. Oikonomidis, A. Argyros, T.-K. Kim
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2018 (spotlight)
PDF
CVPR
Learning Superpixels with Segmentation-Aware Affinity Loss
W.-C. Tu, M.-Y. Liu, V. Jampani, D. Sun, S.-Y. Chien, M.-H. Yang, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2018
CVPR
Deep Semantic Face Deblurring
Z. Shen, W.-S. Lai, T. Xu, J. Kautz, M.-H. Yang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2018
CVPR
MoCoGAN: Decomposing Motion and Content for Video Generation
S. Tulyakov, M.-Y. Liu, X. Yang, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2018
PDF
CVPR
Improving Landmark Localization with Semi-Supervised Learning
S. Honari, P. Molchanov, S. Tyree, C. Pal, P. Vincent, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2018
CVPR
Making Convolutional Networks Recurrent for Visual Sequence Learning
X. Yang, P. Molchanov, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2018
ICCP
ReBlur2Deblur: Deblurring Videos via Self-Supervised Learning
H. Chen, J. Gu, O. Gallo, M.-Y. Liu, A. Veeraraghavan, J. Kautz
International Conference on Computational Photography (ICCP)
May 2018
ICRA
Synthetically Trained Networks for Learning Human-Readable Plans from Real-World Demonstrations
J. Tremblay, T. To, A. Molchanov, S. Tyree, J. Kautz, S. Birchfield
International Conference on Robotics and Automation (ICRA)
May 2018
PDF
AAAI
Learning Binary Residual Representations for Domain-specific Video Streaming
Y.-H. Tsai, M.-Y. Liu, D. Sun, M.-H. Yang, J. Kautz
AAAI Conference on Artificial Intelligence (AAAI)
February 2018 (spotlight)
EI
Learning Adaptive Parameter Tuning for Image Processing
J. Dong, I. Frosio, J. Kautz
Electronic Imaging, Image Processing: Algorithms and Systems XVI
January 2018
PDF
2017
NIPS
Unsupervised Image-to-Image Translation
M.-Y. Liu, T. Breuel, J. Kautz
Neural Information Processing Systems (NIPS)
December 2017 (spotlight)
NIPS
Learning Affinity via Spatial Propagation Networks
S. Liu, S. De Mello, J. Gu, G. Zhong, M.-S. Yang, J. Kautz
Neural Information Processing Systems (NIPS)
December 2017
ICCV
Intrinsic3D: High-Quality 3D Reconstruction by Joint Appearance and Geometry Optimization with Spatially-Varying Lighting
R. Maier, K. Kim, D. Cremers, J. Kautz, M. Niessner
IEEE International Conference on Computer Vision (ICCV)
October 2017, pages 3133-3141
ICCV
A Lightweight Approach for On-The-Fly Reflectance Estimation
K. Kim, J. Gu, S. Tyree, P. Molchanov, M. Niessner, J. Kautz
IEEE International Conference on Computer Vision (ICCV)
October 2017, pages 20-28 (oral)
3DV
Cascaded Scene Flow Prediction using Semantic Segmentation
Z. Ren, D. Sun, J. Kautz, E. Sudderth
International Conference on 3D Vision
October 2017 (oral)
3DV
Multiframe Scene Flow with Piecewise Rigid Motion
V. Golyanik, K. Kim, R. Maier, M. Niessner, D. Stricker, J. Kautz
International Conference on 3D Vision
October 2017 (spotlight)
SIGGRAPH
Computational Zoom: A Framework for Post-Capture Image Composition
A. Badki, O. Gallo, J. Kautz, P. Sen
ACM Transactions on Graphics (Proceedings SIGGRAPH 2017)
36(4), July 2017, pages 46:1-46:14
SIGGRAPH
Mixed-primary Factorization for Dual-frame Computational Displays
F.-C. Huang, D. Pajak, J. Kim, J. Kautz, D. Luebke
ACM Transactions on Graphics (Proceedings SIGGRAPH 2017)
36(4), July 2017, pages 149:1-149:13
PDF
CVPRW
Reconstructing Intensity Images from Binary Spatial Gradient Cameras
S. Jayasuriya, O. Gallo, J. Gu, T. Aila, J. Kautz
IEEE CVPR Embedded Vision Workshop
July 2017
PDF
CVPR
Dynamic Facial Analysis: From Bayesian Filtering to Recurrent Neural Network
J. Gu, S. De Mello, X. Yang, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
July 2017, pages 1531-1540
CVPR
Polarimetric Multi-View Stereo
Z. Cui, J. Gu, B. Shi, P. Tan, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
July 2017, pages 369-378
ICLR
GA3C: GPU-based A3C for Deep Reinforcement Learning
M. Babaeizadeh, I. Frosio, S. Tyree, J. Clemons, J. Kautz
International Conference on Learning Representations
April 2017
ICLR
Pruning Convolutional Neural Networks for Resource Efficient Transfer Learning
P. Molchanov, S. Tyree, T. Karras, T. Aila, J. Kautz
International Conference on Learning Representations
April 2017
PDF
IEEE TCI
Loss Functions for Neural Networks for Image Processing
H. Zhao, O. Gallo, I. Frosio, J. Kautz
IEEE Transactions on Computational Imaging
3(1), March 2017, pages 47-57
2016
SIGGRAPH ASIA
Computational Bounce Flash for Indoor Portraits
L. Murmann, A. Davis, J. Kautz, F. Durand
ACM Transactions on Graphics (Proceedings SIGGRAPH Asia 2016)
35(6), December 2016, pages 190:1-190:9
NIPSW
GA3C: GPU-based A3C for Deep Reinforcement Learning
M. Babaeizadeh, I. Frosio, S. Tyree, J. Clemons, J. Kautz
NIPS Workshop on Efficient Methods for Deep Neural Networks
December 2016
NIPSW
Pruning Convolutional Neural Networks for Resource Efficient Transfer Learning
P. Molchanov, S. Tyree, T. Karras, T. Aila, J. Kautz
NIPS Workshop on Efficient Methods for Deep Neural Networks
December 2016
PDF
ACM MM
Multilayer and Multimodal Fusion of Deep Neural Networks for Video Classification
X. Yang, P. Molchanov, J. Kautz
ACM Multimedia
October 2016, pages 978-987 (oral)
PDF
CVPR
Accelerated Generative Models for 3D Point Cloud Data
B. Eckart, K. Kim, A. Troccoli, A. Kelly, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2016, pages 5497-5505 (spotlight oral)
CVPR
Online Detection and Classification of Dynamic Hand Gestures with Recurrent 3D Convolutional Neural Networks
P. Molchanov, X. Yang, S. Gupta, K. Kim, S. Tyree, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2016, pages 4207-4215
IEEE IVS
Towards Selecting Robust Hand Gestures for Automotive Interfaces
S. Gupta, P. Molchanov, X. Yang, K. Kim, S. Tyree, J. Kautz
IEEE Intelligent Vehicles Symposium 2016
June 2016
PDF
2015
ICCV
Robust Model-based 3D Head Pose Estimation
G. P. Meyer, S. Gupta, I. Frosio, D. Reddy, J. Kautz
IEEE International Conference on Computer Vision (ICCV)
December 2015, pages 3649-3657
CGF
Interactive Sketch-Driven Image Synthesis
D. Turmukhambetov, N. Campbell, D. Goldman, J. Kautz
Computer Graphics Forum
34(8), December 2015, pages 130-142
UIST
Joint 5D Pen Input for Light Field Displays
J. Tompkin, S. Muff, J. McCann, H. Pfister, J. Kautz, M. Alexa, W. Matusik
ACM User Interface Software and Technology (UIST)
November 2015
3DV
MLMD: Maximum Likelihood Mixture Decoupling for Fast and Accurate Point Cloud Registration
B. Eckart, K. Kim, A. Troccoli, A. Kelly, J. Kautz
International Conference on 3D Vision
October 2015, pages 241-249
HPG
An Adaptive Acceleration Structure for Screen-space Ray Tracing
S. Widmer, D. Pajak, A. Schulz, K. Pulli, J. Kautz, M. Goesele, D. Luebke
High-Performance Graphics 2015
August 2015, pages 67-76
EGSR
Filtering Environment Illumination for Interactive Physically-Based Rendering in Mixed Reality
S. Mehta, K. Kim, D. Pajak, K. Pulli, J. Kautz, R. Ramamoorthi
Eurographics Symposium on Rendering 2015
June 2015
CVPR
Modeling Object Appearance using Context-Conditioned Component Analysis
D. Turmukhambetov, N. Campbell, S. Prince, J. Kautz
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
June 2015, pages 4156-4164
PDF
CVPRW
Hand Gesture Recognition with 3D Convolutional Neural Networks
P. Molchanov, S. Gupta, K. Kim, J. Kautz
IEEE CVPR Workshop on Observing and Understanding Hands in Action
June 2015, winner of the VIVA hand gesture challenge
PDF
CVPRW
Locally Non-rigid Registration for Mobile HDR Photography
O. Gallo, A. Troccoli, J. Hu, K. Pulli, J. Kautz
IEEE CVPR Embedded Vision Workshop
June 2015
PDF
APP. OPTICS
Slim Near Eye Display Using Pinhole Aperture Arrays
K. Aksit, J. Kautz, D. Luebke
Applied Optics
54(11), April 2015, pages 3422-3427
CACM
Local Laplacian Filters: Edge-aware Image Processing with a Laplacian Pyramid
S. Paris, S. Hasinoff, J. Kautz
Communications of the ACM – Research Highlight
58(3), March 2015, pages 81-91
ACM MM
Speaker-Following Video Subtitles
Y. Hu, J. Kautz, Y. Yu, and W. Wang
ACM Transactions on Multimedia Computing, Communications, and Applications
11(2), January 2015, pages 32:1-32:17
CV
oct 2018

Vice President of Learning and Perception Research, NVIDIA, USA

Head of the learning and perception research group at NVIDIA.

apr 2017

Senior Director of Visual Computing and Machine Learning Research, NVIDIA, USA

oct 2015

Director of Visual Computing Research, NVIDIA, USA

oct 2014

Senior Research Manager, NVIDIA, USA

Head of the visual computing research group at NVIDIA.

sep 2013

Senior Research Scientist, NVIDIA, USA

Research in comp. photography and computer vision.

oct 2012

Professor of Visual Computing, University College London, UK

oct 2011

Associate Professor (Reader), University College London, UK

oct 2009

Associate Professor (Senior Lecturer), University College London, UK

mar 2006

Assistant Professor (Lecturer), University College London, UK

Research in visual computing, teaching and supervision of students (BSc, MSc, PhD).

jul 2003

Post-Doctoral Researcher, Massachusetts Institute of Technology, USA

Working on appearance editing and realistic, real-time rendering.

sep 1999

PhD Student, Max-Planck-Institut fur Informatik, Germany

Received PhD (summa cum laude).

may 1998

Graduate Student, University of Waterloo, Canada

Received MMath.

oct 1993

Student, University Erlangen-Nurnberg, Germany

Received Diplom-Informatiker (MSc in Computer Science).

Contact
Jan Kautz
NVIDIA | Learning and Perception Research Group
2 Technology Park Drive | Westford, MA 01886