Me Papers CV Research Contact Useful Link

Hsiao-Yu Fish Tung

Machine Learning Department

Carnegie Mellon University

mail: htung at

About me


I am a third-year PhD student in the Machine Learning Department at CMU. My advisor is Professor Katerina Fragkiadaki. My life-long goal is to teach machine to see, learn and think like humans. We develope methods (most of them are deep neural networks) for machine to learn about the 3D world, understand human language and carry out human activities from narrated videos.

I received my M.S. in CMU MLD and my B.S. in Electrical Engineering from National Taiwan University in 2013. During my master, I worked with Professor Alex Smola on spectral method for Bayesian models and succefully designed efficient and provable algorithms. In my undergraduate, I was one of the member in Professor Chih-Jen Lin 's team "Algorithm @ National Taiwan University" and won the KDD CUP 2013 Championship on both Track1 and Track2.

Conference Papers

Reward Learning from Narrated Demonstrations [Paper]

Hsiao-Yu Fish Tung, Adam W. Harley, Liang-Kang Huang, Katerina Fragkiadaki
Conference on Computer Vision and Pattern Recognition, CVPR 2018.

Self-supervised learning of motion capture [Paper] [Code] [Website]

Hsiao-Yu Fish Tung, Hsiao-Wei Tung, Ersin Yumer, Katerina Fragkiadaki
Neural Information Processing Systems, NIPS 2017, spotlight.

Adversarial Inverse Graphics Networks: Learning 2D-to-3D Lifting and Image-to-Image Translation from Unpaired Supervision [Paper]

Hsiao-Yu Fish Tung, Adam W. Harley, William Seto, Katerina Fragkiadaki
International Conference on Computer Vision, ICCV 2017.

Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy [Paper]

Dougal J. Sutherland, Hsiao-Yu Tung, Heiko Strathmann, Soumyajit De, Aaditya Ramdas, Alexander J. Smola, Arthur Gretton
International Conference on Learning Representations, ICLR 2017.

Fast and Guaranteed Tensor Decomposition via Sketching [Paper]

Yining Wang, Hsiao-Yu Tung, Alexander J. Smola, Animashree Anandkumar
Neural Information Processing Systems, NIPS 2015, spotlight.

Spectral Methods for Indian Buffet Process Inference. [Paper]

Hsiao-Yu Fish Tung, Alexander J. Smola
Neural Information Processing Systems, NIPS 2014.

Novel Traffic Light Timing Adjustment Strategy Based On Genetic Algorithm. [Paper]

Hsiao-Yu Tung, Wei-Chiu Ma, and Tian-Li Yu
IEEE Congress on Evolutionary Computation, IEEE CEC 2013, Oral.

Journal Paper

Spectral Methods for Nonparametric Models [Paper]

Hsiao-Yu Fish Tung, Chao-Yuan Wu, Manzil Zaheer, Alexander J. Smola
Master Thesis, CMU MLD

Combination of Feature Engineering and Ranking Models for Paper-Author Identification in KDD Cup 2013 [Paper]

C.-L. Li, Y.-C. Su, T.-W. Lin, C.-H. Tsai, W.-C. Chang, K.-H. Huang, T.-M. Kuo, S.-W. Lin, Y.-S. Lin, Y.-C. Lu, C.-P. Yang, C.-X. Chang, W.-S. Chin, Y.-C. Juan, H.-Y. Tung, J.-P. Wang, C.-K. Wei, Felix Wu, T.-C. Yin, T. Yu, Y. Zhuang, S.-d. Lin, H.-T. Lin, C.-J. Lin
Journal of Machine Learning Research (JMLR), 2015

Effective String Processing and Matching for Author Disambiguation. [Paper]

W.-S. Chin, Y.-C. Juan, Y.-Zhuang, Felix Wu, H.-Y. Tung, T. Yu, J.-P. Wang, C.-X. Chang, C.-P. Yang, W.-C. Chang, K.-H. Huang, T.-M. Kuo, S.-W. Lin, Y.-S. Lin, Y.-\ C. Lu, Y.-C. Su, C.-K. Wei, T.- C. Yin, C.-L. Li, T.-W. Lin, C.-H. Tsai, S.-d. Lin, H.-T. Lin, C.-J. Lin
Journal of Machine Learning Research (JMLR), 2014

Resume(Short version)[ Last Update: 2017/10 ]


Carnegie Mellon University


PhD Student
Machine Learning Department, School of Computer Science

Carnegie Mellon University


Master Student
Machine Learning Department, School of Computer Science

National Taiwan University


B.S. in Electrical Engineering

Work Experience

OpenAI Inc.

2018 Summer (Expected)

Research Intern

Adobe Research

2017 Summer

Research Intern with Ersin Yumer

Google Brain, Google Inc.

2016 Summer

Software Enginnering Intern with Andrew Dai

Intel Parallel Computing Lab

2015 Summer

Research Intern with Sheng Li


2012 Summer

Summer Intern, Home Entertainment group

Honor and Awards

2018 Qualcomm QinF fellowship Finalist (Attending Final Round in mid-April !)

2018 Open Philanthropy Project AI Fellows Finalists (~10% selected for interview)

2018 Facebook Fellowship Finalists

2014, 2017 NIPS Travel Award

Member of Eta Kappa Nu

An academic honor society

2013 KDD Cup Award, Track1&2 Champion

The leading Data Mining competition in the world, organized by ACM.
Competition topic on Author-Paper recognition for data from Microsoft Academic Search

2 times NTU President Award

Awarded to top 5% of students in each department of National Taiwan University.
Rank 1/245 in 2012 Spring

NTU 2013 Student Outstanding Performance Scholarship

2012 Altera Innovate Asia FPGA Design Competition, Outstanding Achievement

Most prestigious FPGA design competition in Asia.
Project on FPGA-based gesture recognition 3-D control panel

2nd Place in NTU Green Grass Award (without a first prize winner)

NTU Art Festival Installation Art Contest

* I am from Taiwan. My Chinese name is My Chinese Name (TUNG, HSIAO-YU). Hsiao-Yu shares the same pronouciation as little fish in Chinese, so I picked Fish as my English name.

* I enjoy all kinds of music in English, Chinese, Taiwanese, Korean, Japanese. I also listen to classic music, light music, Hip-Hop and Jazz.

* I enjoy art works, especially art in the 20th century.

Contact Information


GHC-8215, Gates Hillman Center,
Machine Learning Department,
Carnegie Mellon University,
5000 Forbes Ave,
Pittsburgh, PA 15213


htung at
sfish0101 at