Announcement

Teaching

Research

My primary research interests are in computer vision. I am also very interested in more general questions in AI and machine learning. My current research topics include:

• First Person Vision: A first person camera placed at the person's head captures candid moments in our life, providing detailed visual data of how we interact with people, objects, and scenes. It reveals our future intention and momentary visual sensorimotor behaviors. With the first person vision, can we build a computational model for personalized intelligence that predicts what we see and act by "putting yourself in her/his shoes"?

         NSF Project on First Person Vision: Constructing Social Interactions in 3D Space from First Person Vision

• Human Recognition: We are developing computer algorithms to recognize human at multiple levels of abstractions: from the basic body limb tracking, to human identification, to gesture recognition, to activity inference. The ultimate goal is to develop computation algorithms to understand human behavior in video.

• Image Segmentation and Object Recognition: Our research is motivated by two sets of questions,1) how to extract “interesting” patterns from data, and 2) how to guide the grouping process to achieve specific vision tasks, such as recognizing familiar object shapes. In this direction, we have been pursuing a line of research building upon spectral graph theory.
  

• Past projects: HumanID , Telegraffiti.

Recent Projects and Publications

Object Detection in Video with Spatiotemporal Sampling Networks Gedas Bertasius, Lorenzo Torresani and Jianbo Shi European Conference on Computer Vision(ECCV), 2018 Egocentric Basketball Motion Planning from a Single Image Gedas Bertasius, Aaron Chan, and Jianbo Shi Computer Vision and Pattern Recognition (CVPR), 2018 Unsupervised Learning of Important Objects from First-Person Videos Gedas Bertasius, Hyun Soo Park, Stella X. Yu, and Jianbo Shi International Conference on Computer Vision (ICCV), 2017 Am I a Baller? Basketball Performance Assessment from First-Person Videos??? Gedas Bertasius, and Jianbo Shi International Conference on Computer Vision (ICCV), 2017 First Person Action-Object Detection with EgoNet Gedas Bertasius, Hyun Soo Park, Stella X. Yu, and Jianbo Shi Robotics: Science and Systems (RSS), 2017 Convolutional RandomWalk Networks for Semantic Image Segmentation Gedas Bertasius, Lorenzo Torresani, Stella X. Yu and Jianbo Shi Conference on Computer Vision and Pattern Recognition (CVPR), 2017 Predicting Behaviors of Basketball Players from First Person Videos S. Su, J. P. Hong, J. Shi, and H. S. Park Conference on Computer Vision and Pattern Recognition (CVPR), 2017 Customizing First Person Image Through Desired Actions S. Su, J. Shi, and H. S. Park ARXIV Force from Motion: Decoding Physical Sensation from a First Person Video (webpage) Hyun Soo Park, Jyh-Jing Hwang, and Jianbo Shi Computer Vision and Pattern Recognition (CVPR), 2016 Egocentric Future Localization (webpage) Hyun Soo Park, Jyh-Jing Hwang, Yedong Niu, and Jianbo Shi Computer Vision and Pattern Recognition (CVPR), 2016 Semantic Segmentation with Boundary Neural Fields Gedas Bertasius, and Jianbo Shi, and Lorenzo Torresani Computer Vision and Pattern Recognition (CVPR), 2016 High-for-Low, Low-for-High: Efficient Boundary Detection from Deep Object Features and its Applications to High-Level Vision, Gedas Bertasius, and Jianbo Shi, and Lorenzo Torresani International Conference on Computer Vision (ICCV), 2015 Social Saliency Prediction (webpage) Hyun Soo Park, and Jianbo Shi Computer Vision and Pattern Recognition (CVPR), 2015 DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection Gedas Bertasius, Jianbo Shi and Lorenzo Torresani Computer Vision and Pattern Recognition (CVPR), 2015 Pulling Things out of Perspective Lubor Ladicky, Jianbo Shi, and Marc Pollefeys Computer Vision and Pattern Recognition (CVPR), 2014 Nested Shape Descriptors Jeffrey Byrne, Jianbo Shi International Conference on Computer Vision, ICCV 2013 Pose from Flow and Flow from Pose Katerina Fragkiadaki, Weiyu Zhang, Geng Zhang, and Jianbo Shi Computer Vision and Pattern Recognition (CVPR), 2013 Two Granularity Tracking: Mediating Trajectory and Detection Graphs for Tracking under Occlusions Poster Katerina Fragkiadaki, Weiyu Zhang, Geng Zhang, and Jianbo Shi European Conference on Computer Vision (ECCV), 2012 Video Segmentation by tracing Discontinuities in a Trajectory Embedding Poster Katerina Fragkiadaki, Geng Zhang, Jianbo Shi IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012 Structural-Flow Trajectories for Unravelling Tubular Structure Bundles Poster Katerina Fragkiadaki, Weiyu Zhang, Jianbo Shi, and Elena Bernardis. International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Oct.2012 Multi-hypothesis Motion Planning for Visual Object Tracking Talk slides, Poster Haifeng Gong, Jack Sim, Maxim Likhachev, Jianbo Shi International Conference on Computer Vision (ICCV), 2011 Discriminative Image Warping with Attribute Flow Talk slides, Poster Weiyu Zhang, Praveen Srinivasan, Jianbo Shi IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011 Detection Free Tracking: Exploiting Motion and Topology for Segmenting and Tracking under Entanglement Poster Katerina Fragkiadaki, and Jianbo Shi IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011 Contour cut: identifying salient contour in images by solving a Hermitian eigenvealue problem Poster Ryan Kennedy, Jean Gallier, and Jianbo Shi IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2011 Many-to-one Contour Matching for Describing and Discriminating Object Shape , Talk slides Praveen Srinivasan, Qihui Zhu, and Jianbo Shi IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010 Understanding Videos, Contructing Plots: Learning a Visually Grounded Storyline Model from Annotated Video , Abhinav Gupta, Praveen Srinivasan, Jianbo Shi, and Larry Davis, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2009 A "Shape Aware" Model for semi-supervised Learning for Objects and its Context, Abhinav Gupta, Jianbo Shi, and Larry Davis, Advances in Neural Information Processing Systems (NIPS), 2008 "Grouping Contours Via a Related Image, Praveen Srinivasan, Liming Wang, Jianbo Shi, Advances in Neural Information Processing Systems (NIPS), 2008 Contour Context Selection for Object Detection: A Set-to-Set Contour Matching Approach, Qihui Zhu, Liming Wang, Yang Wu, Jianbo Shi, The 10th European Conference on Computer Vision (ECCV), 2008, Marseille, France Saliency Based Opportunistic Search for Object Part Extraction and Labeling, Yang Wu, Qihui Zhu, Jianbo Shi, Nanning Zheng The 10th European Conference on Computer Vision (ECCV), 2008, Marseille, France Object Detection Combining Recognition and Segmentation, Liming Wang, Jianbo Shi, Gang Song, I-Fan Shen, Eighth Asian Conference on Computer Vision (ACCV), 2007 [Source code and Project Page: download ] [Database: Penn-Fudan Pedstrian Detection and Segmentation] Untangling Cycles for Contour Grouping, Qihui Zhu, Gang Song and Jianbo Shi, IEEE International Conference on Computer Vision (ICCV), 2007 [ Tech. Report: extended discussion on directed random walk.] [ Talk slides ] Bottom-up Recognition and Parsing of the Human Body, Praveen Srinivasan and Jianbo Shi, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2007 an extension with image feature learning appeared in The Sixth International Conference on Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR), 2007 [Talk Slides] [ Project Web Page] Image Matching via Saliency Region Correspondences, Alexander Toshev, Jianbo Shi, and Kostas Daniilidis, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2007 [Talk Slides] Recognizing Objects by Piecing Together the Segmentation Puzzle, Timothee Cour and Jianbo Shi, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2007 [Poster] Solving Markov Random Fields with Spectral Relaxation, Timothee Cour and Jianbo Shi, IEEE International Conference on Artificial Intelligence and Statistics (AISTAT), 2007 [Poster] Balanced Graph Matching, Timothee Cour, Praveen Srinivasan, and Jianbo Shi, Neural Information Processing Systems (NIPS), 2006 [supplement: Affinely Constrained Rayleigh Quotients] [Poster] Shape from Shading: Recognizing mountains through a Global View, Qihui Zhu and Jianbo Shi, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) June 2006 [Talk Slide, Poster] Comparing Classification Performance of Feature Ensemble: Detecting Prostate Cancer from High Resolution MRI, Anant Madabhushi, Jianbo Shi, Mark Rosen, John Tomaszeweski, Michael Feldman, Computer Vision Approaches to Medical Image Analysis Workshop, ECCV 2006 Graph Embedding for Improving Supervised Classification & Novel Class Detection: Prostate Cancer, Anant Madabhushi, Jianbo Shi, M. Rosen, M. Feldman and J. Tomaszweski, International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), Oct.2005 Spectral Segmentation with Multi-Scale Graph Decomposition , Timothee Cour, Florence Benezit, Jianbo Shi, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2005. [Talk Slide, Poster] Learning Spectral Graph Segmentation, Timothee Cour, Jianbo Shi, Nicolas Gogin, IEEE International Conference on Artificial Intelligence and Statistics (AISTAT) 2005. [Talk Slides] Seeing Through Water, A. Efros, V. Isler, J. Shi, M. Visontai, Neural Information Processing Systems (NIPS), 2004 Detecting Unusual Activity in Video , H. Zhong, J. Shi and M. Visontai, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2004 [Web site] Segmentation Given Partial Grouping Constraints, Stella X. Yu and Jianbo Shi, IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(2):173-83, 2004

Selected Publications

Tutorials:

• cvpr 2004: Graph Based Image Segmentation

Former Ph.D. Students:

• Stella Yu, Computational Models of Perceptual Organization, Ph.D. Thesis, Carnegie Mellon University, 2003. Clare Boothe Luce Assistant Professor, Boston College.
• Abhinav Gupta, Beyond Nouns and Verbs, 2009, U. Maryland, Co-advisor with Larry Davis
  
• Qihui Zhu, Shape Detection by Packing Contours and Regions, 2010
  WPE-II Paper on Primal Dual Packing, and WPE-II Talk
  
• Praveen Srinivasan, Holistic Shape-Based Object Recognition Using Bottom-up Image Structures, 2011
• Alexander Toshev, Shape Representation for Object Recognition, 2011, Co-advisee with K. Daniilidis, B. Taskar
• Elena Bernardis, Finding Dots in Microscopic Images, 2011, Co-advisee with S. Yu
• Gang Song, 2013, Quantitative Analysis of Thoracic Computed Tomography Images, Co-advisee with Jim Gee
• Katerina Fragiadaki, Multi-granularity Representations for Human Interactions: Pose, Motion and Intention, 2013
• Jeff Byrne, STR, Shape Representations using Nested Descriptors, April 2014
• Liming Wang, (Fudan University), 2009
• Yang Wu, (Xi'an Jiaotong University), Towards Scene Understanding: Deep and Layered Recognition and Heuristic Parsing of Objects, 2010