Publications

2025

1. Evaluation

   RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents

   Jingyi Yang, Shuai Shao, Dongrui Liu, and Jing Shao

   arXiv, May 2025

   arXiv Code
2. Evaluation

   IS-Bench: Evaluating Interactive Safety of VLM-Driven Embodied Agents in Daily Household Tasks

   Xiaoya Lu, Zeren Chen, Xuhao Hu, Yijin Zhou, Weichen Zhang, and Dongrui Liu

   arXiv, May 2025

   arXiv Code
3. Risk Mitigation

   X-Boundary: Establishing Exact Safety Boundary to Shield LLMs from Multi-Turn Jailbreaks without Compromising Usability

   Xiaoya Lu, Dongrui Liu, Yi Yu, Luxin Xu, and Jing Shao

   EMNLP 2025 Findings, May 2025

   arXiv Code
4. Attack

   Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injection

   Ziqi Miao, Yi Ding, Lijun Li, and Jing Shao

   EMNLP 2025 Main Conference, May 2025

   arXiv Code

2024

1. Risk Mitigation

   SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model

   Yongting Zhang, Lu Chen, Guodong Zheng, Yifeng Gao, Rui Zheng, Jinlan Fu, Zhenfei Yin, Senjie Jin, Yu Qiao, Xuanjing Huang, Feng Zhao, Tao Gui, and Jing Shao

   Arxiv, May 2024

   Website
2. Evaluation

   Assessment of Multimodal Large Language Models in Alignment with Human Values

   Zhelun Shi, Zhipin Wang, Hongxing Fan, Zaibin Zhang, Lijun Li, Yongting Zhang, Zhenfei Yin, Lu Sheng, Yu Qiao, and Jing Shao

   Arxiv, May 2024

   Website
3. Attack

   CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completion

   Qibing Ren, Chang Gao, Jing Shao, Junchi Yan, Xin Tan, Wai Lam, and Lizhuang Ma

   ACL, May 2024

   Website
4. Frontier AI Risk Analysis

   Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models

   Chen Qian, Jie Zhang, Wei Yao, Dongrui Liu, Zhenfei Yin, Yu Qiao, Yong Liu, and Jing Shao

   ACL, May 2024

   Website
5. Evaluation

   SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models

   Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, and Jing Shao

   ACL, May 2024

   Website
6. Frontier AI Risk Analysis

   From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalities

   Chaochao Lu, Chen Qian, Guodong Zheng, Hongxing Fan, Hongzhi Gao, Jie Zhang, Jing Shao, Jingyi Deng, Jinlan Fu, Kexin Huang, Kunchang Li, Lijun Li, Limin Wang, Lu Sheng, Meiqi Chen, Ming Zhang, Qibing Ren, Sirui Chen, Tao Gui, Wanli Ouyang, Yali Wang, Yan Teng, Yaru Wang, Yi Wang, Yinan He, Yingchun Wang, Yixu Wang, Yongting Zhang, Yu Qiao, Yujiong Shen, Yurong Mou, Yuxi Chen, Zaibin Zhang, Zhelun Shi, Zhenfei Yin, and Zhipin Wang

   Technicle Report, May 2024

   Website
7. Attack

   PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety

   Zaibin Zhang, Yongting Zhang, Lijun Li, Hongzhi Gao, Lijun Wang, Huchuan Lu, Feng Zhao, Yu Qiao, and Jing Shao

   ACL 2024, May 2024

   ACL 2024 Outstanding Paper Award Code Website

   https://2024.aclweb.org/program/best_papers/

8. Frontier AI Risk Analysis

   REEF: Representation Encoding Fingerprints for Large Language Models

   Jie Zhang

   ICLR 2025, May 2024

   ICLR 2025 Oral arXiv Code

   https://iclr.cc/virtual/2025/events/oral

9. Frontier AI Risk Analysis

   The Tug of War Within: Mitigating the Fairness-Privacy Conflicts in Large Language Models

   Chen Qian, Dongrui Liu, Jie Zhang, Yong Liu, and Jing Shao

   arXiv, May 2024

   Code
10. Attack

    LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts

    Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, and Jing Shao

    ACL 2025, May 2024

    ACL 2025 Outstanding Paper Award arXiv Code

    https://2025.aclweb.org/program/awards/

11. Frontier AI Risk Analysis

    OASIS: Open Agent Social Interaction Simulations with One Million Agents

    Ziyi Yang, Zaibin Zhang, Zirui Zheng, Yuxian Jiang, Ziyue Gan, Zhiyu Wang, Zijian Ling, Jinsong Chen, Martz Ma, Bowen Dong, Prateek Gupta, Shuyue Hu, Zhenfei Yin, Guohao Li, Xu Jia, Lijun Wang, Bernard Ghanem, Huchuan Lu, Chaochao Lu, Wanli Ouyang, Yu Qiao, Philip Torr, and Jing Shao

    arXiv, May 2024

    arXiv Code

2023

1. Evaluation

   ChEF: A Comprehensive Evaluation Framework for Standardized Assessment of Multimodal Large Language Models

   Zhelun* Shi, Zhipin* Wang, Hongxing* Fan, Zhenfei Yin, Lu† Sheng, Yu Qiao, and Jing† Shao

   Arxiv, May 2023

   Website
2. Evaluation

   LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmark

   Zhenfei* Yin, Jiong* Wang, JianJian* Cao, Zhelun* Shi, Dingning Liu, Mukai Li, Lu Sheng, Lei† Bai, Xiaoshui Huang, Zhiyong Wang, Jing† Shao, and Wanli Ouyang

   NeurIPS, May 2023

   Website

2022

1. CV

   1st Place Solutions for RxR-Habitat Vision-and-Language Navigation Competition (CVPR 2022)

   Dong An, Zun Wang, Yangguang Li, Yi Wang, Yicong Hong, Yan Huang, Liang Wang, and Jing Shao

   CVPR, May 2022
2. CV

   ERGO: Event Relational Graph Transformer for Document-level Event Causality Identification

   Meiqi Chen, Yixin Cao, Kunquan Deng, Mukai Li, Kun Wang, Jing Shao, and Yan Zhang

   Arxiv, May 2022
3. CV

   Democratizing Contrastive Language-Image Pre-training: A CLIP Benchmark of Data, Model, and Supervision

   Yufeng Cui, Lichen Zhao, Feng Liang, Yangguang Li, and Jing Shao

   Arxiv, May 2022
4. CV

   X-Learner: Learning Cross Sources and Tasks for Universal Visual Representation

   Yinan He, Gengshi Huang, Siyu Chen, Jianing Teng, Wang Kun, Zhenfei Yin, Lu Sheng, Ziwei Liu, Yu Qiao, and Jing Shao

   ECCV, May 2022
5. CV

   Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm

   Yangguang Li, Feng Liang, Lichen Zhao, Yufeng Cui, Wanli Ouyang, Jing Shao, Fengwei Yu, and Junjie Yan

   In International Conference on Learning Representations, Mar 2022
6. CV

   MMEKG: Multi-modal Event Knowledge Graph towards Universal Representation across Modalities

   Yubo Ma, Zehao Wang, Mukai Li, Yixin Cao, Meiqi Chen, Xinze Li, Wenqi Sun, Kunquan Deng, Kun Wang, Aixin Sun, and Jing Shao

   In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, May 2022
7. CV

   Prompt for Extraction? PAIE: Prompting Argument Interaction for Event Argument Extraction

   Yubo Ma, Zehao Wang, Yixin Cao, Mukai Li, Meiqi Chen, Kun Wang, and Jing Shao

   In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), May 2022
8. CV

   ST-Adapter: Parameter-Efficient Image-to-Video Transfer Learning for Action Recognition

   Junting Pan, Ziyi Lin, Xiatian Zhu, Jing Shao, and Hongsheng Li

   NeurIPS, May 2022
9. CV

   Few-shot Forgery Detection via Guided Adversarial Interpolation

   Haonan Qiu, Siyu Chen, Bei Gan, Kun Wang, Huafeng Shi, Jing Shao, and Ziwei Liu

   Arxiv, May 2022
10. CV

    Task-Balanced Distillation for Object Detection

    Ruining Tang, Zhenyu Liu, Yangguang Li, Yiguo Song, Hui Liu, Qide Wang, Jing Shao, Guifang Duan, and Jianrong Tan

    PR, May 2022
11. CV

    RePre: Improving Self-Supervised Vision Transformer with Reconstructive Pre-training

    Luya Wang, Feng Liang, Yangguang Li, Honggang Zhang, Wanli Ouyang, and Jing Shao

    In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, Jul 2022
12. CV

    SNCSE: Contrastive Learning for Unsupervised Sentence Embedding with Soft Negative Samples

    Hao Wang, Yangguang Li, Zhen Huang, Yong Dou, Lingpeng Kong, and Jing Shao

    ICIC, Jul 2022
13. CV

    Bamboo: Building Mega-Scale Vision Dataset Continually with Human-Machine Synergy

    Yuanhan Zhang, Qinghong Sun, Yichun Zhou, Zexin He, Zhenfei Yin, Kun Wang, Lu Sheng, Yu Qiao, Jing Shao, and Ziwei Liu

    Arxiv, Jul 2022
14. Evaluation

    Benchmarking Omni-Vision Representation through the Lens of Visual Realms

    Yuanhan Zhang, Zhenfei Yin, Jing Shao, and Ziwei Liu

    ECCV, Jul 2022
15. CV

    Robust Face Anti-Spoofing with Dual Probabilistic Modeling

    Yuanhan Zhang, Yichao Wu, Zhenfei Yin, Jing Shao, and Ziwei Liu

    Arxiv, Jul 2022

2021

1. CV

   Actor-Context-Actor Relation Network for Spatio-Temporal Action Localization

   Junting Pan, Siyu Chen, Mike Zheng Shou, Yu Liu, Jing Shao, and Hongsheng Li

   In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2021
2. CV

   ForgeryNet - Face Forgery Analysis Challenge 2021: Methods and Results

   Yinan He, Lu Sheng, Jing Shao, Ziwei Liu, Zhaofan Zou, Zhizhi Guo, Shan Jiang, Curitis Sun, Guosheng Zhang, Keyao Wang, Haixiao Yue, Zhibin Hong, Wanguo Wang, Zhenyu Li, Qi Wang, Zhenli Wang, Ronghao Xu, Mingwen Zhang, Zhiheng Wang, Zhenhang Huang, Tianming Zhang, and Ningning Zhao

   CoRR, Jun 2021
3. Evaluation

   ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis

   Yinan He, Bei Gan, Siyu Chen, Yichun Zhou, Guojun Yin, Luchuan Song, Lu Sheng, Jing Shao, and Ziwei Liu

   In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2021
4. CV

   A Simple Long-Tailed Recognition Baseline via Vision-Language Model

   Teli Ma, Shijie Geng, Mengmeng Wang, Jing Shao, Jiasen Lu, Hongsheng Li, Peng Gao, and Yu Qiao

   CoRR, Jun 2021
5. CV

   Few-Shot Domain Expansion for Face Anti-Spoofing

   Bowen Yang, Jing Zhang, Zhenfei Yin, and Jing Shao

   CoRR, Jun 2021
6. CV

   BlockQNN: Efficient Block-Wise Neural Network Architecture Generation

   Zhao Zhong, Zichen Yang, Boyang Deng, Junjie Yan, Wei Wu, Jing Shao, and Cheng-Lin Liu

   IEEE Transactions on Pattern Analysis and Machine Intelligence, Jul 2021

   Conference Name: IEEE Transactions on Pattern Analysis and Machine Intelligence

   Abs

   Convolutional neural networks have gained a remarkable success in computer vision. However, most popular network architectures are hand-crafted and usually require expertise and elaborate design. In this paper, we provide a block-wise network generation pipeline called BlockQNN which automatically builds high-performance networks using the Q-Learning paradigm with epsilon-greedy exploration strategy. The optimal network block is constructed by the learning agent which is trained to choose component layers sequentially. We stack the block to construct the whole auto-generated network. To accelerate the generation process, we also propose a distributed asynchronous framework and an early stop strategy. The block-wise generation brings unique advantages: (1) it yields state-of-the-art results in comparison to the hand-crafted networks on image classification, particularly, the best network generated by BlockQNN achieves 2.35 percent top-1 error rate on CIFAR-10. (2) it offers tremendous reduction of the search space in designing networks, spending only 3 days with 32 GPUs. A faster version can yield a comparable result with only 1 GPU in 20 hours. (3) it has strong generalizability in that the network built on CIFAR also performs well on the larger-scale dataset. The best network achieves very competitive accuracy of 82.0 percent top-1 and 96.0 percent top-5 on ImageNet.

2020

1. CV

   1st place solution for AVA-Kinetics Crossover in AcitivityNet Challenge 2020

   Siyu Chen, Junting Pan, Guanglu Song, Manyuan Zhang, Hao Shao, Ziyi Lin, Jing Shao, Hongsheng Li, and Yu Liu

   CoRR, Jul 2020
2. CV

   High-Quality Video Generation from Static Structural Annotations

   Lu Sheng, Junting Pan, Jiaming Guo, Jing Shao, and Chen Change Loy

   International Journal of Computer Vision, Nov 2020

   Abs

   This paper proposes a novel unsupervised video generation that is conditioned on a single structural annotation map, which in contrast to prior conditioned video generation approaches, provides a good balance between motion flexibility and visual quality in the generation process. Different from end-to-end approaches that model the scene appearance and dynamics in a single shot, we try to decompose this difficult task into two easier sub-tasks in a divide-and-conquer fashion, thus achieving remarkable results overall. The first sub-task is an image-to-image (I2I) translation task that synthesizes high-quality starting frame from the input structural annotation map. The second image-to-video (I2V) generation task applies the synthesized starting frame and the associated structural annotation map to animate the scene dynamics for the generation of a photorealistic and temporally coherent video. We employ a cycle-consistent flow-based conditioned variational autoencoder to capture the long-term motion distributions, by which the learned bi-directional flows ensure the physical reliability of the predicted motions and provide explicit occlusion handling in a principled manner. Integrating structural annotations into the flow prediction also improves the structural awareness in the I2V generation process. Quantitative and qualitative evaluations over the autonomous driving and human action datasets demonstrate the effectiveness of the proposed approach over the state-of-the-art methods. The code has been released: https://github.com/junting/seg2vid.

3. CV

   Morphing and Sampling Network for Dense Point Cloud Completion

   Minghua Liu, Lu Sheng, Sheng Yang, Jing Shao, and Shi-Min Hu

   In Proceedings of the AAAI Conference on Artificial Intelligence, Apr 2020

   Number: 07
4. CV

   CelebA-Spoof: Large-Scale Face Anti-spoofing Dataset with Rich Annotations

   Yuanhan Zhang, ZhenFei Yin, Yidong Li, Guojun Yin, Junjie Yan, Jing Shao, and Ziwei Liu

   In Computer Vision – ECCV 2020, Apr 2020
5. CV

   Learning Connectivity of Neural Networks from a Topological Perspective

   Kun Yuan, Quanquan Li, Jing Shao, and Junjie Yan

   In Computer Vision – ECCV 2020, Apr 2020
6. CV

   Powering One-Shot Topological NAS with Stabilized Share-Parameter Proxy

   Ronghao Guo, Chen Lin, Chuming Li, Keyu Tian, Ming Sun, Lu Sheng, and Junjie Yan

   In Computer Vision – ECCV 2020, Apr 2020
7. CV

   Thinking in Frequency: Face Forgery Detection by Mining Frequency-Aware Clues

   Yuyang Qian, Guojun Yin, Lu Sheng, Zixuan Chen, and Jing Shao

   In Computer Vision – ECCV 2020, Apr 2020
8. CV

   PV-NAS: Practical Neural Architecture Search for Video Recognition

   Zihao Wang, Chen Lin, Lu Sheng, Junjie Yan, and Jing Shao

   CoRR, Apr 2020
9. PV-NAS: Practical Neural Architecture Search for Video Recognition

   Zihao Wang, Chen Lin, Lu Sheng, Junjie Yan, and Jing Shao

   CoRR, Apr 2020

2019

1. CV

   Unsupervised Bi-directional Flow-based Video Generation from one Snapshot

   Lu Sheng, Junting Pan, Jiaming Guo, Jing Shao, Xiaogang Wang, and Chen Change Loy

   CoRR, Apr 2019
2. CV

   Improving Referring Expression Grounding With Cross-Modal Attention-Guided Erasing

   Xihui Liu, Zihao Wang, Jing Shao, Xiaogang Wang, and Hongsheng Li

   In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2019
3. CV

   Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis

   Xihui Liu, Guojun Yin, Jing Shao, Xiaogang Wang, and hongsheng Li

   In Advances in Neural Information Processing Systems, Jun 2019
4. CV

   Video Generation From Single Semantic Label Map

   Junting Pan, Chengyu Wang, Xu Jia, Jing Shao, Lu Sheng, Junjie Yan, and Xiaogang Wang

   In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2019

   Abs

   This paper proposes the novel task of video generation conditioned on a SINGLE semantic label map, which provides a good balance between flexibility and quality in the generation process. Different from typical end-to-end approaches, which model both scene content and dynamics in a single step, we propose to decompose this difficult task into two sub-problems. As current image generation methods do better than video generation in terms of detail, we synthesize high quality content by only generating the first frame. Then we animate the scene based on its semantic meaning to obtain temporally coherent video, giving us excellent results overall. We employ a cVAE for predicting optical flow as a beneficial intermediate step to generate a video sequence conditioned on the initial single frame. A semantic label map is integrated into the flow prediction module to achieve major improvements in the image-to-video generation process. Extensive experiments on the Cityscapes dataset show that our method outperforms all competing methods. The source code will be released on https://github.com/junting/seg2vid.

5. CV

   CAMP: Cross-Modal Adaptive Message Passing for Text-Image Retrieval

   Zihao Wang, Xihui Liu, Hongsheng Li, Lu Sheng, Junjie Yan, Xiaogang Wang, and Jing Shao

   In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Oct 2019
6. CV

   Context and Attribute Grounded Dense Captioning

   Guojun Yin, Lu Sheng, Bin Liu, Nenghai Yu, Xiaogang Wang, and Jing Shao

   In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2019

   Abs

   Dense captioning aims at simultaneously localizing semantic regions and describing these regions-of-interest (ROIs) with short phrases or sentences in natural language. Previous studies have shown remarkable progresses, but they are often vulnerable to the aperture problem that a caption generated by the features inside one ROI lacks contextual coherence with its surrounding context in the input image. In this work, we investigate contextual reasoning based on multi-scale message propagations from the neighboring contents to the target ROIs. To this end, we design a novel end-to-end context and attribute grounded dense captioning framework consisting of 1) a contextual visual mining module and 2) a multi-level attribute grounded description generation module. Knowing that captions often co-occur with the linguistic attributes (such as who, what and where), we also incorporate an auxiliary supervision from hierarchical linguistic attributes to augment the distinctiveness of the learned captions. Extensive experiments and ablation studies on Visual Genome dataset demonstrate the superiority of the proposed model in comparison to the state-of-the-art methods.

7. CV

   Semantics Disentangling for Text-To-Image Generation

   Guojun Yin, Bin Liu, Lu Sheng, Nenghai Yu, Xiaogang Wang, and Jing Shao

   In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2019

2018

1. CV

   Multi-Label Image Classification via Knowledge Distillation from Weakly-Supervised Detection

   Yongcheng Liu, Lu Sheng, Jing Shao, Junjie Yan, Shiming Xiang, and Chunhong Pan

   In Proceedings of the 26th ACM international conference on Multimedia, Jun 2018
2. CV

   Localization Guided Learning for Pedestrian Attribute Recognition

   Pengze Liu, Xihui Liu, Junjie Yan, and Jing Shao

   In British Machine Vision Conference 2018, BMVC 2018, Sep 2018
3. CV

   Exploring Disentangled Feature Representation Beyond Face Identification

   Yu Liu, Fangyin Wei, Jing Shao, Lu Sheng, Junjie Yan, and Xiaogang Wang

   In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun 2018
4. CV

   Improving Deep Visual Representation for Person Re-identification by Global and Local Image-language Association

   Dapeng Chen, Hongsheng Li, Xihui Liu, Yantao Shen, Jing Shao, Zejian Yuan, and Xiaogang Wang

   In Computer Vision – ECCV 2018, Jun 2018
5. CV

   Show, Tell and Discriminate: Image Captioning by Self-retrieval with Partially Labeled Data

   Xihui Liu, Hongsheng Li, Jing Shao, Dapeng Chen, and Xiaogang Wang

   In Computer Vision – ECCV 2018, Jun 2018
6. CV

   Transductive Centroid Projection for Semi-supervised Large-Scale Recognition

   Yu Liu, Guanglu Song, Jing Shao, Xiao Jin, and Xiaogang Wang

   In Computer Vision – ECCV 2018, Jun 2018
7. CV

   Zoom-Net: Mining Deep Feature Interactions for Visual Relationship Recognition

   Guojun Yin, Lu Sheng, Bin Liu, Nenghai Yu, Xiaogang Wang, Jing Shao, and Chen Change Loy

   In Computer Vision – ECCV 2018, Jun 2018
8. CV

   Avatar-Net: Multi-scale Zero-Shot Style Transfer by Feature Decoration

   Lu Sheng, Ziyi Lin, Jing Shao, and Xiaogang Wang

   In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun 2018
9. CV

   Practical Block-Wise Neural Network Architecture Generation

   Zhao Zhong, Junjie Yan, Wei Wu, Jing Shao, and Cheng-Lin Liu

   In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun 2018

2017

1. CV

   Orientation Invariant Feature Embedding and Spatial Temporal Regularization for Vehicle Re-identification

   Zhongdao Wang, Luming Tang, Xihui Liu, Zhuliang Yao, Shuai Yi, Jing Shao, Junjie Yan, Shengjin Wang, Hongsheng Li, and Xiaogang Wang

   In 2017 IEEE International Conference on Computer Vision (ICCV), Jun 2017
2. CV

   HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis

   Xihui Liu, Haiyu Zhao, Maoqing Tian, Lu Sheng, Jing Shao, Shuai Yi, Junjie Yan, and Xiaogang Wang

   In 2017 IEEE International Conference on Computer Vision (ICCV), Oct 2017
3. CV

   Crowded Scene Understanding by Deeply Learned Volumetric Slices

   Jing Shao, Chen Change Loy, Kai Kang, and Xiaogang Wang

   IEEE Transactions on Circuits and Systems for Video Technology, Mar 2017
4. CV

   Learning Scene-Independent Group Descriptors for Crowd Understanding

   Jing Shao, Chen Change Loy, and Xiaogang Wang

   IEEE Transactions on Circuits and Systems for Video Technology, Jun 2017
5. CV

   Spindle Net: Person Re-identification with Human Body Region Guided Feature Decomposition and Fusion

   Haiyu Zhao, Maoqing Tian, Shuyang Sun, Jing Shao, Junjie Yan, Shuai Yi, Xiaogang Wang, and Xiaoou Tang

   In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jul 2017

2016

1. CV

   Slicing Convolutional Neural Network for Crowd Video Understanding

   Jing Shao, Chen Change Loy, Kai Kang, and Xiaogang Wang

   In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2016

   ISSN: 1063-6919

   Abs

   Learning and capturing both appearance and dynamic representations are pivotal for crowd video understanding. Convolutional Neural Networks (CNNs) have shown its remarkable potential in learning appearance representations from images. However, the learning of dynamic representation, and how it can be effectively combined with appearance features for video analysis, remains an open problem. In this study, we propose a novel spatio-temporal CNN, named Slicing CNN (S-CNN), based on the decomposition of 3D feature maps into 2D spatio-and 2D temporal-slices representations. The decomposition brings unique advantages: (1) the model is capable of capturing dynamics of different semantic units such as groups and objects, (2) it learns separated appearance and dynamic representations while keeping proper interactions between them, and (3) it exploits the selectiveness of spatial filters to discard irrelevant background clutter for crowd understanding. We demonstrate the effectiveness of the proposed S-CNN model on the WWW crowd video dataset for attribute recognition and observe significant performance improvements to the state-of-the-art methods (62.55% from 51.84% [21]).

2015

1. CV

   Deeply learned attributes for crowded scene understanding

   Jing Shao, Kai Kang, Chen Change Loy, and Xiaogang Wang

   In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2015

2014

1. CV

   Scene-Independent Group Profiling in Crowd

   Jing Shao, Chen Change Loy, and Xiaogang Wang

   In 2014 IEEE Conference on Computer Vision and Pattern Recognition, Jun 2014