Crowd Segmentation and Saliency Detection

Abstract: Although the traits emerged in a mass gathering are often non-deliberative, the act of mass impulse may lead to irrevocable crowd disasters. The two-fold increase of carnage in crowd since the past two decades has spurred significant advances in the field of computer vision, towards effective and proactive crowd surveillance. Computer vision studies related to crowd are observed to resonate with the understanding of the emergent behavior in physics (complex systems) and biology (animal swarm). These studies, which are inspired by biology and physics, share surprisingly common insights, and interesting contradictions. However, this aspect of discussion has not been fully explored. Therefore, this survey provides the readers with a review of the state-of-the-art methods in crowd behavior analysis from the physics and biologically inspired perspectives. We provide insights and comprehensive discussions for a broader understanding of the underlying prospect of blending physics and biology studies in computer vision.

Journal Paper: IEEE Transactions on Cybernetics, vol. 47(5), pp. 1157-1168 (TCy 2017)
GrCS: Granular Computing Based Crowd Segmentation
Ven Jyn Kok and Chee Seng Chan

Journal Paper: Neurocomputing, vol. 177, pp. 342-362 (NeuCom 2016)
Crowd Behavior Analysis: A Review where Physics meets Biology
(A crowd behavior analysis survey with 180 references)
Ven Jyn Kok, Mei Kuan Lim and Chee Seng Chan

Journal Paper: Electronics Letters, vol. 50(5), pp. 363-365 (ELL 2014)
Detection of Salient Regions in Crowded Scenes
Mei Kuan Lim, Chee Seng Chan, Dorothy Monekosso and Paolo Remagnino

Conference Paper: Proc. of IAPR Conference on Pattern Recognition, pp. 3957-3962 (ICPR 2014, ORAL)
Crowd Saliency Detection via Global Similarity Structure [Supplementary Video]
Mei Kuan Lim, Ven Jyn Kok, Chen Change Loy and Chee Seng Chan

Dataset and Code:

A dense crowd dataset with manually annotated groundtruth, collected from different public datasets. This dataset comprises 20 videos that exhibit a multitude of motion behaviors that cover both the obvious and subtle instabilities.

Paper Highlights:

• The main contribution of this study is that we propose the transformation of low-level motion features into global similarity structure. The structure allows the discovery of the intrinsic manifold of the motion dynamics in crowded scenes, which could not be captured by the low-level representation.
• The presented manifold requires
  1. No tracking , as we exploit optical flow representation, and
  2. No prior information or model learning to identify interesting/salient regions in the crowded scenes, as we employ extrema in the intrinsic manifold of motion dynamics as an indicator of saliency.

Qualitative Results: The Experimental results on public crowd scenes demonstrate the effectiveness of using our proposed method in detecting and localizing a broad scope of crowd salient motions.

Benchmark Results:

Acknowledgement

This research is supported by the Fundamental Research Grant Scheme (FRGS) MoE Grant FP027-2013A, H-00000- 60010-E13110 from the Ministry of Education Malaysia. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the University of Malaya.