Researches


Facial Image Analysis

 

Facial image analysis attracted much attention in the past 30 years, due to its potential applications in biometrics, surveillances, and robotics. Facial image analysis includes several interesting research topics, such as, face detection, face alignment, face tracking, face recognition, face sketch recognition, face expression recognition, 3D face modeling, and so on. Currently, we are working on facial expression analysis, which uses computer to understand human being’s emotion or behavior based on facial images. Diversities in face shapes, human races, and environments make this task very challenging. To this end, we put more emphasis on the issues occurring in some current real systems and obtained three main achievements: 1) we designed a novel encoded dynamic feature, which can efficiently describe temporal variations of the face appearance and is also robust to variable time resolution; 2) we developed a ranking model to estimate facial expression intensity for efficiently discovering subtle emotion variations; 3) inspired by psychology studies on action units, we proposed to learn composite features for expression analysis, which are especially useful for low-intensity expression analysis.

Besides face recognition and facial expression analysis, we also contributed to face alignment, which is a very important pre-processing step for facial image analysis systems. We developed a new component based face alignment framework, which incorporates prior information of face geometric structure and adopts a nonlinear global shape model to capture complex face shape variations.

We also did some work on face recognition in our early work. We proposed to use a nonlinear discriminant subspace for face recognition and we did a series of work to further improve the performance. This nonlinear subspace framework not only inherits the advantages of the previous popular linear subspace based face recognition methods, but also handles complex variations of illumination, pose, and expression very well due to its nonlinear property. We also developed a new face recognition system, which aims to identify face sketch from face photo images. This new type of face recognition system is extremely useful in law enforcement applications.

 

Some representative papers are:

[1]. Yuchi Huang, Qingshan Liu, and Dimitris. N. Metaxas, A Component Based Framework for Generalized Face Alignment, IEEE Trans. on System, Man, and Cybernetics, Part B, 41(1):287-298, 2011.

[2]. Peng Yang, Qingshan Liu, and Dimitris. N. Metaxas, Dynamic Soft Encoded Patterns for Facial Event Analysis, Computer Vision and Image Understanding, 115(3):456-465, 2011.

[3]. Peng Yang, Qingshan Liu, and Dimitris N. Metaxas, Exploring Facial Expression with Compositional Features, Int'l Conf. Computer Vision and Pattern Recognition (CVPR), 2010.

[4]. Peng Yang, Qingshan Liu, and Dimitris N Metaxas, Boosting Encoded Dynamic Features for Facial Expression Recognition, Pattern Recognition Letters, 30(2): 132-139, 2009.

[5]. Peng Yang, Qingshan Liu, and Dimitris N. Metaxas, RankBoost with l1  Regularization for Facial Expression Recognition and Intensity Estimation, Intl Conf. on Computer Vision (ICCV), 2009

[6]. Peng Yang, Qingshan Liu, Dimitris N. Metaxas, Similarity Features for Facial Event Analysis, European Conf. Computer Vision (ECCV), 2008.

[7]. Peng Yang, Qingshan Liu, Xinyi Cui, and Dimitris N. Metaxas, Facial Expression Recognition Based on Dynamic Binary Patterns, Int'l Conf. Computer Vision and Pattern Recognition (CVPR), 2008.

[8]. Peng Yang, Qingshan Liu, and Dimitris. N. Metaxas, Boosting Coded Dynamic Features for Facial Action Units and Facial Expression Recognition, Intl Conf. Computer Vision and Pattern Recognition (CVPR), 2007.

[9]. Yuchi Huang, Qingshan Liu, and Dimitris. N. Metaxas, A Component Based Deformable Model for Generalized Face Alignment, Intl Conf. Computer Vision (ICCV), 2007.

[10]. Qingshan Liu, Xiaoou Tang, Hanqing Lu, and Songde Ma, Face Recognition Using Kernel Scatter- Difference based Discriminant analysisIEEE Trans. on Neural Networks17(4):1081-1085, 2006. 

[11]. Qingshan Liu, Xiaoou Tang, Hongliang Jin, Hanqing Lu, and Songde Ma,  A Nonlinear Approach For Face Sketch Synthesis and Recognition,  Int'l Conf. Computer Vision and Pattern Recognition (CVPR), 2005.

[12]. Qingshan Liu, Hanqing Lu, and Songde Ma,  Improving Kernel Fisher Discriminant Analysis for Face Recognition,  IEEE Trans. on Circuits and Systems for Video Technology, 14(1): 42-49, 2004.

[13]. Qingshan Liu, Rui Huang, Hanqing Lu, and Songde Ma,  Face Recognition Using Kernel Based Fisher Discriminant Analysis,  Int'l Conf. Automatic Face and Gesture Recognition (FGR),  2002.

 

Graph-& Hypergraph-based Image and Video Understanding

 

It is well known that a set of image or video data endowed with pairwise relationships can be naturally organized as a pairwise graph. Moreover, the graph-based clustering and classification methods have attracted much attention in recent years, and obtained much success in many computer vision tasks. We also successfully developed a hierarchal graph model for the web image annotation task. However, the traditional graphs only take account of pairwise relationship among the data, and ignore the high-order information lies in the data, which is also important for data clustering and classification. To explore the high-order relationship, we proposed to use the hypergraph to model a set of image or video data, in which a subset of data with same attributes is taken as an edge of the hypergraph. So far, we have seen success in a hypergraph-based motion object segmentation algorithm, an unsupervised image categorization algorithm, and a hypergraph ranking based image retrieval approach. Experimental results demonstrated the importance of the high-order information, such as grouping information, in the image understanding task.       

 

Some representative papers are:

[1]. Yuchi Huang, Qingshan Liu, Fengjun Lv, Yihong Gong, and Dimitris. N. Metaxas, Unsupervised Image Categorization by Hypergraph Partition, IEEE Trans. on Pattern Analysis and Machine Intelligence, 6(33): 1266-1273, 2011.

[2]. Qingshan Liu, Yuchi Huang, and Dimitris. N. Metaxas, Hypergraph with Sampling for Image Retrieval, Pattern Recognition, 10(44): 2255-2262, 2011.

[3]. Yuchi Huang, Qingshan Liu, Shaoting Zhang, and Dimitris N. Metaxas, Image Retrieval via Probabilistic Hypergraph Ranking. Int'l Conf. Computer Vision and Pattern Recognition (CVPR), 2010.

[4]. Yuchi Huang, Qingshan Liu, and Dimitris N. Metaxas, Video Object Segmentation by Hypergraph Cut,  Int'l Conf. Computer Vision and Pattern Recognition (CVPR), 2009.

[5]. Jing Liu, Mingjing Li, Qingshan Liu, Hanqing Lu, and Songde Ma, Image Annotation via Multi-Graph Learning, Pattern Recognition, 42(2): 218-228, 2009.

 

Medical Image Analysis

 

In the area of medical image analysis, we have mostly focused on diagnosing cardiac diseases based on the MR images and CT images. We have developed a novel tensor-based classification framework that better conserves the spatial-temporal structure of the myocardial deformation pattern than the conventional vector-based algorithms using MRIs. Especially, the tensor-based projection function keeps more information of the original feature space, so that abnormal tensors in the subspace can be back-projected to reveal the regional cardiac abnormality in a more physically meaningful way. We also proposed a lesion-specific coronary artery calcium (CAC) quantification framework for predicting cardiac event with multiple instance support vector machines. Besides predicting the event risks, the proposed method can also give a better insight of the characterization of the vulnerable and culprit lesions in CAC.

Additionally, we designed a new active contour model based on the Lennard-Jones (L-J) force field, which can be applied to medical image segmentation. Different from previous work, this new model does not rely on any computed edge map and is directly calculated from image data. We proposed a new registration method for deformable soft tissues between fluoroscopic images and digitally reconstructed radiograph images from planning CT images using active shape models. It can be used to determine beam-on timing and treatment window for the radiation beam gating technology, and can potentially greatly improve the radiation treatment quality.

 

Some representative papers are:

[1]. Qingshan Liu, Zhen Qian, Idean Marvasty, Sarah Rinehart, Szilard Voros, and Dimitris N. Metaxas, Lesion-Specific Coronary Artery Calcium Quantification for Predicting Cardiac Event with Multiple Instance Support Vector Machines, Intl conf. on Medical Image Computing and Computer Assisted Intervention (MICCAI), 2010.

[2]. Zhen Qian, Qingshan Liu, Dimitris N. Metaxas, and Leon Axel, Identifying Regional Cardiac Abnormalities from Myocardial Strains Using Non-Tracking Based Strain Estimation and Spatio-Temporal Tensor Analysis, To appear in IEEE Trans. on Medical Imaging, 2011.

[3]. Zhenlong Li, Qingshan Liu, Hanqing Lu, and Dimitris N. metaxas, Lennard-Jones Force Field for Geometric Active Contour, Signal Processing, 90(4):1249-1266, 2010.

[4]. Zhen Qian, Qingshan Liu, Dimitris N. Metaxas, and Leon Axel, Identifying Regional Cardiac Abnormalities from Myocardial Strains Using Spatio-Temporal Tensor Analysis, Intl Conf. Medical Image Computing And Computer Assisted Intervention (MICCAI) 2008.

[5]. Sukmoon Chang, Jinghao Zhou, Qingshan Liu, Dimitris N. Metaxas, ect, Registration of Lung Tissue between Fluoroscope and CT Images: Determination of Beam Gaiting Parameters in Radiotherapy, Intl Conf. Medical Image Computing And Computer Assisted Intervention (MICCAI) 2007.

 

Event-based Video Analysis

 

We have also been involved in some projects on event based video analysis, especially on sports video and TV video analysis. One of the interesting projects is to analysis the soccer games on TV. As we know, during a 90-minute soccer game, the most interesting highlights usually last less than 30 minutes. So how to extract highlights becomes very important for efficient managing, indexing, retrieving, and browsing the video data. To reach this goal, we developed an efficient and robust shot classification method, a three-layer event detection framework, and a highlight ranking scheme. Based on the results on sports video analysis, we also developed a mobile browsing system to customize the highlights and deliver them to those who cannot watch the game on spot by multimedia messages. Similarly, with ever-increasing TV channels, we are exposed to overwhelming amounts of TV programs, other than sports. It is necessary to develop TV broadcast video analysis techniques for effective retrieval, summarization and browsing. We proposed a multimodal scheme to segment and represent TV video streams, which aims to recover the temporal and structural characteristics of TV programs using visual, auditory, and textual information. We also designed a coarse-to-fine scheme to robustly retrieve commercial videos, and developed a robust approach based on multispectral images gradient to detect and track TV logos in video streams.

 

Some representative papers are:

[1]. Xinyi Cui, Qingshan Liu, Mingchun Gao, and Dimitris. N. Metaxas, Abnormal Detection Using Interaction Energy Potential, Int'l Conf. Computer Vision and Pattern Recognition (CVPR), 2011.

[1]. Jinqiao Wang, Lingyu Duan, Qingshan Liu, Hanqing Lu, and Jesse Jin, A Multi-model Segementation and Representation Scheme for Broadcast Video, IEEE Trans. on Multimedia10(3): 393-408, 2008.

[2]. Xiaofeng Tong, Qingshan Liu, and Hanqing Lu, Shot Classification in Broadcast Soccer Video, Electronic Letters on Computer Vision and Image Analysis, 7(1): 16-25, 2008.

[3]. Jinqiao Wang, Qingshan Liu, Lingyu Duan, Hanqing Lu, and Changsheng Xu, Automatic TV Logo Detection, Tracking, and Removal in Broadcast Video, Intl Conf. Multimedia Modeling (MMM), 2007.

[4]. Jinqiao Wang, Lingyu Duan, Qingshan Liu, Hanqing Lu, Robust Commercial Retrieval in Video Streams. Intl Conf. Multimedia and Expo (ICME), 2007.

[5]. Qingshan Liu, Zhigang Hua, Cunxun Zang, Xiaofeng Tong, and Hanqing Lu, Providing On-Demand Sports Video to Mobile Devices, ACM Conf. Multimedia (MM), 2005.

[6]. Xiaofeng Tong, Qingshan Liu, Yifan Zhang, Hanqing Lu, Highlights Ranking for Sports Video Browsing, ACM Conf. Multimedia (MM), 2005.

[7]. Xiaofeng Tong, Qingshan Liu, Hanqing Lu, and Hongliang Jin, Semantic Units Based Event Detection in Soccer Videos, Acta Automatica Sinica, 31(4): 523-529, 2005

 

Some projects I involved in China are here

 

More details will come soon


[ Home ]  [ Resume ]  [ Research ]  [ Publications ]  [ Professional Activities ] [ Other links]  [ Chinese ]