Gait Recognition from Motion Capture Data

Overview

Recent rapid improvement in motion capture (MoCap) sensor accuracy brought affordable technology that can identify walking people. MoCap technology provides video clips of walking individuals containing structural motion data. The format keeps an overall structure of the human body and holds estimated 3D positions of major anatomical landmarks as the person moves. MoCap data can be collected online by a system of multiple cameras (Vicon) or a depth camera (Microsoft Kinect). To visualize MoCap data, a simplified stick figure representing the human skeleton (graph of joints connected by bones) can be recovered from body point spatial coordinates.

In the field of gait recognition from motion capture data, designing human-interpretable gait features is a common practice of many fellow researchers. To refrain from ad-hoc schemes and to find maximally discriminative features we may need to explore beyond the limits of human interpretability. MoCap-assisted human identification, as a pattern recognition discipline, can be optimized using a machine learning approach. Yet in some applications such as video surveillance new identities can appear on the fly and labeled learning data for all encountered people may not always be available. Therefore, we believe that gait is best utilized in person re-identification.

This work introduces the concept of learning robust, universal gait features directly from raw joint coordinates. The features are learned by a modification of the Fisher’s Linear Discriminant Analysis with Maximum Margin Criterion so that the identities are maximally separated.

Experiments on the CMU MoCap database show that these features can discriminate other people than who they are learned on, but also that the number of learning identities can be much smaller than the number of walkers encountered in the real operation. Additional experiments indicate that this method is a leading concept for rank-based classifier systems by outperforming other relevant methods in terms of the distribution of biometric templates in respective feature spaces expressed in four class separability coefficients.

On April 6th, Michal Balazia obtained the PhD degree with dissertation Gait Recognition from Motion Capture Data.

Database

Data in the Acclaim ASF/AMC format used in this project was created with funding from NSF EIA-0196217 and was obtained from the CMU Graphics Lab.

Original database original.zip (967 MB) contains original bone rotational data in the "amcOriginal" folder. The material also contains a prototypical skeleton in the "skeleton.asf" file as the average of the original skeletons and a prototypical gait cycle "gaitcycle.amc" that was manually selected as one of the clearest gait cycles in the original database annotated as walk.

Normalized database normalized.zip (881 MB) contains original bone rotational data normalized with respect to person’s position and walk direction. We set the root translation and rotation to zero in all frames of all AMC files, thus center of the coordinate system is moved to the position of root joint and axes are adjusted to the walker’s perspective: the X axis is from right (negative) to left (positive), the Y axis is from down (negative) to up (positive), and the Z axis is from back (negative) to front (positive). Normalized bone rotational data are in the "amc" folder.

Extracted databases contain only gait cycles extracted from the normalized database. The larger database we pick the less its motions resemble gait cycles, that is, are less similar to the prototypical gait cycle. Names of the AMC files have the form "subject_id[from-to].amc" determining the submotion extracted from the original "subject_id.amc" file starting with "from" frame and ending with "to" frame. We have extracted multiple databases of variable sizes:
extracted-56.3.zip (1 MB) of 2 identities and 35 gait cycles
extracted-59.4.zip (2 MB) of 4 identities and 67 gait cycles
extracted-63.3.zip (5 MB) of 8 identities and 130 gait cycles
extracted-73.7.zip (11 MB) of 16 identities and 302 gait cycles
extracted-173.3.zip (75 MB) of 32 identities and 2047 gait cycles
extracted-302.0.zip (135 MB) of 54 identities and 3843 gait cycles, used at evaluation in [2,3,4,5]
extracted-495.3.zip (200 MB) of 64 identities and 5923 gait cycles, used at evaluation in [1]

We also attach this ASF/AMC Motion Capture Player software mocapplayer.exe (1 MB).

Framework

Publications

[1] Balazia M., Sojka P.: You Are How You Walk: Uncooperative MoCap Gait Identification for Video Surveillance with Incomplete and Noisy Data. In: Proceedings of the IEEE/IAPR International Joint Conference on Biometrics (IJCB), IEEE, pp 208-215, Denver, USA, October 2017. DOI, PREPRINT, POSTER.

[2] Balazia M., Sojka P.: Gait Recognition from Motion Capture Data. In: ACM Transactions on Multimedia Computing (TOMM), special issue on Representation, Analysis and Recognition of 3D Humans, ACM, volume 14(1s), pp 22:1-22:18, New York, USA, February 2018. DOI, PREPRINT.

[3] Balazia M., Sojka P.: Learning Robust Features for Gait Recognition by Maximum Margin Criterion. In: Proceedings of the IEEE/IAPR International Conference on Pattern Recognition (ICPR), IEEE, pp 901-906, Cancun, Mexico, December 2016. DOI, PREPRINT, POSTER.

[4] Balazia M., Sojka P.: Walker-Independent Features for Gait Recognition from Motion Capture Data. In: Proceedings of the joint IAPR International Workshops on Structural and Syntactic Pattern Recognition (SSPR) and Statistical Techniques in Pattern Recognition (SPR), Springer, LNCS 10029, pp 310-321, Merida, Mexico, November 2016. DOI, PREPRINT.

[5] Balazia M., Sojka P.: An Evaluation Framework and Database for MoCap-Based Gait Recognition Methods. In: Proceedings of the IAPR Workshop on Reproducible Research in Pattern Recognition (RRPR), Springer, LNCS 10214, pp 33-47, Cancun, Mexico, December 2016. DOI, PREPRINT, POSTER.

[6] Balazia M., Plataniotis K.N.: Human Gait Recognition from Motion Capture Data in Signature Poses. In: IET Biometrics, IET, volume 6(2), pp 129-137, London, United Kingdom, March 2017. Best Paper. DOI.

[7] Balazia M., Sedmidubsky J., Zezula P.: Semantically Consistent Human Motion Segmentation. In: Proceedings of the International Conference on Database and Expert Systems Applications (DEXA), Springer, LNCS 8644, pp 423-437, Munich, Germany, September 2014. DOI

[8] Sedmidubsky J., Valcik J., Balazia M., Zezula P.: Gait Recognition Based on Normalized Walk Cycles. In: Proceedings of the International Symposium on Visual Computing (ISVC), Springer, LNCS 7432, pp 11-20, Rethymno, Greece, July 2012. DOI

[9] Valcik J., Sedmidubsky J., Balazia M., Zezula P., Identifying Walk Cycles for Human Recognition. In: Proceedings of the Pacific Asia Workshop on Intelligence and Security Informatics (PAISI), Springer, LNCS 7299, pp 127-135, Kuala Lumpur, Malaysia, May 2012. DOI

Awards

2018 Joseph Fourier Prize -- 1st place
by French Institute in Prague
to Michal Balazia
for project Gait Recognition from Motion Capture Data.

2018 IET Biometrics Premium Award
by Institution of Engineering and Technology
to Michal Balazia and Konstantinos Plataniotis
for paper Human Gait Recognition from Motion Capture Data in Signature Poses.

2013 Swedish Innovation Prize -- 1st place in Civil Security
by Embassy of Sweden in Prague
to Michal Balazia
for project Gait Recognition for Biometric Surveillance.

2013 IT Security Award -- 2nd place
by Laboratory of Security and Applied Cryptography at Masaryk University
to Michal Balazia
for project Human Gait Recognition Based on Body Component Trajectories.

Contact

Michal Balazia
Room C516, Faculty of Informatics, Masaryk University, Botanicka 68a, 602 00 Brno, Czech Republic
xbalazia [at] mail [dot] muni [dot] cz
ORCID:0000-0001-7153-9984
Curriculum Vitae

Petr Sojka
Room C523, Faculty of Informatics, Masaryk University, Botanicka 68a, 602 00 Brno, Czech Republic
sojka [at] fi [dot] muni [dot] cz
ORCID:0000-0002-5768-4007