ArXiv Preprint
In this paper, we tackle the new task of video-based Activated Muscle Group
Estimation (AMGE) aiming at identifying currently activated muscular regions of
humans performing a specific activity. Video-based AMGE is an important yet
overlooked problem. To this intent, we provide the MuscleMap136 featuring >15K
video clips with 136 different activities and 20 labeled muscle groups. This
dataset opens the vistas to multiple video-based applications in sports and
rehabilitation medicine. We further complement the main MuscleMap136 dataset,
which specifically targets physical exercise, with Muscle-UCF90 and
Muscle-HMDB41, which are new variants of the well-known activity recognition
benchmarks extended with AMGE annotations. With MuscleMap136, we discover
limitations of state-of-the-art architectures for human activity recognition
when dealing with multi-label muscle annotations and good generalization to
unseen activities is required. To address this, we propose a new multimodal
transformer-based model, TransM3E, which surpasses current activity recognition
models for AMGE, especially as it comes to dealing with previously unseen
activities. The datasets and code will be publicly available at
https://github.com/KPeng9510/MuscleMap.
Kunyu Peng, David Schneider, Alina Roitberg, Kailun Yang, Jiaming Zhang, M. Saquib Sarfraz, Rainer Stiefelhagen
2023-03-02
