We present BiomechGPT, the first motion-language model built to answer detailed, clinically meaningful questions about how a person moves.
How someone moves carries rich information about their health, and accessible markerless motion capture now makes it easier to record biomechanics in clinical settings. But each clinical question, from measuring a gait parameter to suggesting a diagnosis, usually needs its own custom pipeline. Multi-modal language models offer a natural way to unify these tasks. We show that recent advances in motion-language modeling can be effectively translated to clinical movement analysis using accessible biomechanical data, and that clinical motion understanding in turn provides a precise, measurement-grounded benchmark for evaluating these models.
We trained BiomechGPT on 71 hours of clinical data from 750 participants, many with movement impairments from causes such as stroke or lower-limb prosthesis use, performing tasks common in clinical mobility assessment. As a second dataset, we used HumanML3D, a public general-purpose motion dataset. We built a cross-format tokenizer that embeds motion from heterogeneous body models into a shared latent space without requiring paired samples across formats, converting both biomechanical and SMPL-format motion into the same vocabulary for a language model to process. This let us pool training data from different sources and share annotations across them, directly addressing the scarcity of labeled clinical motion data and letting us study scaling effects.
On top of this, we built a large set of motion question–answer pairs to fine-tune the Gemma 3 language model. A single BiomechGPT model handles ten clinical tasks at once, spanning classification (activity, impairment, diagnosis, assistive-device use, and fall history) and regression (walking speed, cadence, and timed test scores). It performs competitively with non-language models, with clear advantages on activity recognition, walking cadence and speed prediction. It holds up strongly across biomechanics data from different clinical recording setups and follows instructions reliably. More importantly, performance improves steadily as we scale up both the base model and the training data, including data pooled from different sources. We characterize per-task performance and additionally, provide analysis on the effect of tokenization method.
BiomechGPT gives clinicians and researchers a natural-language interface to raw biomechanical data and offers evidence for a scalable path toward AI-supported movement analysis in rehabilitation and beyond. To support further research and clinical translation, we will release our full code and trained model checkpoints.
