ECE Master's @ UBC | Engineering Physics Bachelor's @ UBC | ML/DL Researcher | ex-Ubisoft | ex-Robert Bosch
Home
Human Lower Body Prediction
This project is sponsored by LifeBooster Inc. to help them generate lower body
prediction based on their upper body sensors. Since there is no real dataset provided, an
alternative dataset from Carnegie Mellon (CMU Graphics Lab Motion Capture Database) was used
for training and validation.
The Deep Learning Model used here is a modified LSTM-stack, which is able to recognize hidden
patterns in the upper body time series data on different motions, such as walking, running, jumping,
etc. Below is its performance on a test set in CMU animator.
Honestly, the model’s performance is actually quite decent.
- | Waiting 4 bus | Stepping | Exercising in Playgroud |
---|---|---|---|
Ground Truth | |||
Prediction |
The model did a great job on the basic motions.
- | Running | Jumping | Walking |
---|---|---|---|
Ground Truth | |||
Prediction |
What you see below are the raw input with only upper body information (Left) and forecasted result
(Right) in Unity.
Input Data | Output Result |
---|---|
Instead of blindly using standard MSE or RMSE as accuracy measurement, each joint’s prediction is
evaluated based on 2 metrics, namely Cosine Similarity for axis accuracy and Exponential of Negative
Absolute for rotation magnitude accuracy. For both metrics, value that is closer to 1 represents
better prediction. Here is an example.
Metrics Radar Chart |
---|
Due to confidential reason, product code cannot be made public.
For more detailed information, please contact me directly.
For more detailed information, please contact me directly.