Recording, Playback, and Gesture Recognition Overview
In this overview, explore the functionalities of Kinect Studio in recording sensor data, playback of data, and gesture recognition. Meet the key program managers, Rob Relyea and Johan Marien, driving advancements in this field. Learn about the modules such as heuristic recognition and machine learning-based recognition with Gesture Builder. Discover the major improvements in version 2.0 of Kinect Studio, enabling audio recording and playback without the sensor attached. Dive into the programmatic detection of gestures and poses demonstrated in demos.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Recording, Playback, and Gesture Recognition Rob Relyea | Program Manager, Kinect Team Johan Marien | Program Manager, Kinect Team
Rob Relyea | @rrelyea Principal Program Manager, Microsoft Member of Kinect for Windows and Xbox Teams Currently focused on Hand Pointer Gestures Deep on WPF/XAML Formerly worked as Program Manager and Architect on WPF and XAML teams
Johan Marien | @jmarien Sr. Program Manager, Microsoft Johan Marien is a Program Manager on the Kinect Team working on developer tooling.
Module Overview Kinect Studio Recording sensor data Playback of data Gesture Recognition Heuristic Recognition Machine Learning Based Recognition [w/ Gesture Builder]
Click to edit Master subtitle style Kinect Studio Recording and Playback
DEMO Kinect Studio 2.0 Connecting | Monitoring | Configuring Views
Recordable Data Sources Infrared 13 MB/s Audio Depth 32 KB/s 13 MB/s Legend Record/Play Record Only Color BodyFrame 120 MB/s BodyIndex
DEMO Kinect Studio 2.0 Recording | Playback
Kinect Studio Major Improvements V2.0 V2.0 Now can record Audio Playback works without sensor attached API access to playback/record Kinect Studio, only uses these public APIs!
Click to edit Master subtitle style Gesture Recognition Heuristic + Machine Learning Approaches
DEMO Programmatic detection of a gesture/pose
Gesture Builder New tool, shipping with v2 SDK Organize data using projects and solutions Give meaning to data by tagging gestures Build gestures using machine learning technology Adaptive Boosting (AdaBoost) Trigger Determines if player is performing gesture Random Forest Regression (RFR) Progress Determines the progress of the gesture performed by player Analyze / test the results of gesture detection Live preview of results
Visual Gesture Builder Your Application
DEMO Machine Learning Demo
Gesture Recognition Heuristic Heuristic Gesture is a coding problem Quick to do simple gestures/poses (hand over head) ML can also be useful to find good signals for Heuristic approach Machine Learning (ML) with G.B. Machine Learning (ML) with G.B. Gesture is a data problem Signals which may not be easily human understandable (progress in a baseball swing) Large investment for production Danger of over-fitting, causes you to be too specific eliminating recognition of generic cases
DEMO Calling gesture detector from code
Summary Kinect Studio Recording sensor data Playback of data APIs with Microsoft.Kinect.Tools.dll Gesture Recognition Heuristic Recognition Machine Learning Based Recognition [w/ Gesture Builder]
