Hand Tracking Streamer is a lightweight hand telemetry utility that turns a Meta Quest headset into a precision controller for robotics teleoperation and motion capture. Built on the Meta Interaction SDK, it streams 21-landmark hand data and 6-DoF wrist pose to a PC in real time over Wi-Fi using UDP for ultra-low latency or TCP for reliable data logging. The app supports left, right, or dual-hand modes with in-headset configuration, and includes a live log console and phantom hand visualization for debugging. Data is sent as structured packets of joint positions and orientations in Unity world space, making it suitable for robot control, imitation learning, and gesture-based prototyping.
Click to expand changelog
v1.1.0:
- Added video streaming from host
- Added head tracking frame
- Added debug info for frame header and HUD display
- Now uses numerical keyboard for IP and Port input
- HUD now uses TMP to improve readability
v1.0.4:
- Added wireless TCP connection
- This performs more consistently than wireless UDP and is not affected by batching
- Now gracefully handles TCP connection interruption
- Enabled menu button to pause and evoke main menu during streaming
- Added network status indication on main menu
- Now loads last used connection config automatically on start up
v1.0.3:
- Added landmark visualization for tracked hands
- Adjusted panel menu to include visualization toggle
- Fixed the issue where index of little finger were shifted by one
v1.0.2:
- Improved panel menu text readability
- Hand tracking frequency is now set to MAX
v1.0.1:
- Fixed the issue where sometimes panel menu would spawn on the floor
- Fixed missing splash screen
v1.0.0:
-
Added in-app menu to allow streaming configurations:
- Network protocol
- IP Address
- Port number
- Hand side
-
Added full bimanual tracking.
-
Added synthetic hand visualization on tracked hands.
-
Now available on SideQuest.
You can download the app from Meta Quest Store, or sideload the app with SideQuest for free.
Alternatively, you can build from source by loading hand_tracking_streamer project in Unity or directly upload hand_tracking_streamer.apk via ADB to your device.
Note
Before direct upload, make sure your device is set to developer mode, and allow USB connection. This app is built and tested on Unity 6000.0.65f1
See CONNECTIONS page for detailed documentation on connections and data format.
Not ready to integrate into your system yet? Check out the simple socket and the visualizer script provided under /scripts for quickly testing data streamed from your device.
Click to see visualizer in action
Install dependencies, connect HTS, and simply run:
python ./scripts/visualizer.py --protocol [YOUR PROTOCOL] --host [YOUR HOST IP] --port [YOUR PORT] --show-fingers
For integrating HTS data into your own pipelines, use the official Python SDK. It provides typed data structures, parsers for the HTS packet format, and utilities for real-time visualization and logging, so you can go from streamed packets to usable hand pose data quickly. The package is published on PyPI; see the documentation for API details and examples.
# if using uv
uv add hand-tracking-sdk
# or install via pip
pip install hand-tracking-sdk
Find more examples in SDK here
For support or privacy inquiries related to Hand Tracking Streamer, please email: wengmister@gmail.com
If you use Hand Tracking Streamer in your research or project, please cite it as:
@software{weng2026hts,
author={Weng, Zhengyang K.},
title={Hand Tracking Streamer: Meta Quest VR App for Motion Capture and Teleoperation},
url={https://github.com/wengmister/hand-tracking-streamer},
year={2026}
}
Apache-2.0
