feature request: allow interaction & camera adjustments for VRM on Desktop

[nekomeowww]

Clear and concise description of the problem

For Web, we can pitch or swipe to interact the VRM model, but Desktop we cannot do this.

Suggested solution / Ideas

Perhaps, add a new Controls Island button https://github.com/moeru-ai/airi/tree/main/apps/stage-tamagotchi/src/renderer/components/StageIslands/ControlsIsland here to allow interaction? Perhaps use solar:cursor-square-line-duotone this icon?

Alternative

No response

Additional context

No response

Validations

• Follow our Code of Conduct
• Read the Contributing Guide.
• Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

[shinohara-rin]

Reopening; this was closed by mistake in cdf2788

[nekomeowww]

Nevermind, this should be supported if auto hide disabled

[hiroAkikoDy]

Hi! 👋 I'm interested in this feature and would like to contribute.

I've previously implemented VMC (Virtual Motion Capture) protocol camera control for a VTuber project using VSeeFace and VOICEVOX. Here's my reference implementation:

My Project: voicevox_Api_integration_pro

What I learned from VMC camera control:

The VMC protocol uses OSC messages like this:

/VMC/Ext/Cam (x, y, z, qx, qy, qz, qw)

This controls camera position and rotation with quaternion values.

Possible implementation approach for Desktop:

1. Input Capture (Tauri):

   • Listen to mouse drag events → Orbit camera
   • Listen to scroll events → Zoom in/out
   • Listen to right-click drag → Pan camera

2. Transform to Camera State:

   • Calculate camera position from mouse deltas
   • Convert to quaternion for rotation

3. Apply to VRM Renderer:

   • If AIRI uses Three.js + @pixiv/three-vmc, we can reuse web logic
   • Or use Tauri's invoke system to communicate with Rust backend

Questions before I start:

1. VRM Renderer: Does the desktop version use Three.js with @pixiv/three-vmc (same as web)?
2. Architecture: Is camera state managed in frontend (Vue/TS) or backend (Rust)?
3. VMC Server: Does AIRI have an internal VMC server that could receive OSC messages?

I'm new to the AIRI codebase but eager to learn. Any guidance on where to start would be appreciated! 🙏

---

Reference: VMC Protocol Specification