Skip to content

V2.55.0#321

Merged
doranteseduardo merged 14 commits intomainfrom
v2.54.1
Apr 27, 2026
Merged

V2.55.0#321
doranteseduardo merged 14 commits intomainfrom
v2.54.1

Conversation

@doranteseduardo
Copy link
Copy Markdown
Member

ReactVision ViroCore 2.55.0

This release lands the OpenXR backend for Meta Horizon OS,
a per-source input pipeline that lets multi-pointer backends like Quest
dispatch hover and click independently without breaking single-pointer backends,
and a new Studio scenes API in the bundled Cloud Anchor SDK.
It also ships three platform fixes — Android 2025 ABI compliance, an AR image-marker regression,
and continued iOS portal stability work.

User-facing JS components and hooks built on top of these native changes
(ViroXRSceneNavigator, StudioSceneNavigator, useAnySourceHover,
useAnySourcePressed, the isQuest / hasOpenXRSupport constants, and
the platform guards on ViroARSceneNavigator / ViroVRSceneNavigator)
are documented in the companion Viro 2.55.0 release notes.


Highlights

OpenXR backend for Meta Horizon OS

Native VR rendering on Meta Quest 3 / Quest Pro / Quest 2 / Quest 1 via
Khronos OpenXR. Validated end-to-end at 90 Hz on Quest 3 with App = 5–6 ms
frame time. The backend covers the full Activity / surface / session
lifecycle plus input and presentation:

  • ViroViewOpenXR.java — Android view selected by the bridge when the
    package is registered with ReactViroPackage.ViroPlatform.QUEST. Defers
    Renderer construction until onAttachedToWindow so xrCreateSession
    binds to the Activity that actually owns the surface — resolved via
    window-token matching against live activities from ActivityThread,
    the only signal that's stable under Android's multi-resumed mode.
    Idempotent onResume / onPause (mResumed boolean) tolerate the
    multi-source RN lifecycle (Application.ActivityLifecycleCallbacks +
    RN LifecycleEventListener firing for the same Activity transition).
    Null-guards every base-class setter / getter that touches the deferred
    renderer.

  • VROSceneRendererOpenXR.cpp — C++ scene renderer.
    XR_SESSION_STATE_IDLE → READY → SYNCHRONIZED → VISIBLE → FOCUSED,
    per-frame xrWaitFrame / xrBeginFrame / xrEndFrame,
    projection-layer compositor submission, swapchain image acquisition and
    release. onResume and onPause are idempotent (early-return when
    already in the target state) so the renderer survives lifecycle events
    arriving from multiple sources without re-assigning the render
    std::thread.

  • VROInputControllerOpenXR.{h,cpp} — action-based input controller.
    Per-frame xrSyncActions for Touch controllers (triggers, grips,
    A / B / X / Y, menu, thumbsticks, haptics) and per-hand
    xrLocateHandJointsEXT + XR_FB_hand_tracking_aim for fingertip-aimed
    pointing. Pinch-to-click via FB pinch strength with a thumb-to-index
    joint-distance fallback; grip-to-grab via middle-tip-to-palm distance.
    Two-pointer dispatch. Right and left aim poses are captured without
    dispatching, then dispatched once per side via a dispatchSide lambda:
    updateHitNode(source, …)onMove(source, …)
    processGazeEvent(source) → laser update. Per-source pose hysteresis
    (~5-frame cache) keeps the aim source from flipping between controller
    and synthesized hand pose on transient single-frame xrLocateSpace
    invalidity.

  • VROInputPresenterOpenXR.h — visual presenter. World-space
    (non-headlocked) reticle plus a per-source
    unordered_map<int, Laser> of cyan VROPolyline children under the
    presenter _rootNode, lazy-initialized on first dispatch and hidden
    cleanly when no aim is available. Both lasers can be visible
    simultaneously when both hands are tracked.

  • OpenXR extensions in use: XR_KHR_opengl_es_enable,
    XR_KHR_android_create_instance, XR_FB_passthrough,
    XR_FB_display_refresh_rate, XR_EXT_hand_tracking,
    XR_FB_hand_tracking_aim. com.oculus.feature.PASSTHROUGH must be
    declared in the Android manifest for XR_FB_passthrough to be advertised
    by the OpenXR runtime — the Viro Expo plugin emits this automatically
    when xRMode includes "QUEST".

The integration goes deep into the dual-Activity story: surface mounted
inside the VR Activity, xrCreateSession bound to it (not to MainActivity),
the OpenXR session machine cycling cleanly to FOCUSED, the input dispatcher
producing stable hover / click events across both pointers, and reflection-
based VR Activity discovery from the launcher module so exitVRScene
finishes the right Activity even when the React context's currentActivity
remains pinned to MainActivity.

Per-source input state in VROInputControllerBase

VROInputControllerBase previously carried a single _hitResult,
_lastHoveredNode and _lastClickedNode — fine for single-pointer backends
(iOS / AR / Cardboard / OVR / Daydream / Wasm), wrong for multi-pointer
backends like OpenXR where two simultaneously tracked hands need
independent hover and click resolution.

This release adds additive per-source state without breaking any
existing backend:

  • New maps _hitResultsBySource, _lastHoveredNodesBySource,
    _lastClickedNodesBySource keyed by input source ID.
  • New overload updateHitNode(int source, camera, origin, ray) writes both
    the per-source map and the legacy _hitResult for backward compat.
  • New helper getHitResultForSource(source) falls back to the legacy
    _hitResult when no per-source entry exists.
  • processGazeEvent and onButtonEvent use the source-aware lookups when
    an entry is present and dispatch hover / click events independently per
    source. Otherwise they behave exactly as before.

iOS, AR, Cardboard, OVR, Daydream and Wasm — which all call
updateHitNode(camera, origin, ray) without a source parameter — see zero
behavior change. Only the OpenXR backend opts into per-source state by
calling the new overload.

Studio scenes API in libreactvisioncca

The bundled Cloud Anchor SDK (libreactvisioncca, shipped inside
virocore) gains two new endpoints that power the Studio scene fetcher in
@reactvision/react-viro:

  • getScene(sceneId, callback) — wraps
    GET /functions/v1/scenes/{scene_id}. Returns the full scene response:
    metadata, asset list with placement and material configs, animations,
    collision bindings, scene functions, project info.

  • getSceneAssets(sceneId, callback) — wraps
    GET /functions/v1/scenes/{scene_id}/assets. Asset-list-only variant
    for clients that already have scene metadata cached.

Both authenticate with the project API key and return their result inside
the existing ApiResult<T> wrapper. The full asset shape (SceneAPIAsset)
and scene-response shape (SceneData) are added to RVCCAApiTypes.h for
direct C++ consumers.

The endpoints are exposed to JS via
ViroARSceneNavigator.rvGetScene(sceneId) and
ViroARSceneNavigator.rvGetSceneAssets(sceneId). Existing Cloud Anchor and
Geospatial endpoints are unchanged.


Fixes

  • 16 KB .so page alignment for libvrapi.so (Issue A). The Android
    2025 ABI requires all shipped .so files to align to 16 KB pages.
    libvrapi.so is now repackaged with -Wl,-z,max-page-size=16384,
    resolving load failures on devices with the new page size.

  • AR Image Markers — children fixed-on-screen after re-detection
    (Android, GitHub viro #465).
    Models parented to a ViroARImageMarker no longer pin to screen
    coordinates after the target was lost and re-acquired in v2.54.0. Markers
    re-anchor cleanly to the detected world pose every time, including
    subsequent re-detection.

  • iOS ViroPortalScene portal-tree stability
    (GitHub viro #452).
    Continued portal-render-pass hardening on top of the v2.54.0 fix
    (VROPortalTreeRenderPass.cpp + VROPortal.cpp):

    • Portal stencil silhouette no longer drops transparent entry fragments
      before the alpha-discard modifier runs
      (_silhouetteMaterial->setAlphaCutoff(0.0f)).
    • 360° background inside a portal is no longer overwritten by the AR
      camera background drawn afterwards (depth = 0.9999 plus depth-write
      enabled on the portal background sphere / cube).
    • The interior of a portal hole no longer reveals the portal interior
      when the user is outside a nested exit-frame portal (skip stencil
      DECR for isExit = true, use Equal stencil function when
      anyChildIsExit = true).
    • AR occlusion is disabled inside the portal interior for
      recursionLevel > 0 so virtual content is no longer discarded by
      depth-based occlusion in nested portals.

Compatibility

  • Backward compatible with all existing single-pointer backends. iOS,
    AR (ARKit/ARCore), Cardboard, OVR, Daydream and Wasm continue to call the
    unchanged updateHitNode(camera, origin, ray) overload and observe no
    behavior change in hover / click / drag dispatch.

  • Android 2025 ABI (16 KB-aligned .so files) requires Android
    Gradle Plugin 8.7+ in the consuming app. Minimum SDK unchanged.

  • Meta Quest support requires the consuming app to register
    ReactViroPackage(ReactViroPackage.ViroPlatform.QUEST) in
    MainApplication.kt and declare the dual-Activity manifest entries
    (VRActivity, the com.oculus.intent.category.VR intent filter, Quest
    hardware features, hand-tracking and passthrough permissions). The Viro
    Expo plugin emits all of this automatically when xRMode includes
    "QUEST". Bare Android consumers will need to wire it manually.


Install path

Bare React Native is not tested for this release. ViroCore should work but
will require a substantial amount of manual wiring on the Android side
(VRActivity, manifest features, package registration in
MainApplication.kt) and iOS Podfile entries. For now we recommend
installing @reactvision/react-viro with the Expo Dev Client, where
the Viro Expo plugin emits all of the platform configuration
automatically. Bare RN support will be revisited in a follow-up release.

benmotanoti and others added 14 commits April 8, 2026 13:42
- Add OpenXR AAR (1.1.38) via local Maven repo and Khronos headers
- Configure CMake with OpenXR::openxr_loader and c++_shared STL
- Implement VROSceneRendererOpenXR: session, EGL, swapchains, frame loop
- Implement VROInputControllerOpenXR: action set, Touch/Touch Plus bindings
- Add VRODriverOpenGLAndroidOpenXR and VRODisplayOpenGLOpenXR (per-eye FBO)
- Add ViroViewOpenXR (Java bootstrap) and nativeCreateRendererOpenXR JNI
- Add standalone openxrtest/ module for on-device validation
- Fix 7 on-device crashes: STL linkage, loader init order, session state,
  EGL context ownership, uninitialised renderer

Result: stereo render at 90fps on Quest 3, no crashes.
…er events

- VROSceneRendererOpenXR: replace POC color-clear with real VRORenderer pipeline;
  prepareFrame() + renderEye() per eye; fix _driver shadowing bug (_openxrDriver +
  base _driver both set in constructor); initPassthrough() loads 8 XR_FB_passthrough
  function pointers, creates XrPassthroughFB + XrPassthroughLayerFB; setPassthroughEnabled()
  start/pause subsystem and layer; proper destroySession() teardown
- VROInputControllerOpenXR: wire right aim pose -> updateHitNode/onMove/processGazeEvent;
  right trigger -> ClickDown/Up; menu/B button -> BackButton ClickDown/Up; add file-local
  xrPoseToMatrix + xrAimForward helpers (fixes linkage bug); _prevMenuButton edge detection
- VRORenderer_JNI: nativeSetPassthroughEnabled JNI method with dynamic_pointer_cast
- Renderer.java / ViroViewOpenXR.java: setPassthroughEnabled() public API surface
- openxrtest/MainActivity: dummy Viro scene (red box at 0,0,-2) for device validation
zero m[12..14] before passing to prepareFrame; per-eye view matrices (which need the full pose) are computed separately in renderEye.
… Quest 3

All renderer features confirmed working in stereo at 90fps on Quest 3.

Two bugs found and fixed during M1 device validation:

1. Reference space: createReferenceSpace() was trying STAGE first. After the
   Quest Guardian was configured between M0 and M1 testing, STAGE succeeded —
   placing Y=0 at the floor with the eye at Y≈1.6m. Objects at Viro world
   origin appeared 39° below center, past the Quest 3's physical downward FOV,
   invisible with no error. Fixed: force XR_REFERENCE_SPACE_TYPE_LOCAL so the
   world origin matches eye level at session start, consistent with all other
   Viro platforms.

2. endFrame() never called: VROSceneRendererOpenXR::renderFrame() called
   prepareFrame() and renderEye() but never _renderer->endFrame(). endFrame()
   is where VROFrameScheduler::processTasks() runs — without it, texture
   hydration tasks queue forever. Models parsed and downloaded correctly but
   onObject3DLoaded never fired and nothing appeared, with zero error logs.
   Fixed: call _renderer->endFrame(_driver) after both eye renders, before
   xrEndFrame.
Dual-controller input (VROInputControllerOpenXR)
- Separate FLOAT trigger/grip actions per hand (threshold 0.5), edge-detected
- A/B/X/Y/Menu buttons as BOOLEAN actions; B+Menu both map to BackButton
- Thumbstick Vector2f actions → onScroll with dead zone 0.15
- Haptic output: xrApplyHapticFeedback per hand via triggerHaptic()
- New ViroOculus sources in VROInputType.h: LeftController=4, AButton=5,
  XButton=6, YButton=7, LeftGrip=8, RightGrip=9, LeftThumbstick=10,
  RightThumbstick=11
- recenterTracking() rewritten: VIEW space → head pose → yaw extraction →
  new LOCAL reference space at head XZ; was broken (located space against itself)
- openxrtest scene updated with interactive hover+click nodes for validation

Hand tracking (XR_EXT_hand_tracking + XR_FB_hand_tracking_aim)
- initHandTracking() loads 3 EXT function pointers, creates left/right trackers
- processHands(): per-frame xrLocateHandJointsEXT (26 joints), chains
  XrHandTrackingAimStateFB for aimPose + pinchStrengthIndex
- Pinch: pinchStrengthIndex >= 0.7 (FB aim) or thumb-tip<2cm (fallback)
- Grab: middle-tip to palm < 6cm → RightGrip/LeftGrip click events
- Right hand = primary pointer (updateHitNode + processGazeEvent)
- Left hand = pose-only (onMove, no hit test)
- Graceful no-op if extensions unavailable at runtime

Infrastructure fixes (blocked all input silently):
- VROInputControllerBase::processGazeEvent: add null guard on _hitResult;
  updateHitNode returns early when _scene==nullptr, leaving _hitResult null,
  previously causing SIGSEGV on first frames with hand tracking active
- VROPlatformUtil: add direct C++ renderer task queue
  (VROPlatformSetUseDirectRendererQueue / VROPlatformDrainRendererQueue);
  OpenXR has no GLSurfaceView so mRenderQueue==null and all
  VROPlatformDispatchAsyncRenderer tasks were silently dropped — setScene
  never executed, _scene never set, zero input events
- VROPlatformUtil: replace FindClass("com/viro/core/internal/PlatformUtil")
  with GetObjectClass(sPlatformUtil) in 4 call sites; native threads attached
  with AttachCurrentThread(nullptr) use bootstrap classloader, app classes
  not found → null jclass → ART JNI abort in VROPlatformDispatchAsyncBackground

Device validated on Quest 3: trigger/grip/A/B/X/Y clicks and hover confirmed
in logcat; hand aim ray hover and grab events confirmed.
@doranteseduardo doranteseduardo merged commit 4ff15e5 into main Apr 27, 2026
4 of 5 checks passed
@doranteseduardo doranteseduardo deleted the v2.54.1 branch April 27, 2026 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants