Public Core

Reusable XR data contracts before framework adapters.

Rusty XR keeps shared Rust/XR concepts small, typed, testable, and independent of any one application shell. App repositories keep package identity, native platform integration, rendering policy, assets, releases, and target-device validation.

Workspace members: 14 reusable crates plus 4 public example or probe packages.

Dynamic mesh paths documented: surface coordinates, SDF fields, and collider surfaces.

Sensitive private names found by the current public boundary scan.

Current Shape

Area	Public Surface	Status
Core contracts	Math, pose, timing, views, camera/depth metadata, hands, home/menu panels, render payloads, plain layers, effect-stack diagnostics, native passthrough style descriptors, safety-gated visual strobe descriptors, counters, interaction rays, room mesh state, capture lifecycle metadata, and depth query summaries.	In progress
Runtime config	Generic runtime keys, typed values, config maps, Android property naming helpers.	In progress
BLE / LSL / OSC / Polar	BLE permission and GATT models, LSL stream schemas, OSC packet and UDP helpers, Polar H10 HR/RR/ECG/ACC helpers.	In progress
Broker model	Command envelopes, acknowledgements, client hello, stream manifests, sample headers, session manifests, transport endpoints, timing stamps, broker-clock snapshots/stamps/correlations, heartbeat state, drop counters, replay records, and synthetic stream samples.	In progress
Eye model	Screen-space gaze points, XR gaze rays, AOI hits, derived processor events, validity flags, provenance, and deterministic synthetic eye samples.	In progress
Debug canvas	Normalized diagnostic HUD documents, draw lists, text runs, tones, and input-neutral HUD commands.	In progress
Quest diagnostics	Device readiness, package launch state, frame-rate summaries, runtime counters.	In progress
Camera / depth / SDF	Projection helpers, timestamp matching, environment-depth diagnostics, unique capture cadence, readback policies, per-view metadata, packed SDF, sparse TSDF snapshots, dynamic mesh-to-SDF reference conversion, and depth-derived support/impact query contracts.	In progress
Particles	Deterministic particle storage, seeded sphere helper, fixed-step clock, render payload generation, dynamic mesh coordinate sampling, live deformed-mesh anchor updates, same-surface neighbor tiers, cross-surface neighbor links, hand-mesh snapshot sampling, dynamic mesh collider surfaces with diagnostic shells, and SDF surface-attraction helpers.	In progress

What Belongs Here

Framework-neutral data contracts.
Small deterministic helpers with synthetic tests.
Documentation for public shell boundaries and workflows.
Examples authored cleanly for this repository.

What Stays Downstream

App package identity, signing, release payloads, and launch scripts.
Native Android/OpenXR/Vulkan/renderer ownership.
Product-specific scenes, tuning, shader behavior, and generated data.
Non-public planning notes and raw audit artifacts.

Start Here

Architecture

See dependency direction, crate roles, and the shell/core split.

Docs Index

Route from current architecture to examples, validation, runbooks, and historical investigations.

Module Map

Scan public crate roles, composition boundaries, and extraction pressure.

Makepad Strategy

Understand why Rusty XR uses Makepad for Rust-native app shells while keeping camera texture lanes explicit.

Examples Matrix

Match each public example to reusable modules, hardware needs, and validation commands.

Entrypoints

Keep Rust APIs, schemas, broker APIs, CLIs, and optional MCP routes consistent.

Boundaries

Check what can move into public Rusty XR before extracting code.

Extraction Gate

Move interfaces first, implementations second, and adapters last.

Passthrough Boundary

Separate compositor passthrough, raw camera frames, environment depth, and final-display inspection.

Validation

Run the smallest command set that matches the changed surface.

Diagrams

Open the Mermaid maps for crate flow, media flow, and extraction workflow.

Utility Foundation

The public examples include contracts-only layout, composite feedback, native passthrough style catalog, audio-reactive passthrough style, visual strobe profile demos, portable hand-mesh coordinate, collider, and SDF particle demos, a minimal Rust-native Android APK smoke test, a camera-driven Quest OpenXR/Vulkan custom-layer example with optional screen streaming, OSC ingress, diagnostic HUD, environment-depth diagnostics, scene-owned real-time depth particle mapping, and live hand-mesh particle visualization, plus a broker APK proof for localhost WebSocket, optional LSL forwarding, OSC drive events, bounded media probes, H.264 live stereo projection diagnostics, and source-only broker/eye stream contracts. Native adapters remain narrow and example-owned while the public utility surface stabilizes.

Serialization

Serde is opt-in and schemas are generated to ignored locations first.

Adapters

Native integrations stay behind explicit features or separate adapter crates.

Passthrough

Native compositor passthrough layer purposes, style modes, and source boundaries.

Visual Source Taxonomy

Keep raw camera, environment depth, native passthrough, and operator display streams distinct.

Strobes

Safety-gated flicker descriptors, 120 Hz timing constraints, and photoepilepsy warnings.

OSC

Generic OSC packet, UDP, headset listener, and broker drive-event guidance.

Research Broker

Public-safe EDIA/RCAS comparison and broker bridge boundaries for research XR streams.

EDIA Track

Maintainer-facing notes on what is implemented, what is deferred, and where feedback is useful.

Debug HUD

Shared stereo-surface rendering and real-font atlas guidance for headset diagnostics.

Depth Particles

Scene-owned real-time environment-depth mapping, active visible free-space correction, and small opaque particle rendering.

Mesh Coordinates

Stable barycentric coordinates on dynamic triangle meshes, with same-surface and cross-surface neighborhoods.

Mesh Fixtures

Schema-backed public synthetic mesh manifests for topology, sampling, SDF/depth, particle, collider, and render-payload tests.

Mesh To SDF

Convert dynamic triangle meshes into packed SDF grids and attract public particles toward the surface.

Particle Rendering

Billboard construction modes, animation-mask lookup tradeoffs, trail snapshots, and Quest artifact isolation.

Mesh Colliders

Build collider-ready dynamic mesh surfaces and diagnostic visual shells from live hand or scan meshes.

Provenance

Public utility entries record broad source families without leaking private paths.

API Review

Current public surface decisions before hardware and native-adapter examples.

Companion

Shared catalog schema plus the Windows companion boundary for managed Quest tooling and optional media runtimes.

hzdb Provider

Optional Meta Quest CLI/MCP provider plan, safety gates, docs-first verification, Perfetto reports, and ADB fallback boundaries.

Launchers

Normal headset app launching, package visibility, and the boundary for ADB-launched shell helpers.

Rusty Kiosk

Data-only contracts for developer-home panels, launcher entries, settings shortcuts, helper state, and focus recovery.

Broker Clock

Broker-owned elapsed-realtime clock, query API, stream stamps, sync probes, and OpenXR comparison boundary.

Effect Diagnostics

Data-only pass graphs, diagnostic layer taps, and scalar comparison reports for app-owned visual pipelines.

OpenGL/OpenXR Stack

Implementation plan for SurfaceTexture/OES ingestion, public FBO pass graphs, projection diagnostics, and cross-lane scorecards.

Distribution And ADB

Split Store-style apps, developer builds, external ADB hosts, Wi-Fi ADB, and shell-helper positioning.

Tracking Access

Foreground OpenXR pose sampling, Android sensor limits, and why ADB is diagnostics rather than a fused tracking stream.

Streaming Diagnostics

Compare synthetic compositor, direct Camera2 projection, broker receive/decode, and broker live projected stereo, including the fast 0.75 renderer-parity profile.

Q2Q Agent Brief

Orient a Mac-based collaborator or agent toward a proper multi-headset Quest-to-Quest streaming setup.

Online Q2Q Streaming

Plan the path from laptop-loop diagnostics to LAN, mediated online, and WebRTC-based two-way stereo Quest streaming.

Projection Parity

Track direct and broker stereo projection parity targets without copying downstream app-specific behavior.

Coordinate Ledger

Name camera, texture, surface, OpenXR, depth, and screenshot spaces before raw projection work moves into blur.

Temporal Projection

Plan camera projection smoothing with target-vs-visual state, bounded applied motion, held-frame metrics, depth fallback, and optional space-warp probes.

Low Latency

Keep media control, packet framing, codec configuration, and renderer timing split for streaming diagnostics.

Makepad Android Build

Plan the move from Windows-oriented APK helper scripts toward cargo-makepad packaging and hotload-friendly iteration.

Examples

Run contract demos, mesh utility demos, the minimal APK smoke test, the Quest custom-layer example, or the broker sidecar proof.

Unity Example

Use the public Big Red Button Quest scene to compare direct Unity OSC/BLE input against broker-routed stream events.

Hand Mesh Particles

Drive stable particle coordinates from live HandMeshSnapshot frames without adding platform SDKs to core crates.