Mesmer Prism

Mixed-ability XR research

Shared bodies,
inspectable systems

This Duisburg-facing pitch proposes Mixed-Ability Human-Swarm Interaction: disabled and non-disabled collaborators co-create a shared, weakly bounded swarm body whose inputs, dynamics, privacy choices, pacing, authorship, and repair remain visible enough to negotiate.

Three-step method Full HSI article
Portrait of Till Holzapfel
Till Holzapfel. XR / VR developer and researcher working across embodied interaction, biosignals, swarm-like bodies, and open research tooling.

Orientation

The project before the biography.

The central proposal is not a new avatar effect. It is a method for studying how mixed-ability groups negotiate shared agency through a mutable swarm body: a body made from particles, fields, auras, roles, and rules rather than a single humanoid shell.

My route matters because I can build the instrument: XR systems, biosignal routing, Quest tooling, open Rust prototypes, and weakly bounded embodiment vocabularies that can be inspected instead of treated as sealed effects.

Mixed-ability HSI

Human-swarm interaction becomes an access question when several people with different energy budgets, devices, communication modes, and visibility preferences share one many-part body.

Abstract bodies are usable bodies

The project builds on work showing that luminous bodies, particle auras, abstract creatures, and telepresent touch metaphors can support ownership, co-presence, emotion, relation, and social meaning without humanoid realism.

Open instruments

Rusty Morphospace, Rusty XR, and Polar H10 tooling give the project a current public base for sensor and package lanes, Quest validation, and separated contracts. The planned HSI layer uses that base for routing roles, mappings, diagnostics, consent settings, and participant inspection across XR prototypes.

Duisburg fit

The Inclusive Technology and Collective Engagement group frames emerging technologies through accessibility, co-design, marginalized experience, and mutual engagement. This proposal turns that agenda toward shared swarm bodies and inspectable mediation.

Thesis

Access is a relation inside the group.

Ability-based design argues that interfaces should adapt to what people can and want to do; interdependence work adds that access is often negotiated between people, tools, environments, and care relations (Wobbrock et al., 2011; Bennett et al., 2018).

How can mixed-ability collaborators co-create a swarm body that is legible enough to coordinate, expressive enough to support connectedness, and inspectable enough to repair?

Swarm concepts

Aesthetic swarms and technical swarms are related, not identical.

A swarm can mean two different things. In robotics, swarm behavior usually comes from many individual units interacting through local rules, sensing, and physical constraints. In aesthetics, multiplicity can become a readable body through Gestalt grouping, common fate, rhythm, density, texture, light, and shared motion. Mixed-Ability HSI needs both vocabularies, but it should not collapse them.

Aesthetic swarm body

The first studies can use particles, light, fields, boids, soft boundaries, and physics-like animation to make co-presence, privacy, authorship, refusal, and repair visible. The swarm does not need to be physically possible to be socially legible (Gilland, 2009; Glowacki, 2024).

Technical swarm body

When the work moves toward robots, furniture, tangible swarms, or modular assembly, the same design words become harder constraints: mass, motors, batteries, localization error, latency, collision, surfaces, safety, and failure recovery (Kolling et al., 2016; Kegeleirs and Birattari, 2025).

Physics changes role

In the aesthetic track, gravity, drag, turbulence, elasticity, viscosity, resistance, and collision cues make transformation readable. In robotics, those same forces become load cases, platform limits, and safety obligations.

What should transfer

The bridge is a constraint test: which mappings remain useful when agency, safety, visibility, pacing, and repair must survive a change of medium from visual body to tangible or robotic system?

Research arc

From connectedness to goal-directed dynamics.

The public-facing route is deliberately staged. It starts with low-pressure exploration and connectedness, adds game-oriented collaboration only after the group understands its own mappings, and treats robotics as a later translation gate rather than an early promise.

  1. Access practices first

    Participants name existing devices, rhythms, supports, fatigue patterns, communication habits, and off-limits signals before sensors or mappings are proposed.

  2. Exploration and connectedness

    A low-demand swarm body lets the group test glow, density, rhythm, gathering, hiding, shielding, coalescence, rest, and repair before task pressure.

  3. Abstract-body precedents

    Glowacki's light-body work, Desnoyers-Stewart's particle-body lineage, Emotional Beasts, Galea, and Crip Sensorama show different routes into non-humanoid but socially meaningful bodies.

  4. Game-oriented collaboration

    Structured play introduces roles, goals, asymmetric contribution, conflict, resource limits, recovery, and shared consequences without reducing access to productivity.

  5. Inspectable stack

    Rusty Morphospace connects the current public contracts and validation lanes to a planned HSI-facing stack for channels, bindings, dynamic targets, feedback, consent, logs, provenance, and retirement.

  6. Translation gate

    The later question is which aesthetic, social, and control principles survive physical consequence in robotics, assembly, adaptive furniture, or teleoperated contexts.

  7. One starting context

    Homebound and energy-limited participation makes pacing, sensory load, asynchronous contribution, and visibility choices concrete design constraints.

  8. Contribution

    The outcome is a prototype family, a participatory method, and design knowledge for mutual engagement through expressive, negotiable shared bodies.

Research route

Three steps, with elicitation before step one.

Before anyone tries to optimize control, the project has to ask what the group is controlling and why. A mouth gesture, breath estimate, switch, gaze event, controller pose, or biosignal can change color, density, attraction, boundary softness, a goal, a field, a safety veto, or only a private cue. Those are not equivalent mappings.

The first study should therefore begin with access-practice elicitation, then move through three research steps. Luminous and weakly bounded VR bodies can support connectedness and coalescence, while swarm-body work shows that size, density, distribution, and correspondence change embodiment and agency (Glowacki et al., 2022; Glowacki, 2024; Ichihashi et al., 2024).

This matches a group agenda where emerging technology is evaluated through expression, communication, participation, trust, and real social context (RC Trust, 2025).

1

Exploration and connectedness

Participatory workshops discover input channels, comfort boundaries, privacy choices, visibility preferences, expressive mappings, and shared-body states.

2

Game-oriented collaboration

Goals, roles, timing, resource limits, and constraints tune coordination, asymmetric contribution, recovery, trust, and shared authorship.

3

Robotics translation gate

Tangible swarms, adaptive furniture, robotic avatars, assembly systems, and teleoperation become later tests of which mappings survive physical constraint.

One possible participant context

Low-demand swarm worlds for home-mediated participation.

ME/CFS, Long COVID, severe fatigue, and prolonged social withdrawal are not the same thing. They do, however, make ordinary co-present participation difficult in ways that matter for interaction design: pacing, exertion risk, travel, sensory load, identity exposure, and the need for asynchronous contribution (CDC, 2024; NICE, 2021; Kato et al., 2020).

Constraints

Low energy and pacing Travel barriers Sensory load Visibility choice

Design responses

Desktop first Saved states Short visits Optional VR

The environment would not treat loneliness or social withdrawal. It would study how shared presence, pacing, privacy, and authorship can be designed with people whose access to social worlds is already mediated by home and network. Remote VR methods and social VR disability work provide useful precedents, but also show why recruitment, disclosure, sensory load, and participation modes need careful handling (Mottelson et al., 2021; Zhang et al., 2022; Gualano et al., 2024).

Research instrument

Open tooling makes the mapping stack contestable.

Rusty Morphospace is not the research question. It is the implementation layer that can make the mediation inspectable: what input enters the system, what scale it controls, what dynamic changes, who can see that contribution, which logs are kept, how consent is represented, and when a mapping can be retired. In RC Trust terms, the mediation should be understandable and contestable, not merely adaptive (RC Trust).

The distinction matters. The current Morphospace base is public module boundaries, package lanes, a bioelectric Matter/Optics teaching model, and a bounded Quest Makepad/Hostess validation route. The HSI layer planned on top is participant-facing: mapping authoring, consent, provenance, replay, version comparison, accessible facilitator views, and adapter swaps.

That matters because input diversity is not enough. Mixed-ability HSI needs a full mapping from social dynamics to technological affordances: access practice, human channel, binding granularity, dynamic target, feedback, social contract, provenance, and retirement.

Access practice and channel

gesture, text, switch timing, breath, gaze, EMG, EOG, ECG, controller pose, assisted communication, rest strategy

Binding and dynamic target

particle, sub-swarm, whole body, field, SDF seed, color, density, attraction, boundary, goal, role, veto

Feedback and social contract

private cue, public trace, sound, caption, haptic, log, visibility, consent, authorship, rest, replay, remap

Implementation split

current contracts in Lattice, Manifold, Matter, and Optics; planned GUI, Studio, Hostess, Quest, and Makepad surfaces for authoring, consent, validation, and deployment

Boundaries

What the system can test, and what it cannot claim.

Swarm control

Human-swarm interaction is a model for influencing collective tendencies, not micromanaging every unit. Relevant controls include behavior selection, parameter setting, environmental influence, leaders, sub-swarms, attractors, and mixed-granularity commands (Kolling et al., 2016; Brown et al., 2014).

Robotics

XR is the first design laboratory. Physical robot swarms, adaptive furniture, and assembly systems add safety, localization, latency, object weight, actuation, maintenance, and sim-to-real constraints (Kegeleirs and Birattari, 2025; Petersen et al., 2019).

Physics

In the visual phase, physics can be an enabling scaffold for felt materiality. In the robotics phase, physics becomes a stricter accountability regime: mass, friction, impact, torque, clearance, power, tracking error, and failure recovery.

AI

AI can suggest mappings, support turn-taking, or summarize interaction histories. It should remain transparent, reversible, and subordinate to participant-defined expression (Zhou et al., 2025).

Bioelectricity

Bioelectric-inspired graph fields are optional model vocabulary for coupling, repair, and local-to-global patterning, not biological prediction. DiffeoMorph belongs even farther out: a possible agentic dynamics family for learned target-forming swarms, not a current access method (Levin, 2021; Pahng et al., 2025/2026).

Connectedness

The study can examine co-presence, authorship, and shared agency. It should not promise treatment for loneliness, fatigue, or social withdrawal.

Deep dives

Where the longer arguments live.

The Duisburg page carries the compact pitch. These pages hold the deeper source work, implementation context, and translation lanes that should not crowd the custom page.

Pain and body representation

Pain Translation

A narrower companion lane on boundary precision, avatar materiality, protective fields, ownership, agency, and pain-related hypotheses without treatment claims.

Essay Pain sources

References

Source anchors.

These sources separate group context, mixed-ability access theory, plasmatic and swarm embodiment, open input mapping, low-demand participation, physics aesthetics, and robotics translation. The same source set is available as Markdown, text, BibTeX, and CSL JSON.

Group, job, and communication context

Mixed-ability access and distributed embodiment

Plasmatic, abstract, and expressive bodies

Input mapping, HSI control, and robotics horizon

Home-mediated participation and disclosure

Bioelectric-inspired model family

Public project context

Page exports

Markdown Text BibTeX CSL JSON