The first study
The initial experiment compares researcher-configured mappings with mappings that participants inspect, revise, and co-author.
Small mixed-ability groups would work with a swarm-inspired shared body: a many-part form whose behavior can be rendered visually, sonically, tactilely, or textually. The first implementation can be projection-led because projection makes collective change easy to inspect in a shared room. It must also expose the same semantic events through nonvisual and non-HMD routes. A failure in one rendering is evidence about that rendering, not proof that the underlying mapping is inaccessible.
Participants
Small groups of disabled and nondisabled adults, recruited with pre-session access planning. Participants may bring existing devices, communication methods, support people, or established access practices. Diagnosis disclosure is not a condition of participation.
Prototype
One semantic swarm state with synchronized projection, text, spatial audio, and optional tactile or tangible feedback. HMD use is optional. Each contribution has a visible or otherwise inspectable scope, attribution setting, and undo path.
Task
The group guides the shared body through a cooperative sequence that requires gathering, dividing, holding, yielding, and recovering. The task is simple enough that interaction failures remain interpretable.
Comparison
Each group encounters a researcher-configured condition and a co-authored condition. Order is counterbalanced. The study compares what the system records with what participants say they perceived, intended, and considered fair.
Research questions
- Causal legibility. Can participants identify how their own and other people’s actions altered the shared state?
- Negotiability. Can participants pause, change, conceal, veto, or repair a mapping without relying on the facilitator to interpret the system for them?
- Agency and fairness. How does co-authorship change felt agency, authorship, role satisfaction, and judgments about fair influence?
- Access cost. How do mappings differ in fatigue, sensory demand, calibration effort, privacy cost, and willingness to use them again?
Measures and experiential method
The primary outcomes are causal-attribution accuracy, successful repair, time and steps required to repair, perceived fairness, and access cost. Secondary outcomes include felt agency, authorship, privacy comfort, sensory load, connectedness, role satisfaction, and willingness to reuse a mapping. System logs record pauses, remappings, rejected signals, fallback-input use, facilitator interventions, and differences between the logged account and participant accounts.
Global ratings such as “I felt in control” are too coarse for an emergent system. The study therefore uses event marking and replay-assisted individual interviews before group discussion. A participant can return to the moment when the swarm began to split, a contribution became public, or a command was changed for safety, then describe what they first noticed, expected, attempted, and attributed to themselves or others. Conflicting accounts are evidence about the interaction, not noise to be averaged away.
Pause and veto
- A safety stop immediately reduces motion, sound, and demand.
- Personal withdrawal removes or hides one participant’s contribution.
- A discussion pause holds the shared state while the group inspects it.
- A group veto prevents a proposed transition under agreed rules.
Access operations
- Rest and return are ordinary participation states, not dropout.
- Every consequential command has confirmation, correction, and undo.
- Public feedback and private feedback can be configured separately.
- Venue, device, or facilitator barriers are recorded as system failures.
The Mixed-Ability HSI Mapping Model
The central design object is the contract that turns an access practice into an effect on a shared body.
A mapping is more than an input binding. It specifies what is sensed, what the action can affect, how the result returns to the participant, who else can perceive it, how consent is maintained, who receives credit, and how the mapping can be revised or retired. The shared state sits between input and rendering so that movement, speech, AAC, switches, text, gaze, breath, direct manipulation, or facilitated input can reach the same semantic commands without pretending to be equivalent bodily acts.
Operational terms
- Shared semantic state
- The medium-independent collective variables and event history.
- Shared swarm body
- A many-part rendering that a group can perceive as one collective, several subgroups, or a surrounding field.
- Causal influence
- What a participant’s action actually changes in the system.
- Felt agency
- Whether the resulting change is experienced as mine, ours, negotiated, or outside my control.
- Authorship
- Who designed or substantially revised the mapping and its meaning.
- Responsibility
- Who is accountable for a consequential outcome, including designers and institutions.
- Mapping
- The complete social and technical contract linking an access practice to a collective effect.
- Repair
- A technical, representational, interactional, social, or morphogenetic recovery from breakdown.
These definitions prevent several useful but different observations from collapsing into “embodiment.” Participants may perceive one coherent body without identifying with it. They may influence it without feeling ownership. They may feel connected to the group while rejecting bodily fusion. The study keeps perceptual unity, identification, causal influence, felt agency, ownership, authorship, and connectedness separate.
Four system statuses
The word swarm also needs discipline. A swarm-inspired visual body is a rendered many-part form and may be centrally computed. A multi-agent interface has several individually addressable elements. A decentralized robot swarm produces collective behavior through local interaction and partial autonomy. A morphogenetic system develops or repairs form through a generative process. Evidence from one category does not automatically validate the others.
A repair typology
Technical repair
Recalibrate a sensor, restore an output, or change a failed access route.
Representational repair
Correct a misunderstanding about what a signal means or controls.
Interactional repair
Recover from missed timing, conflicting commands, or an interrupted turn.
Social repair
Address unfair influence, unwanted disclosure, or an ignored refusal.
Morphogenetic repair
Let the collective restore or reorganize form after disruption.
What direct accessibility research establishes
Existing studies do not amount to an accessible decentralized swarm, but they already impose concrete design constraints.
BotMap
Visually impaired participants used mobile robots as tactile landmarks while panning and zooming through maps. The system was usable, but transformations demanded training and could be cognitively difficult. Reconfiguration therefore needs stable anchors, perceptible transitions, and easy return to a known state (Ducasse et al., 2018).
RoboGraphics
A static tactile overlay supplied a reference frame while mobile robots carried changing information. Seven participants with varying levels of vision explored the dynamic graphics. The useful principle is selective motion: keep invariants stable and move only what has changed (Guinness et al., 2019).
Robots for Inclusive Play
Co-design with visually impaired and sighted children and educators produced an inclusive game organized around shared goals, closely coupled roles, and interaction symmetry. Accessible control is insufficient if the social role remains peripheral (Metatla et al., 2020).
The Robot Made Us Hear Each Other
In a study with 78 children, including 26 with visual impairments, a directive robot balanced participation more strongly, while a less directive strategy made children feel more heard. Observable equality and experienced inclusion are not the same outcome (Neto et al., 2023).
TACTOPI
Ten mixed-visual-ability dyads played in a shared multisensory environment. Tangible, sonic, narrative, and robotic elements supported one collaborative activity without requiring identical sensory access. A shared event can have different but coordinated renderings (Pires et al., 2023).
Shape-changing emotion
Fifty children, including 26 with visual impairments, associated changing tactile forms with emotions. Several associations differed with visual experience. A morphing interface has no universal expressive vocabulary; meanings must be tested across sensory histories (Neto et al., 2024).
Two complementary interface lineages
Tangent: the swarm as interface material
Zooids established small robots as programmable physical interface elements. UbiSwarm treated collective motion as display; SwarmHaptics distributed touch across several agents; later work studied user-defined control, motion legibility, programmable fidgeting, ADHD co-design, and affective interaction (Tangent Lab; Le Goc et al., 2016; Kim and Follmer, 2019; Pulatova and Kim, 2024).
iSpace: the swarm as relational body
John Desnoyers-Stewart’s iSpace work shows how projection, tracked bodies, particle exchange, constellation-like avatars, pseudo-haptic contact, and biosignal-linked form can produce abstract shared embodiment. These projects establish a mixed-reality vocabulary for relation, not mixed-ability validation (iSpace profile; Desnoyers-Stewart et al., 2020; Desnoyers-Stewart et al., 2023).
Tangent contributes the physical interaction grammar; iSpace contributes the relational and abstract-body grammar. The proposed study adds the missing mixed-ability layer: modality-independent commands, participant-authored mappings, inspectable asymmetry, private as well as public feedback, and repair.
When many parts become a body
A swarm can appear as many separate agents, one macro-agent, or a field that surrounds action. The transition is not determined by robot count alone. Synchrony, spacing, velocity, sound, occlusion, task, and bodily proximity all matter. A participant may move from “I control those elements” to “we are shaping this” and then to “the system moved on its own” during one short event.
The term body is therefore a question, not a result. The first study asks whether the group perceives unity, identifies with the collective, experiences influence, attributes ownership, or feels connected. These outcomes can diverge. Swarm Body demonstrates why a changing population of robots may become incorporated into bodily action, while the iSpace works show that a radically abstract form can remain relationally legible. Neither makes incorporation universal.
Morphogenesis adds a further distinction. In Slavkov et al.’s 300-robot experiment, local interactions generated adaptable shapes that recovered from damage. That work establishes an engineering process. It does not show how mixed-ability participants experience damage, continuity, or repair. Biological morphogenesis likewise supplies concepts for local-to-global formation and regeneration, not direct evidence for human-swarm experience.
Access is part of the swarm architecture
Equal control is not always fair control. One participant may shape rhythm continuously, another may contribute occasional text, and another may hold a high-authority veto. The relevant questions are whether each role is meaningful, chosen, understandable, revisable, and protected from being silently overruled by a faster or more continuous signal.
This shifts accessibility from a menu of alternative inputs to the distribution of authority. A control is operationally accessible only if the person can configure, initiate, verify, correct, pause, undo, and recover through an available route. Public systems must also remain accessible to people who do not operate them, including bystanders, support workers, maintainers, and people who want no interaction. The W3C XR Accessibility User Requirements provide a useful baseline for alternatives, customization, and avoiding unnecessary sensory or motor assumptions.
Provenance and collective data
Every saved mapping needs a version, origin, access cost, scope, feedback routes, visibility setting, safety bounds, consent state, reuse status, and retirement record. Raw voice, video, motion, gaze, physiology, or AAC data should be stored only when the research question requires it. Participants need a way to inspect and contest the system’s account of what happened.
Group data creates a harder problem: one person may later withdraw from an event that also contains other people’s contributions. The protocol must decide in advance whether to delete raw media, retain only an agreed derived annotation, re-consent the group, or exclude the event from public reuse. A collectively authored mapping cannot be treated as the institution’s unencumbered property.
How the interaction can fail
- A visual transformation can preserve the system state while destroying a tactile mental map.
- A continuous high-frequency input can appear more valuable than waiting, restraint, or refusal.
- A participant can feel obliged to disclose pain or fatigue to justify a pause.
- A private bodily signal can become visible to the group through an expressive effect.
- Synchrony can feel coercive even when the resulting form looks harmonious.
- A repair process can remain unfair if it is controlled by the person or institution that caused the imbalance.
These failures are not edge cases to remove from the report. They are tests of whether the mapping model exposes enough of the interaction for the group to respond.
Claims, limits, and translation
The research can currently claim
- A defined mapping model for mixed-ability shared agency.
- A multimodal architecture in which no single rendering is normative.
- A bounded comparison between configured and co-authored mappings.
- Operational measures for legibility, repair, fairness, and access cost.
It cannot yet claim
- Therapeutic or diagnostic effect.
- Validated control of a decentralized physical swarm.
- Universal meanings for shape, motion, touch, or synchrony.
- Safe deployment in workplaces, public space, or clinical settings.
Later work can render selected collective variables through a Tangent-like physical interface swarm, then compare the same logged event across projection, sound, touch, tangible tokens, and physical robots. The interface swarm need not reproduce every virtual particle, and it need not be the working swarm. A small number of physical agents may instead provide stable landmarks, direction, subgroup summaries, or a tactile warning that the shared state has fragmented.
Physical morphogenesis, city-scale logistics, and living or biohybrid systems belong beyond that gate. Their material risks, maintenance demands, and institutional responsibilities cannot be inferred from a reversible room-scale prototype. The useful bridge is narrower: preserve the semantic commands, event history, consent, attribution, and repair rights while the substrate changes.