Why POSOMAT Magic Move, our interaction layer for Unreal Engine 5, keeps all of its AI in the room. And what BASHBALL shows about trust in front of an LED wall.
People walk up to a large LED wall, and something happens straight away. A figure made of light lifts its arm at the same moment they do. It mirrors their own movements, so nobody needs an explanation. It is dead easy. Someone waves. The figure waves back. A ball comes flying in, and the game has already started.
It looks like magic. Technically, it is a stereo camera, a skeleton and a lot of physics. And one decision that nobody in front of the wall ever sees: where the AI runs.
A controller that is already there
BASHBALL is an art and technology project. You step in front of an LED wall or a projection, a camera captures your whole body in real time, and your avatar mirrors you. The instruction on screen is four words long: "Move & Catch the Balls!" People jump, wave, kick and punch. Alone, or with several people at once.
Children got it much faster than adults, and they played it more intensely. The adults first tested it carefully with one hand. By then the kids had already gone wild.
Underneath BASHBALL sits POSOMAT Magic Move, the interaction layer for Unreal Engine 5 that we developed at POSOMAT. Stereoscopic AI depth sensing scans the space, isolates the people in it and infers their skeletons on site. The platform can follow the skeletal data of up to 50 people at the same time. It needs neither markers nor wearables, and nobody has to stand in a T-pose to calibrate.
The skeleton coordinates then flow straight into Unreal Engine 5: into vector fields, physics assets and Niagara particle systems. Every collision is calculated in real time. The result renders from 1080p up to 4K at 60 frames per second.
The best controller in a public space is the one people bring with them anyway. Their body.

The budget is one frame
Magic Move responds in under one frame.
At 60 frames per second, one frame lasts about 16.7 milliseconds. In that slice of time, the camera has to see you, the AI has to find your skeleton, and the engine has to move your avatar and calculate what happens to the ball. If the answer comes noticeably later, the figure on the wall stops being you and becomes a puppet with a delay.
Any detour through a data centre would have to fit into the same 16.7 milliseconds, network included. So the first reason Magic Move runs locally is not ideology. It is physics. All processing happens on a single computer on site, with NVIDIA CUDA graphics and Unreal Engine 5. It needs no internet connection for processing.
Privacy by design is a hardware decision
The second reason is privacy.
A camera in a public space is a delicate thing. People are rightly sceptical when a screen looks back at them. With Magic Move, the answer to "where does my image go?" is short: nowhere. There is no cloud connection. The camera data is turned into movement on the spot and stays in the room. There is no sign-up, no app and no user profile, so no personal data is collected in the first place.
This privacy is a property of the architecture, written into the setup rather than into a policy. A system without an upload cannot leak an upload. Nobody has to trust a server they will never see.
For museums, schools, festivals and brands, this changes the conversation. Nobody has to trade their data for a round of play. The installation asks for nothing except presence.
Visitors were not really interested in the technical details. The organisers were. What mattered most to them was that everything runs locally: high-quality rendering and AI together, on one PC.
Sand, sun and a queue that never ends
Lab demos are polite. Public spaces are not.
BASHBALL has already been used at a beach event, with a constant stream of new visitors. Every new visitor is a body the system has never seen before, and there is no second attempt at a first impression.
The wall stood in a tent. The wind still blew in a lot of sand and dust. Then the weather, and above all the light: sun, then none, bright, then dark. The system ran stably throughout.

This is where local processing pays off a second time. A system that runs on one machine in the room does not depend on a network connection on site. The POSOMAT AI CORTEX, the hardware appliance built for this platform, boots headless: power on, and it goes straight into the interactive state and auto-calibrates the sensor field. No technician has to start a program. That sounds like a detail. On site, it is the difference between an installation and a support ticket.
It is often said that the first three seconds decide whether someone steps in or walks past. In my experience it is even faster. Everyone gets drawn in by their own movement and tries it out. Some only raise a hand in passing to see how the wall reacts. An avatar that answers instantly explains more than any tutorial.
Built to be taken apart
My doctoral thesis was titled Hacking as Playful Strategy for Designing Artistic Games.
Magic Move is built so that it can be taken apart and recombined. It serves as a sandbox for artistic research. Its architecture is node-based and decoupled, so new generative AI models, autonomous visual behaviours and multimodal sensor input can be plugged in and tested quickly.
BASHBALL is the first game on top of it. The flying objects can be swapped for 3D models of products, from sneakers to beverage cans. The interaction stays the same: step in, move, play.
So where should the AI live?
We are putting cameras and AI into more and more public spaces. Most of the debate is about what these systems can recognise. The more interesting question is simpler. Where does the thinking happen, and what leaves the room afterwards?
When an AI can see your body, should it run anywhere other than right there, in the room with you? For Magic Move, the answer is no. The thinking happens next to the wall, and nothing leaves.
If you want to see Magic Move or BASHBALL in your museum, at your festival or on your stand, you will find the installations on posomat.com and the background to my work on elorx.com.
