Designing One Experience, Building a Tool for Many

The University of Salento asked for a virtual museum inside its real one: a way to experience objects visitors can never touch. Radium watch dials, uranium glass, radioactive lightning rods, all out of reach for good reason. Design problems and technical problems carried the same weight, and AI played a central role in both. I tested several tools, then asked a simpler question: why not build custom software for experiences like this one?

Room 1, the model for the ten that follow, was presented at the Festival della Scienza in Genoa and is now online. Concept, design, 3D and the software were mine.

This article follows the same split: first the experience, then the tool that builds it.

Six Questions Before a Single Model

Every later decision answers one of these.

With those six questions answered, I started producing quick concepts with AI and presented them in a pitch, to agree on a direction with the University before building anything.

Custom Paths, Not a Boring Slideshow

There is no kids' mode and no expert mode. Every visitor walks the same path and decides how deep to go, so each visit becomes a path of its own. Names and groups of objects were rewritten for a general audience before anything was modeled.

The 3D has two jobs. It informs: objects you cannot touch rise from their pedestal and turn in your hand. And it orients: the room is circular, every group visible from the center, so the space is its own map.

Interaction Design

The interface was built around one idea: simple enough for anyone, complete enough to never leave you stuck, and shaped around moving through space. Four principles came before any button.

Marie, the Guide

Marie, named after Marie Curie, welcomes visitors, introduces the museum and answers their questions, with answers written together with the curators. She is built in Character Creator, speaks with a voice generated from the text and a lip sync derived from it, and moves through a mixer inside the tool.

The next step is already planned: Marie will become a conversational guide, able to answer free questions from the museum's own knowledge, with the same face and the same voice.

User Tests

The tests were observational: while the scene was being built, I watched people use it without guiding them. Fifteen visitors of different ages, eight baby boomers, five millennials and Gen Z, two Gen Alpha, on desktop and phone. Two behaviors changed the design.

One Tool to Build Them All

Technique was there to serve the design, not the other way around. I evaluated several solutions, mainly Unity and Spline, but the experience needed a lean system, built on three.js, easy to edit and made to measure. So I designed an editor that runs in the browser with the museum inside it, speaking the museum's language: rooms, exhibits, shots, the guide. It is internal, but designed as a product for small teams building exhibitions and showrooms.

An Architecture Built to Scale

Rules and settings were built custom, for exactly what the museum needed. Desktop is the base, Totem and Mobile are its differences, and every difference is blue: the only other color with a meaning in the editor is bronze, for act here. Cameras follow rules set once for the whole museum, so the second room never starts from zero.

The Blue Rule

A setting exists on every output or none.

Any change from Desktop, even position and order, shows in blue: in the field, in the list and in a counter next to the output.

Mobile settings used to live in nine places. Now one number answers "what changes on the phone?".

Performance by Rule

Level of detail by context, computed on load with no extra files.

Models compressed as far as they go, light baked into the textures, textures compressed on arrival in the right format for their role.

On the phone, same view: draw calls from 828 to 297, triangles from 1.07 million to about 380,000.

Hands on the Engine

A true planar reflection redrawn only when something changes. Lights written into shaders that touch only the guide or the open object. Hair as one mesh in two passes. Glass that refracts only up close. Every effect measured on the real machine.

Words, Written Once

The same text lives on a 3D label, in a card and in Marie's voice. It is written once in the editor, and from there the tool orchestrates the rest through APIs: it calls ElevenLabs to generate the voice, gets the audio back already linked to the right moment, derives the lip sync from the text and updates every label and card that shares it.

Directing the Agent

Around 46,000 lines, 279 commits in a month, written by an AI agent, Claude Code. My work was the architecture: the vocabulary, the rules, the interface approved as a mockup before any code, every result verified on the real screen.

Claude did not stop at building the tool. It also uses it: some of the manual operations inside the editor are run by Claude directly, as if through a connector. So AI works on every side of the software. An agent builds it, an agent operates it, and the tool in turn orchestrates AI services like ElevenLabs to produce the museum.

From a Museum to a Showroom

This was a stimulating project, and a complete one: design, technology and craft in the same piece of work. Testing showed that exploration has to be defended at every step, because the slideshow kept trying to come back through the buttons. The tool is what keeps that discipline cheap, room after room.

Swap radioactive objects for products, and the same layer of control runs a brand showroom. Ten more rooms come first, then a tablet output and VR as a new way into the same spaces. And the other natural evolution is Marie herself, growing into a conversational guide.