Suwappu

Suwappu is a set of small stackable two-piece woodland-animal toys whose tops and bottoms can be swapped between species. The toys come to life through a phone camera held above them. Films and stories play out in augmented space around each character, while the toys themselves stay physical, on the table or in the hand. Made at BERG for Dentsu London, 2011.

Introducing Suwappu. Directed by Timo Arnall and Jack Schulze, BERG / Dentsu, April 2011.

A small brown plastic toy figure with antlers stands on a tabletop, surrounded by augmented-reality grass tufts in pale yellow-green and a pink rolled log. Soft white particles drift in the air. The toy is real; the world around it is rendered. — Deer in his augmented forest. The toy is plastic; everything else is rendered around it through the phone’s camera.

Five characters made up the cast. Badger lived in a harsh, troubled world. Deer in a forest utopia. Fox in an urban garden. Tuna in a paddling pool of nicely rendered water. Each toy was the same physical object — a stacking pair of moulded plastic blocks with a face printed on top — and each carried its own world inside it, surfaced by a camera held above. Pop the head off one and put it on another and the species changed; the world changed too.

Two Suwappu toys side by side on a desk: a blue Tuna with a fish-shaped face standing in a circular augmented pool of pale water, and a brown Deer with antlers standing on a series of augmented paving stones beneath a virtual streetlamp. A small speech bubble above Deer reads 'Badger needs your help, Tuna'. — Tuna in his pool, Deer on his street. The speech bubble is part of the augmented layer; the toys hold a script between them.

The toys themselves were the markers. Each character’s printed pattern was distinct enough that the camera could identify which species it was looking at and its rough orientation, without needing a fiducial barcode or QR code spoiling the design. That meant the toy could be a toy first — handled, given to a child, swapped piece-for-piece with another toy — and the augmented layer could attach itself when a phone happened to be nearby. The films, the speech bubbles, the rendered habitats: all of that lived in software, attached to the toys via the phone.

The pattern recognition was a hand-tuned implementation written specifically for the printed forms, well before ARKit or ARCore existed; today the same idea would be a few lines of CoreML. At the time it was a small piece of computer-vision craft built on OpenCV and a custom training set.

Production

Directed by Jack Schulze and me. Character design by Camille Bozzini. The toy industrial design and the moulds were developed at BERG. The films were shot in the studio with the toys on a tabletop and the augmented elements composited in After Effects against the camera’s plate. The pattern-recognition software ran on iPhone hardware of the time. Dentsu London commissioned and produced.

Selected coverage

Suwappu was acquired into MoMA’s Talk to Me exhibition (24 July to 7 November 2011), curated by Paola Antonelli, alongside four other works of mine: Immaterials: Ghost in the Field, Nearness, Media Surfaces, and The Journey. Fast Company wrote about it. Original launch posts on BERG’s blog and an app-prototype follow-up describe the project further.

Suwappu was made at BERG (London) by Jack Schulze and Timo Arnall, with character design by Camille Bozzini, for Dentsu London. With thanks to the BERG studio and to Dentsu’s creative team.