The Immaterials project

Posted on Sep 4, 2013 in Interaction design, Research

RFID-Touch-Project

The Immaterials project is concerned with the increasing invisibility of interfaces and infrastructures. The systems we interact with everyday such as WiFi and 3G networks have a profound impact on how we experience the world. As Adam Greenfield says:

the complex technologies the networked city relies upon to produce its effects remain distressingly opaque, even to those exposed to them on a daily basis. [...] it’s hard to be appropriately critical and to make sound choices in a world where we don’t understand the objects around us.

And as James Bridle has eloquently and disturbingly observed:

Those who cannot perceive the network cannot act effectively within it, and are powerless.

The project set out to expose some of the phenomena and mechanisms of technological infrastructures through visual, photographic, narrative, animated and cinematic techniques. Over the last five years I have worked with Einar Sneve Martinussen, Jørn Knutsen, Jack Schulze and Matt Jones towards a body of work that is now brought together in an exhibition for the first time.

From 2004–2008 I speculated about the ways in which wireless interactions inhabited physical space, through my work on a Graphic language for touch, and also through films such as Wireless in the world. Some of my students made beautiful but fictional speculations about the physical qualities of different kinds of radio.

Jack Schulze and I also made a short, playful film called Nearness about action at a distance. In the film, a series of simple reactions are set off by immaterial phenomena, such as radio waves, mobile networks, light, magnetism and wind.

In 2006 we ran a Touch workshop with BERG where we became concerned about the invisibility of RFID technology, and the effect that had on our ability to design with it. We found it extraordinary that a technology that was defined as a proximity or ‘touch’-based interface, was so opaque in terms of its physical, spatial, gestural materiality. How do we as designers make these materials visible, so we can have reflective conversations with them?

We developed Experiments in Field Drawing as a method of revealing, literally drawing, the physical presence of RFID interactions. We revealed these fields in a much richer, multi-dimensional way using photography, animation and light painting in the film Immaterials: Ghost in the Field.

Matt Jones coined the term immaterials to describe the project and gave a great talk about some ways of understanding the immaterials of interaction design. Matt and I also looked at machine vision, another phenomena that increasingly becomes a material for design in Robot Readable World.

In 2011 at AHO, as part of a research project called Yourban, we extended the investigations to WiFi, using similar light painting techniques we revealed the enormous scale and pervasiveness of ad-hoc WiFi networks in urban spaces in Immaterials: Light Painting WiFi.

Finally, over the last two years we’ve become increasingly interested in the Global Positioning System (GPS), that has become a central part of both the vision and the implementation of contemporary interfaces. We have built a series of Satellite Lamps that sense the presence of the 24 GPS satellites in orbit. The lamps change brightness according to the strength of GPS signals they receive, showing how the technology itself is messy and unpredictable, and revealing how GPS is a negotiation between radio waves, earth-orbit geometry and the urban environment.

Satellite Lamps is on display at Lighthouse, and a film will be available online later this year.

satellite_lamp_fjord

The visual languages that we’ve developed have ended up in advertising, on the BBC and Discovery Channel, and the techniques have been extended in research at MIT and CIID, and by many designers, enthusiasts and hackers. It’s exciting that both the subject and the methods are being taken up and used broadly by other people, and we’re looking forward to seeing more.

the truly pressing need is for translators: people capable of opening these occult systems up, demystifying them, explaining their implications to the people whose neighborhoods and choices and very lives are increasingly conditioned by them. — Adam Greenfield (2009)

The Immaterials project emerged from the humble preoccupations of a few designers dealing with some of the invisible, immaterial, intangible stuff we had in front of us. These small experiments led to larger and more visually and narratively communicative work. In the end what I think we’ve developed is an approach to technology that revolves around material exploration, explanation and communication. Because images and language, as well as materials, form our understandings of technology, Immaterials has shown how we can use ‘design and playful explorations to shape or stir the popular imagination’.

The exhibitions

All the Immaterials projects are on display at Lighthouse in Brighton from 5 September until 13 October 2013.

Satellite Lamps and Robot Readable World are on display at Dread in Amsterdam from 7 September until 24 November 2013.

No to NoUI

Posted on Mar 13, 2013 in Interaction design, Ubicomp

Invisible-design

The best design is invisible’ is the interaction design phrase of the moment. The images above are from my ever-expanding collection of quotes about how design and technology will ‘disappear’, become ‘invisible’ or how the ‘best interface is no interface’.

The Verge has recently given both Oliver Reichenstein and Golden Krishna a platform to talk about this. This has spawned manifestos, films, talks, books, #NoUI hashtags and some debates about what it might mean. I’ll call this cluster of things ‘invisible design’.

I agree with some of the reasons driving this movement; that design’s current infatuation with touchscreens is really problematic. I’ve spent the last eight years rallying against glowing rectangles, studying our obsession with screens and the ways in which this has become a cultural phenomena. In response I have been researching and inventing interfaces for taking interaction out from under the glass.

But I also take issue with much of the thinking for a few reasons that I’ll outline below.

1. Invisible design propagates the myth of immateriality

We already have plenty of thinking that celebrates the invisibility and seamlessness of technology. We are overloaded with childish mythologies like ‘the cloud’; a soft, fuzzy metaphor for enormous infrastructural projects of undersea cables and power-hungry data farms. This mythology can be harmful and is often just plain wrong. Networks go down, hard disks fail, sensors fail to sense, processors overheat and batteries die.

Computing systems are suffused through and through with the constraints of their materiality. – Jean-François Blanchette

Invisible design propogates the myth that technology will ‘disappear’ or ‘just get out of the way’ rather than addressing the qualities of interface technologies that can make them difficult or delightful.

Intentionally hiding the phenomena and materiality of interfaces, smoothing over the natural edges, seams and transitions that constitute all technical systems, entails a loss of understanding and agency for both designers and users of computing. Lack of understanding leads to uncertainty and folk-theories that hinder our ability to use technical systems, and clouds the critique of technological developments.

As systems increasingly record our personal activity and data, invisibility is exactly the wrong model.

By removing our knowledge of the glue that holds the systems that make up the infrastructure together, it becomes much more difficult, if not impossible, to begin to understand how we are constructed as subjects, what types of systems are brought into place (legal, technical, social, etc.) and where the possibilities for transformation exist. – Matt Ratto (2007)

In other words, as both users and designers of interface technology, we are disenfranchised by the concepts of invisibility and disappearance.

2. Invisible design falls into the natural/intuitive trap

The movement tells us to ‘embrace natural processes’ and talks about the ‘incredibly intuitive’ Mercedes car interface. This language is a trap (we should ban the use of natural and intuitive btw) that doesn’t give us any insight into how complex products might actually become simple or familiar.

Invisible design leads us towards the horrors of Reality Clippy. Does my refrigerator light really go off? Why was my car unlocked this morning? How did my phone go silent all of a sudden? Without highly legible systems for managing and understanding all of this ‘smartness’ we are going to get very lost and highly frustrated. The tricky business of push notifications and the Facebook privacy train wreck is just the tip of the iceberg.

The example of the Nest thermostat invisibly ‘learning’ your habits to control your home temperature is a good one. But the Nest has a highly visible interface that reassures you as to its status, tells you when it is learning, and a large dial for adjusting temperature. Beautiful, legible microinteractions. A Nest without these visual and direct manipulation interfaces would be useless, uncanny and frustrating. Nest wants UI.

The discussion around invisible design often talks about using sensors and tangible interfaces instead of visual interfaces. But these systems are not inherently simpler or more familiar. They have their own material qualities with edges and ‘grain’ that need to be understood and learnt. Their literal invisibility can cause confusion, even fear, and they often increase unpredictability and failure.

In our work with interface technologies such as RFID and computer vision, we’ve discovered that it takes a lot of work to make sense of the technologies as design materials. So it’s not useful to say that UI is ‘disappearing’ into sensing, algorithms and tangible interfaces, when we don’t fully understand them as UI yet.

3. Invisible design ignores interface culture

Interfaces are the dominant cultural form of our time. So much of contemporary culture takes place through interfaces and inside UI. Interfaces are part of cultural expression and participation, skeuomorphism is evidence that interfaces are more than chrome around content, and more than tools to solve problems. To declare interfaces ‘invisible’ is to deny them a cultural form or medium. Could we say ‘the best TV is no TV’, the ‘best typography is no typography’ or ‘the best buildings are no architecture’?

Much of our work at BERG is not just about solving problems, but about cultural invention:

We’re not interested in this idea of the invisible technology in a modernist sense. Tech won’t be visible but only if it’s embedded into the culture that it exists within. By foregrounding the culture, you background the technology. It’s the difference between grinding your way through menus on an old Nokia, trying to do something very simple, and inhabiting the bright bouncy bubbly universe of iOS. The technology is there, of course, but it’s effectively invisible as the culture is foregrounded.” – Jack Schulze (in Domus 965 / January 2013)

We should be able to simultaneously celebrate the fantastic explosion of diversity in UI, and develop healthy critique around the use of interfaces like touch screens. But by calling for UI to disappear altogether so that things can be more efficient, we remain in the same utilitarian and rational mindset that produces inert technological visions like this, rather than seeing interfaces as part of the cultural landscape.

4. Invisible design ignores design and technology history

The movement ignores at least thirty years of thinking in design and technology. A few examples:

Much of the recent invisible design discussions repeat the thinking in Jared Spool’s ‘Great Designs Should Be Experienced and Not Seen’ and Donald Norman’s ‘Invisible Computer. But a better reference point would be Don Norman’s earlier book, The design of everyday things, where he instead talks about the ‘problems caused by inadequate attention to visibility’ and supporting or managing our mental models of systems. We need a lot more thinking about our mental models of algorithms in particular.

Adam Greenfield has investigated the social and ethical issues around the development of ubiquitous computing systems, and is particularly concerned by its disappearance:

“Ubiquitous systems must contain provisions for immediate and transparent querying of their ownership, use, and capabilities. Everyware must, in other words, be self-disclosing. Whether such disclosures are made graphically, or otherwise, they ensure that you are empowered to make informed decisions as to the level of exposure you wish to entertain.” – Adam Greenfield (2006)

Some designers have talked about the actual qualities they want from ubiquitous computing interfaces, such as polite, pertinent and pretty:

“The vast quantities of information that personal informatics generate need not only to be clear and understandable to create legibility and literacy in this new world, but I’d argue in this first wave also seductive, in order to encourage play, trial and adoption” Matt Jones & Tom Coates (2008)

Matthew Chalmers has, more than anyone else, revealed the history of seamlessness. Seamlessness is ‘the deliberate “making invisible” of the variety of technical systems, artifacts, individuals and organizations that make up an information infrastructure. This work actively disguises the moments of transition and boundary crossing between these various parts in order to present a solid and seemingly coherent interface to users.’ (Ratto 2007). Although Mark Weiser is often thought of as an advocate of seamless systems, Chalmers found that:

Weiser describes seamlessness as a misleading or misguided concept. In his invited talks to UIST94 and USENIX95 he suggested that making things seamless amounts to making everything the same, reducing components, tools and systems to their ‘lowest common denominator’. He advocated seamful systems (with “beautiful seams”) as a goal. Around Xerox PARC, where many researchers worked on document tools, Weiser used an example of seamful integration of a paint tool and a text editor (Weiser, personal communication). He complained that seamless integration of such tools often meant that the user was forced to use only one of them. One tool would be chosen as primary and the others reduced and simplified to conform to it, or they would be crudely patched together with ugly seams. Seamfully integrated tools would maintain the unique characteristics of each tool, through transformations that retained their individual characteristics. This would let the user brush some characters with the paint tool in some artful way, then use the text editor to ‘search and replace’ some of the brushstroked characters, and then paint over the result with colour washes. Interaction would be seamless as the features of each tool were “literally visible, effectively invisible”. Seamful integration is hard, but the quality of interaction can be improved if we let each tool ‘be itself’. – Matthew Chalmers (2003)

Matt Ratto investigates the darker side of this drive towards invisibility, revealing that seamlessness encourages:

“a particular kind of passivity and lack of engagement between people and their actions and between people and their social and material environment” and that we must “critique the clean, orderly, and homogenous future that is at the heart of these modernist visions” – Matt Ratto (2007)

And Anne Galloway suggests that it is in the seams where the design work can be done:

“Although seamlessness may remain a powerful and effective metaphor to guide particular projects, when it comes to actually getting the work done—and the challenges of having to do it with people who can be very different from each other—then I suggest it is in everyone’s best interests to recognise the importance of seams and scars in marking places where interventions can be made, or where potential can be found and acted upon.” – Anne Galloway (2007)

In interaction design we need to look at the long history of Durrell Bishop’s work, one of the strongest advocates for self-evident design, whether it is physical or virtual, through his teaching and design practice. Durrell’s ‘Platform 12’ in the RCA Design Products course attempts to see design as:

“a celebration of a model for how things work, where once again we can treat function as beauty, instead of merely treating design as form and image.”

Durrell’s work on the Marble Answering Machine (1992) is a brilliant piece of self-evident design, and remains a touchstone for all interaction design work.

Designers also need to look at the first four chapters of ‘Where the action is’ by Paul Dourish which give a coherent account of the relationships between human abilities and computer interfaces over the last 50-60 years. Dourish shows how interfaces are not becoming invisible, but how they are increasingly social and tangible.

And finally, from a design perspective, there is a long tradition of making complex products legible and understandable. Industrial designer Konstantin Grcic talks about the relationship between the technologies and the use of an object:

“A machine is beautiful when it’s legible, when its form describes how it works. It isn’t simply a matter of covering the technical components with an outer skin, but finding the correct balance between the architecture of the machine… and an expressive approach that is born out of the idea of interaction with those using the object.” – Konstantin Grcic (2007)

And perhaps more famously, Dieter Rams has always talked of honesty and understanding in his product design practice. Making a product understandable is one of his Ten Principles of “Good Design”.

dieter_rams-10principles-04

This drive for understanding needs to go further than physical form (as it has done at Apple) and start to inform the design of systems and UI.

Towards legible, evident interaction

We must abandon invisibility as a goal for interfaces; it’s misleading, unhelpful and ultimately dishonest. It unleashes so much potential for unusable, harmful and frustrating interfaces, and systems that gradually erode users and designers agency. Invisibility might seem an attractive concept at first glance, but it ignores the real, thorny, difficult issues of designing and using complex interfaces and systems.

We might be better off instead taking our language from typography, and for instance talk about legibility and readability without denying that typography can call attention to itself in beautiful and spectacular ways. Our goal should be to ‘place as much control as possible in the hands of the end-user by making interfaces evident’.

Of course the interfaces we design may become normalised in use, effectively invisible over time, but that will only happen if we design them to be legible, readable, understandable and to foreground culture over technology. To build trust and confidence in an interface in the first place, enough that it can comfortably recede into the background.

Immaterials at the Vimeo awards 2012

Posted on May 22, 2012 in Film
  • Immaterials-Wifi

The film Immaterials: Light painting WiFi made with Einar Sneve Martinussen and Jørn Knutsen as part of the Yourban project is a finalist at the Vimeo Awards 2012.

The awards ceremony is on the 7 June at 8PM in New York City. The Vimeo Festival also looks great!

Robot readable world

Posted on Feb 7, 2012 in Research

This is a short film, an experiment in machine-vision footage. It uses found-footage from computer vision research to explore how machines are making sense of the world.

As robots begin to inhabit the world alongside us, how do they see and gather meaning from our streets, cities, media and from us? Machines have a tiny, fractional view of our environment, that sometimes echoes our own human vision and often doesn’t.

Read more about the film and have a look at Matt Jones’ talk of the same title.

Three films on communication and networks

Posted on Nov 5, 2011 in Film, Media, Narrative, Technology, Television

In the last two weeks I’ve seen three documentaries dealing with communication and networks.

Firstly, a broad and ambitious film from Ericsson, taking on the ‘networked society’ including interviews with David Weinberger, Catarina Fake and Eric Wahlforss.

Each of the interviewees discusses the emerging opportunities being enabled by technology as we enter the Networked Society. Concepts such as borderless opportunities and creativity, new open business models, and today’s ‘dumb society’ are brought up and discussed.

The next film from Nokia brings daily life around networked communication technologies to the forefront, and does it through lovely experiential sequences. However it does come across much more as a branding exercise or promotional piece, and doesn’t offer to explain or explore the practices it shows.

Third is a film by Ben Mendelsohn and Alex Chohlas-Wood about the physical, geographic and material infrastructure that goes into running the internet.

Lower Manhattan’s 60 Hudson Street is one of the world’s most concentrated hubs of Internet connectivity. This short documentary peeks inside, offering a glimpse of the massive material infrastructure that makes the Internet possible.

There is clearly a need to unpack the increasingly technology-inflected geography, and social and cultural practices of the world we inhabit, so it is good to see films like this being made.

Talk to Me

Posted on Sep 4, 2011 in Film, Interaction design

I have five works in MoMA’s latest exhibition ‘Talk to Me’ in New York. The works are Nearness, Immaterials, Media Surfaces, The Journey and Suwappu.

I have written more about the exhibition and the works at the Touch and BERG weblogs. The exhibition has also been reviewed by CNN, the New York Times, Fast Company and the Wall Street Journal amongst others.

The films of Adam Lisagor

Posted on Jul 27, 2011 in Film, video

I’ve been quite taken with the films of Adam Lisagor for a while.

I make small, palatable videos, like commercials, for companies involved in tech, to figure out how to convey the essence of their products in concise, accessible ways.

I like to think that I’m able to do this because I think slowly enough to notice the exact points while using a product at which I respond with the most delight. And if I can reproduce those moments on-screen, without explicitly saying that they’re delightful, an audience will intuitively understand the delight they might feel themselves.

(From an interview in Business Insider.)

He’s good at surfacing the joy and pleasure in some of the smallest interactions, particularly evident in this ad for the Jambox by Jawbone.

Mother. Father. Always you wrestle inside me.

Posted on Jul 10, 2011 in Film, Photography

A truth that releases a waterfall of emotion. It is this energy that propels us through The Tree of Life. A voluptuous, bulging energy shaped and encouraged by sweeping camera movement, ultra wide lenses, lyrical blocking, the safe-harbor of Jessica Chastain’s face, and the vacillation in Hunter McCracken’s. These combine to create scenes that perfectly capture the rapturous feelings of childhood. Sensations evoked when light & dark entwine, and our instinctual knowledge that these things are the same.

And on how to approach the film:

A moment long enough for me to relax, and I was suddenly taken by a feeling of great tenderness and calm. I don’t completely understand why I felt this, but the inclusion of these CGI dinosaurs struck me as an particularly affectionate and loving decision. Terrence Malick believes in his audiences, and has faith that we also can believe. It’s the feeling of your mother brushing the hair off your forehead as she tells you a bedtime story. You protest because she’s changed a part of the usual tale, or it’s not the way you want it to be, but smiling, she says “Shhh shhh. Just listen.”

From the brilliant Kartina Richardson.

Branding with badges

Posted on Jul 10, 2011 in Graphic design, Interaction design

Many of the early ideas were fairly conceptual: Not a place or a thing, but an idea tied to the use of Foursquare itself (“10 check-ins”) or the kind of real-world social behavior the service was attempting to leverage (checking into the same place three times in one week, or checking in with two people of the opposite sex). The round shape and circular border directly referenced Boy Scouts merit badges. Beyond that, Sheibley says the relevant design context wasn’t logos, it was the familiar instructional iconography meant to signal ideas without words: “How do you communicate to people in an airport, who don’t speak the same language, where the bathroom is?”

Summify – Rob Walker: On Slate: Branding with badges

Midnight Run

Posted on Jul 9, 2011 in Film, Narrative
My favourite movie of all time.

A film with one compelling relationship at its center might not survive the bombardments of the action formula its script demands, and that’s another reason “Midnight Run” is so special. There are at least half a dozen relationships throughout the film surrounding Grodin and De Niro that ring true in their entirety. A phone call between an angry mob boss and his bumbling enforcer, for example, could very easily be made into a transitional scene devoid of intrinsic value, but this film does something different on those occasions: it utilizes shards of moments as points of insight and endows them with authenticity largely nonessential to the plot.

Stealthie!

Posted on Jul 6, 2011 in Graphic design

Nice work from Richard Hogg: Hi I’m Stealthie!

Connecting the digital world with print

Posted on Jun 16, 2011 in Interaction design, Ubicomp

A rare piece of writing from Durrell Bishop:

What if there was a generic tool to link the digital and the physical worlds? A way to touch an object, to select and see its digital augmentation? It would just send a message via your chosen device: laptop, mobile, home wifi et al. The equivalent of a digital finger. Passive objects would act as physical buttons to the digital world. All that these objects would require is that we perceive their purpose, and see how to act on them. We would need to have an accessible tool to make our selection, and carry out the link between the two worlds.

Read the whole thing: Connecting the digital world with print.

Science Fiction Film as Design Scenario Exercise for Psychological Habitability

“They deliver potentially fundamental insights (Schneider, 2005) into the interaction between humans and the constructed environment surrounding them, including the mundance aspects of everyday routine (Carroll, 2000), even the potential subversion of the system or setting through its agents (Blythe & Wright, 2006). The user is advanced into a character or specific persona placed in fictional but feasible settings (Nielsen, 2002). The representation of scenarios through prototypes, use of storyboards, video, rapid prototyping tools and stories, annotated sketches, cartoons, photographs, role-playing or live dramatization (Suri & Marsh, 2000), allows the vision conveyed through the scenario to be opened up to critique (Carroll, 2000).”

“Design, here, does not assume the traditional role of problem-solving, but acts as a critical agent in the enquiry about real human needs and values by evoking reflection (Carroll, 1995) and stimulating debate amongst designers, industry and the public (Dunne & Raby, 2001). This critical strand in design, termed speculative design, critical design or design fiction, has emerged as a field in its own right and is establishing itself with the main markers of a new discipline, such as seminal publications (Dunne, 2005; Antonelli, 2008), exhibitions and conferences.”

Science Fiction Film as Design Scenario Exercise for Psychological Habitability: Production Designs 1955-2009

A phone to save us from our screens?

Posted on Oct 11, 2010 in Film, Interaction design, Mobility, Ubicomp

Microsoft has two new ads, anticipating their upcoming Windows Phone 7 launch. The first is an almost post-apocalyptic vision of humanity stuck with their heads in their mobile devices:

Here’s David Webster, chief strategy officer in Microsoft’s central marketing group, explaining their anti-screen strategy:

“Our sentiment was that if we could have an insight to drive the campaign that flipped the category on its head, then all the dollars that other people are spending glorifying becoming lost in your screen or melding with your phone are actually making our point for us.”

The problem of glowing rectangles is a subject close to my heart, and Matt Jones has been bothered by the increase in mobile glowing attention-wells.

I think Microsoft & Crispin Porter + Bogusky’s advertising strategy stands out in a world full of slick floaty media. The only problem is that without any strategy towards tangible interaction, I’m not sure the ‘tiles’ interaction concept is strong enough to actually take people’s attention out of the glass.

Load More