Making Visible

Posted on Jun 26, 2014 in Interaction design, Research

Making-Visible-Cover-2

My PhD thesis called ‘Making Visible’ was submitted in December 2013 and successfully defended on 12 June 2014. The thesis reflects upon the design material exploration research from the Touch and Yourban projects. It uses these explorations to situate design research with technology as a cultural, material and mediational practice:

In Making Visible I outline how interaction design may engage in the material and mediation of new interface technologies. Drawing upon a design project called Touch, that investigated an emerging interface technology called Radio Frequency Identification or RFID, I show how interaction design research can explore technology through material and mediational approaches. I demonstrate and analyse how this research addresses the inter-related issues of invisibility, seamlessness and materiality that have become central issues in the design of contemporary interfaces. These issues are analysed and developed through three intertwined approaches of research by design: 1. a socio- and techno-cultural approach to understanding emerging technologies, 2. through material exploration and 3. through communication and mediation. When taken together these approaches form a communicative mode of interaction design research that engages directly with the exploration, understanding and discussion of emerging interface technologies.

The thesis is made up of four peer-reviewed journal articles accompanied by a ‘meta-reflection’ document that reflects upon and situates these publications through theory, concepts and models.

Meta-reflection

This document develops the concept of mediational material that focuses attention on the material and communicative practices in interaction design. These are used to explore, develop and share knowledge of technologies as design materials.

Making-Visible-Thumbnails2web

This document is 178 pages with 53 illustrations. I’ve made it available in a number of different digital formats:

It will also be available through AHO’s open-access archive ADORA.

Articles

The four included articles have been published in peer-reviewed journals.

Article 1: Exploring ‘immaterials’: mediating design’s invisible materials

This article takes up the issues of so-called ‘immaterial’ and ‘seamless’ technologies and asks how designers might explore them in order to consider them as design materials. It situates interaction design as a sociocultural practice that is concerned with culture, critical approaches and with engaging the technocultural imagination. It concludes by analysing its mediational strategies, such as the use of documentary formats, online film and weblog writing, and the ways in which new material perspectives have been shared, discussed and developed by others.

Arnall, T. (in press). ‘Exploring ‘immaterials’: mediating design’s invisible materials’. International Journal of Design, 29. Will be available at the International Journal of Design.

Article 2: Visualizations of digital interaction in daily life

This article explores how visual signage may make aspects of ubiquitous computing technologies visible and how digital tools and platforms impact that visual design and semiosis. It looks at a range of ‘identification’ technologies such as barcodes and rfid, that only become ‘visible’ or ‘interactional’ through a designer’s intervention in physical or visual expression. It finds that designers should emphasize the bindings and distinctions between design processes and visual mediations, and symbols and signs, in engaging with emerging technologies as material for creative and communicative composition.

Morrison, A., & Arnall, T. (2011). Visualizations of Digital Interaction in Daily Life. Computers and Composition, 28(3), 224-234. Available at Computers and Composition.

Article 3: Satellite Lamps

Satellite lamps is a project about using design to investigate and reveal one of the fundamental constructs of the networked city: GPS – the Global Positioning System. It extends the concepts of ‘mediational materials’ to an investigation of the ways in which GPS technology inhabits urban spaces. The article takes up how a discursive and reflexive interaction design practice can contribute to new perspectives on networked city life. Importantly, Satellite Lamps is a multimediational web text, involving different media (film, media, notebooks and a host of images) allowing for the richness of the work to come to the surface in a way that would not have been possible in traditional means of academic publishing.

Martinussen, E, Knutsen, J & Arnall, T. (forthcoming 2014). Satellite Lamps. Kairos: A Journal of Rhetoric, Technology, and Pedagogy. Will be available at Kairos.

Article 4: Depth of Field: Discursive design research through film

This article is about the role of film in interaction and product design research with technology, and the use of film in exploring and explaining emerging technologies in multiple contexts. It concludes by looking towards the potentials for a discursive design practice, where the object of design and analysis is the discourse that is catalysed by new artefacts, and the emphasis of design research is on communication.

Arnall, T., & Martinussen, E. S. (2010). Depth of field: discursive design research through film. FORMakademisk, 3(1). Available at FORMakademisk

The Immaterials project

Posted on Sep 4, 2013 in Interaction design, Research

RFID-Touch-Project

The Immaterials project is concerned with the increasing invisibility of interfaces and infrastructures. The systems we interact with everyday such as WiFi and 3G networks have a profound impact on how we experience the world. As Adam Greenfield says:

the complex technologies the networked city relies upon to produce its effects remain distressingly opaque, even to those exposed to them on a daily basis. [...] it’s hard to be appropriately critical and to make sound choices in a world where we don’t understand the objects around us.

And as James Bridle has eloquently and disturbingly observed:

Those who cannot perceive the network cannot act effectively within it, and are powerless.

The project set out to expose some of the phenomena and mechanisms of technological infrastructures through visual, photographic, narrative, animated and cinematic techniques. Over the last five years I have worked with Einar Sneve Martinussen, Jørn Knutsen, Jack Schulze and Matt Jones towards a body of work that is now brought together in an exhibition for the first time.

From 2004–2008 I speculated about the ways in which wireless interactions inhabited physical space, through my work on a Graphic language for touch, and also through films such as Wireless in the world. Some of my students made beautiful but fictional speculations about the physical qualities of different kinds of radio.

Jack Schulze and I also made a short, playful film called Nearness about action at a distance. In the film, a series of simple reactions are set off by immaterial phenomena, such as radio waves, mobile networks, light, magnetism and wind.

In 2006 we ran a Touch workshop with BERG where we became concerned about the invisibility of RFID technology, and the effect that had on our ability to design with it. We found it extraordinary that a technology that was defined as a proximity or ‘touch’-based interface, was so opaque in terms of its physical, spatial, gestural materiality. How do we as designers make these materials visible, so we can have reflective conversations with them?

We developed Experiments in Field Drawing as a method of revealing, literally drawing, the physical presence of RFID interactions. We revealed these fields in a much richer, multi-dimensional way using photography, animation and light painting in the film Immaterials: Ghost in the Field.

Matt Jones coined the term immaterials to describe the project and gave a great talk about some ways of understanding the immaterials of interaction design. Matt and I also looked at machine vision, another phenomena that increasingly becomes a material for design in Robot Readable World.

In 2011 at AHO, as part of a research project called Yourban, we extended the investigations to WiFi, using similar light painting techniques we revealed the enormous scale and pervasiveness of ad-hoc WiFi networks in urban spaces in Immaterials: Light Painting WiFi.

Finally, over the last two years we’ve become increasingly interested in the Global Positioning System (GPS), that has become a central part of both the vision and the implementation of contemporary interfaces.

Satellite Lamps

We have built a series of Satellite Lamps that sense the presence of the 24 GPS satellites in orbit. The lamps change brightness according to the strength of GPS signals they receive, showing how the technology itself is messy and unpredictable, and revealing how GPS is a negotiation between radio waves, earth-orbit geometry and the urban environment.

Satellite Lamps has so far been exhibited at Lighthouse and Dread, you can watch the film and read the extensive article detailing our process as well as a cultural history of GPS.

The visual languages that we’ve developed have ended up in advertising, on the BBC and Discovery Channel, and the techniques have been extended in research at MIT and CIID, and by many designers, enthusiasts and hackers. It’s exciting that both the subject and the methods are being taken up and used broadly by other people, and we’re looking forward to seeing more.

the truly pressing need is for translators: people capable of opening these occult systems up, demystifying them, explaining their implications to the people whose neighborhoods and choices and very lives are increasingly conditioned by them. — Adam Greenfield (2009)

The Immaterials project emerged from the humble preoccupations of a few designers dealing with some of the invisible, immaterial, intangible stuff we had in front of us. These small experiments led to larger and more visually and narratively communicative work. In the end what I think we’ve developed is an approach to technology that revolves around material exploration, explanation and communication. Because images and language, as well as materials, form our understandings of technology, Immaterials has shown how we can use ‘design and playful explorations to shape or stir the popular imagination’.

The exhibitions

All the Immaterials projects are on display at Lighthouse in Brighton from 5 September until 13 October 2013.

Satellite Lamps and Robot Readable World are on display at Dread in Amsterdam from 7 September until 24 November 2013.

No to NoUI

Posted on Mar 13, 2013 in Interaction design, Ubicomp

Invisible-design

The best design is invisible‘ is the interaction design phrase of the moment. The images above are from my ever-expanding collection of quotes about how design and technology will ‘disappear‘, become ‘invisible‘ or how the ‘best interface is no interface‘.

The Verge has recently given both Oliver Reichenstein and Golden Krishna a platform to talk about this. This has spawned manifestos, films, talks, books, #NoUI hashtags and some debates about what it might mean. I’ll call this cluster of things ‘invisible design’.

I agree with some of the reasons driving this movement; that design’s current infatuation with touchscreens is really problematic. I’ve spent the last eight years rallying against glowing rectangles, studying our obsession with screens and the ways in which this has become a cultural phenomena. In response I have been researching and inventing interfaces for taking interaction out from under the glass.

But I also take issue with much of the thinking for a few reasons that I’ll outline below.

1. Invisible design propagates the myth of immateriality

We already have plenty of thinking that celebrates the invisibility and seamlessness of technology. We are overloaded with childish mythologies like ‘the cloud’; a soft, fuzzy metaphor for enormous infrastructural projects of undersea cables and power-hungry data farms. This mythology can be harmful and is often just plain wrong. Networks go down, hard disks fail, sensors fail to sense, processors overheat and batteries die.

Computing systems are suffused through and through with the constraints of their materiality. – Jean-François Blanchette

Invisible design propogates the myth that technology will ‘disappear’ or ‘just get out of the way’ rather than addressing the qualities of interface technologies that can make them difficult or delightful.

Intentionally hiding the phenomena and materiality of interfaces, smoothing over the natural edges, seams and transitions that constitute all technical systems, entails a loss of understanding and agency for both designers and users of computing. Lack of understanding leads to uncertainty and folk-theories that hinder our ability to use technical systems, and clouds the critique of technological developments.

As systems increasingly record our personal activity and data, invisibility is exactly the wrong model.

By removing our knowledge of the glue that holds the systems that make up the infrastructure together, it becomes much more difficult, if not impossible, to begin to understand how we are constructed as subjects, what types of systems are brought into place (legal, technical, social, etc.) and where the possibilities for transformation exist. – Matt Ratto (2007)

In other words, as both users and designers of interface technology, we are disenfranchised by the concepts of invisibility and disappearance.

2. Invisible design falls into the natural/intuitive trap

The movement tells us to ‘embrace natural processes’ and talks about the ‘incredibly intuitive’ Mercedes car interface. This language is a trap (we should ban the use of natural and intuitive btw) that doesn’t give us any insight into how complex products might actually become simple or familiar.

Invisible design leads us towards the horrors of Reality Clippy. Does my refrigerator light really go off? Why was my car unlocked this morning? How did my phone go silent all of a sudden? Without highly legible systems for managing and understanding all of this ‘smartness’ we are going to get very lost and highly frustrated. The tricky business of push notifications and the Facebook privacy train wreck is just the tip of the iceberg.

The example of the Nest thermostat invisibly ‘learning’ your habits to control your home temperature is a good one. But the Nest has a highly visible interface that reassures you as to its status, tells you when it is learning, and a large dial for adjusting temperature. Beautiful, legible microinteractions. A Nest without these visual and direct manipulation interfaces would be useless, uncanny and frustrating. Nest wants UI.

The discussion around invisible design often talks about using sensors and tangible interfaces instead of visual interfaces. But these systems are not inherently simpler or more familiar. They have their own material qualities with edges and ‘grain’ that need to be understood and learnt. Their literal invisibility can cause confusion, even fear, and they often increase unpredictability and failure.

In our work with interface technologies such as RFID and computer vision, we’ve discovered that it takes a lot of work to make sense of the technologies as design materials. So it’s not useful to say that UI is ‘disappearing’ into sensing, algorithms and tangible interfaces, when we don’t fully understand them as UI yet.

3. Invisible design ignores interface culture

Interfaces are the dominant cultural form of our time. So much of contemporary culture takes place through interfaces and inside UI. Interfaces are part of cultural expression and participation, skeuomorphism is evidence that interfaces are more than chrome around content, and more than tools to solve problems. To declare interfaces ‘invisible’ is to deny them a cultural form or medium. Could we say ‘the best TV is no TV’, the ‘best typography is no typography’ or ‘the best buildings are no architecture’?

Much of our work at BERG is not just about solving problems, but about cultural invention:

We’re not interested in this idea of the invisible technology in a modernist sense. Tech won’t be visible but only if it’s embedded into the culture that it exists within. By foregrounding the culture, you background the technology. It’s the difference between grinding your way through menus on an old Nokia, trying to do something very simple, and inhabiting the bright bouncy bubbly universe of iOS. The technology is there, of course, but it’s effectively invisible as the culture is foregrounded.” – Jack Schulze (in Domus 965 / January 2013)

We should be able to simultaneously celebrate the fantastic explosion of diversity in UI, and develop healthy critique around the use of interfaces like touch screens. But by calling for UI to disappear altogether so that things can be more efficient, we remain in the same utilitarian and rational mindset that produces inert technological visions like this, rather than seeing interfaces as part of the cultural landscape.

4. Invisible design ignores design and technology history

The movement ignores at least thirty years of thinking in design and technology. A few examples:

Much of the recent invisible design discussions repeat the thinking in Jared Spool’s ‘Great Designs Should Be Experienced and Not Seen‘ and Donald Norman’s ‘Invisible Computer. But a better reference point would be Don Norman’s earlier book, The design of everyday things, where he instead talks about the ‘problems caused by inadequate attention to visibility’ and supporting or managing our mental models of systems. We need a lot more thinking about our mental models of algorithms in particular.

Adam Greenfield has investigated the social and ethical issues around the development of ubiquitous computing systems, and is particularly concerned by its disappearance:

“Ubiquitous systems must contain provisions for immediate and transparent querying of their ownership, use, and capabilities. Everyware must, in other words, be self-disclosing. Whether such disclosures are made graphically, or otherwise, they ensure that you are empowered to make informed decisions as to the level of exposure you wish to entertain.” – Adam Greenfield (2006)

Some designers have talked about the actual qualities they want from ubiquitous computing interfaces, such as polite, pertinent and pretty:

“The vast quantities of information that personal informatics generate need not only to be clear and understandable to create legibility and literacy in this new world, but I’d argue in this first wave also seductive, in order to encourage play, trial and adoption” Matt Jones & Tom Coates (2008)

Matthew Chalmers has, more than anyone else, revealed the history of seamlessness. Seamlessness is ‘the deliberate “making invisible” of the variety of technical systems, artifacts, individuals and organizations that make up an information infrastructure. This work actively disguises the moments of transition and boundary crossing between these various parts in order to present a solid and seemingly coherent interface to users.’ (Ratto 2007). Although Mark Weiser is often thought of as an advocate of seamless systems, Chalmers found that:

Weiser describes seamlessness as a misleading or misguided concept. In his invited talks to UIST94 and USENIX95 he suggested that making things seamless amounts to making everything the same, reducing components, tools and systems to their ‘lowest common denominator’. He advocated seamful systems (with “beautiful seams”) as a goal. Around Xerox PARC, where many researchers worked on document tools, Weiser used an example of seamful integration of a paint tool and a text editor (Weiser, personal communication). He complained that seamless integration of such tools often meant that the user was forced to use only one of them. One tool would be chosen as primary and the others reduced and simplified to conform to it, or they would be crudely patched together with ugly seams. Seamfully integrated tools would maintain the unique characteristics of each tool, through transformations that retained their individual characteristics. This would let the user brush some characters with the paint tool in some artful way, then use the text editor to ‘search and replace’ some of the brushstroked characters, and then paint over the result with colour washes. Interaction would be seamless as the features of each tool were “literally visible, effectively invisible”. Seamful integration is hard, but the quality of interaction can be improved if we let each tool ‘be itself’. – Matthew Chalmers (2003)

Matt Ratto investigates the darker side of this drive towards invisibility, revealing that seamlessness encourages:

“a particular kind of passivity and lack of engagement between people and their actions and between people and their social and material environment” and that we must “critique the clean, orderly, and homogenous future that is at the heart of these modernist visions” – Matt Ratto (2007)

And Anne Galloway suggests that it is in the seams where the design work can be done:

“Although seamlessness may remain a powerful and effective metaphor to guide particular projects, when it comes to actually getting the work done—and the challenges of having to do it with people who can be very different from each other—then I suggest it is in everyone’s best interests to recognise the importance of seams and scars in marking places where interventions can be made, or where potential can be found and acted upon.” – Anne Galloway (2007)

In interaction design we need to look at the long history of Durrell Bishop‘s work, one of the strongest advocates for self-evident design, whether it is physical or virtual, through his teaching and design practice. Durrell’s ‘Platform 12′ in the RCA Design Products course attempts to see design as:

“a celebration of a model for how things work, where once again we can treat function as beauty, instead of merely treating design as form and image.”

Durrell’s work on the Marble Answering Machine (1992) is a brilliant piece of self-evident design, and remains a touchstone for all interaction design work.

Designers also need to look at the first four chapters of ‘Where the action is‘ by Paul Dourish which give a coherent account of the relationships between human abilities and computer interfaces over the last 50-60 years. Dourish shows how interfaces are not becoming invisible, but how they are increasingly social and tangible.

And finally, from a design perspective, there is a long tradition of making complex products legible and understandable. Industrial designer Konstantin Grcic talks about the relationship between the technologies and the use of an object:

“A machine is beautiful when it’s legible, when its form describes how it works. It isn’t simply a matter of covering the technical components with an outer skin, but finding the correct balance between the architecture of the machine… and an expressive approach that is born out of the idea of interaction with those using the object.” – Konstantin Grcic (2007)

And perhaps more famously, Dieter Rams has always talked of honesty and understanding in his product design practice. Making a product understandable is one of his Ten Principles of “Good Design”.

dieter_rams-10principles-04

This drive for understanding needs to go further than physical form (as it has done at Apple) and start to inform the design of systems and UI.

Towards legible, evident interaction

We must abandon invisibility as a goal for interfaces; it’s misleading, unhelpful and ultimately dishonest. It unleashes so much potential for unusable, harmful and frustrating interfaces, and systems that gradually erode users and designers agency. Invisibility might seem an attractive concept at first glance, but it ignores the real, thorny, difficult issues of designing and using complex interfaces and systems.

We might be better off instead taking our language from typography, and for instance talk about legibility and readability without denying that typography can call attention to itself in beautiful and spectacular ways. Our goal should be to ‘place as much control as possible in the hands of the end-user by making interfaces evident‘.

Of course the interfaces we design may become normalised in use, effectively invisible over time, but that will only happen if we design them to be legible, readable, understandable and to foreground culture over technology. To build trust and confidence in an interface in the first place, enough that it can comfortably recede into the background.

Talk to Me

Posted on Sep 4, 2011 in Film, Interaction design

I have five works in MoMA‘s latest exhibition ‘Talk to Me‘ in New York. The works are Nearness, Immaterials, Media Surfaces, The Journey and Suwappu.

I have written more about the exhibition and the works at the Touch and BERG weblogs. The exhibition has also been reviewed by CNN, the New York Times, Fast Company and the Wall Street Journal amongst others.

Branding with badges

Posted on Jul 10, 2011 in Graphic design, Interaction design

Many of the early ideas were fairly conceptual: Not a place or a thing, but an idea tied to the use of Foursquare itself (“10 check-ins”) or the kind of real-world social behavior the service was attempting to leverage (checking into the same place three times in one week, or checking in with two people of the opposite sex). The round shape and circular border directly referenced Boy Scouts merit badges. Beyond that, Sheibley says the relevant design context wasn’t logos, it was the familiar instructional iconography meant to signal ideas without words: “How do you communicate to people in an airport, who don’t speak the same language, where the bathroom is?”

Summify – Rob Walker: On Slate: Branding with badges

Connecting the digital world with print

Posted on Jun 16, 2011 in Interaction design, Ubicomp

A rare piece of writing from Durrell Bishop:

What if there was a generic tool to link the digital and the physical worlds? A way to touch an object, to select and see its digital augmentation? It would just send a message via your chosen device: laptop, mobile, home wifi et al. The equivalent of a digital finger. Passive objects would act as physical buttons to the digital world. All that these objects would require is that we perceive their purpose, and see how to act on them. We would need to have an accessible tool to make our selection, and carry out the link between the two worlds.

Read the whole thing: Connecting the digital world with print.

Science Fiction Film as Design Scenario Exercise for Psychological Habitability

“They deliver potentially fundamental insights (Schneider, 2005) into the interaction between humans and the constructed environment surrounding them, including the mundance aspects of everyday routine (Carroll, 2000), even the potential subversion of the system or setting through its agents (Blythe & Wright, 2006). The user is advanced into a character or specific persona placed in fictional but feasible settings (Nielsen, 2002). The representation of scenarios through prototypes, use of storyboards, video, rapid prototyping tools and stories, annotated sketches, cartoons, photographs, role-playing or live dramatization (Suri & Marsh, 2000), allows the vision conveyed through the scenario to be opened up to critique (Carroll, 2000).”

“Design, here, does not assume the traditional role of problem-solving, but acts as a critical agent in the enquiry about real human needs and values by evoking reflection (Carroll, 1995) and stimulating debate amongst designers, industry and the public (Dunne & Raby, 2001). This critical strand in design, termed speculative design, critical design or design fiction, has emerged as a field in its own right and is establishing itself with the main markers of a new discipline, such as seminal publications (Dunne, 2005; Antonelli, 2008), exhibitions and conferences.”

Science Fiction Film as Design Scenario Exercise for Psychological Habitability: Production Designs 1955-2009

A phone to save us from our screens?

Posted on Oct 11, 2010 in Film, Interaction design, Mobility, Ubicomp

Microsoft has two new ads, anticipating their upcoming Windows Phone 7 launch. The first is an almost post-apocalyptic vision of humanity stuck with their heads in their mobile devices:

Here’s David Webster, chief strategy officer in Microsoft’s central marketing group, explaining their anti-screen strategy:

“Our sentiment was that if we could have an insight to drive the campaign that flipped the category on its head, then all the dollars that other people are spending glorifying becoming lost in your screen or melding with your phone are actually making our point for us.”

The problem of glowing rectangles is a subject close to my heart, and Matt Jones has been bothered by the increase in mobile glowing attention-wells.

I think Microsoft & Crispin Porter + Bogusky’s advertising strategy stands out in a world full of slick floaty media. The only problem is that without any strategy towards tangible interaction, I’m not sure the ’tiles’ interaction concept is strong enough to actually take people’s attention out of the glass.

Olars: physical toy inspired by karl sims evolved creatures

Posted on Jun 25, 2010 in Film, Interaction design

A lovely piece of work by Lars M. Vedeler and Ola Vågsholm from the Tangible Interactions course at The Oslo School of Architecture & Design:

Olars is an electronic interactive toy inspired by Karl Sims’ evolved virtual creatures. Having thousands of varieties in movement and behaviour by attaching different geometrical limbs, modifying the angle of these, twisting the body itself, and by adjusting the deflection of the motorised joints, results in both familiar and strange motion patterns.

Olars on Vimeo

Negotiating futures. Design fiction

Posted on Apr 6, 2010 in Conferences, Film, Interaction design, Media, mediation,

Swiss Design Network Conference 2010:

Designers see the world not simply as it is, but rather as it could be. In this perspective, the world is a laboratory to explore the contingency of the existing and the thinking in options. Imaginations of the contra factual are a key source for the creation of alternative political, technological, social, or economic constellations of artefacts, interfaces, signs, actors, and spaces. At the same time, strategies of materialization are pivotal to shift the boundary between the fictional and the real and to finally bring possible new realities into being. The conference addresses the questions of how fictions are designed and how the multiplicity of possible new futures is negotiated and realized.

Design Fiction, Negotiating Futures October 28-30, 2010.

3D secret – hidden pictures

Posted on Feb 27, 2010 in Graphic design, Interaction design, Play, video

Beautiful new exploratory game for the Nintendo DS, that uses the front-facing camera and face tracking to calculate a perspective that renders like a window on a new world.

DSi「立体かくし絵 アッタコレダ. Via BERG

Augmentia

Posted on Feb 2, 2010 in Interaction design, Media, Technology, Ubicomp

Anselm lays out the emerging issues with Augmented Reality (AR). In doing so he relates it to a whole host of known and unknown problems associated with ubiquitous computing, semantic publishing and data platforms.

Below are some clippings of bits that seem particularly insightful:

bq. It puts own embodiment at risk. And whomsoever can mitigate that risk while providing reward will probably do well. I believe that organizations such as Apple and Google see this and are pursuing not merely real-time, or hyper-local or crowd-sourced apps but ownership of the “view”.

bq. Everybody wants a part of the lens of reality, the zero-click base layer beneath the beneath. As Gene Becker puts it “The World is the Platform”. And an ecosystem is starting to emerge.

bq. Suddenly game developers are arguing with GIS experts and having to unify their very different ways of describing mirror worlds.

bq. [I]nterfaces move from being heavy and solid with big heavy buttons and knobs and rotary dials to becoming liquid and effortless like the dynamic UI of the iPhone to becoming like air itself.

bq. By making hidden things visible, and visible things cheap, it will make other things possible that we don’t entirely realize yet.

bq. There will be user interface interaction issues. What will be the conventions for hand-swipes, grabs, drags, pulls and other operations to manipulate objects in our field of view.

bq. [AR] is not simply “memory” – it isn’t just a mnemonic that helps bring understanding closer to the surface of consciousness. Clearly we are surrounded by our own memories, signage, advertising, radio, friends voices and an already rich complicated teeming natural landscape loaded with signifiers and cues. But it is another bridge between personal lived experience and the experience of others. It seems to lower costs of knowing, and it seems to provide stronger subjective filters.

bq. Augmented Reality seems to at least offer the possibility that we can punch some holes in the boxes. It seems to offer a bridge between structure and chaos rather than just structure.

Telling stories with interfaces

Posted on Jan 25, 2010 in Film, Information design, Interaction design, Media

“But where does it go from here? Is this really just a micro-genre best suited to ads for internet companies? Or does the fact that we spend so much time on this stage our selves mean that it really can be the venue for more (and more kinds of) storytelling?

via Telling stories with interfaces « Snarkmarket.

Augmented reality experiments

I’m really not a fan of the goggle/glasses/helmet variety of AR, where the user wears something in front of their eyes that superimposes 3D objects into the physical world. In my experience this has been slow, inaccurate, cumbersome, headache inducing, the worst of VR plus a lot more problems. But AR is really interesting when it’s just a screen and a video feed, it becomes somehow magical: to see the same space represented twice: once in front of you, and once on screen with magical objects. I can imagine this working really well on mobile phones: the phone screen as magic lens to secret things.

Hand drawing markers

On that afternoon we didn’t have a printer handy for making the AR marks, so we took to drafting them by hand, stencilling them off the screen with a pencil and inking them in. This hand-crafted process led to all sorts of interesting connections between the possibilities of craft and digital information.

AR nail decorations

We had lots of ideas about printing the markers on clothes, painting them on nails, glazing them into ceramics, etc. We confused ARtoolkit by drawing markers in perspective, and tried to get recursive objects by using screen based markers and video feedback.

Confusing ARtoolkit

Now as it turns out there is an entire research programme dedicated to looking at just this topic. “Variable Environment”:http://sketchblog.ecal.ch/variable_environment/ is a research programme involving partners like “ECAL”:http://www.ecal.ch/pages/home_new.asp and “EPFL”:http://www.epfl.ch. The great thing is that they are blogging the entire exploratory (they call it ‘sketch’) phase and curating the results online. The work is multi-disciplinary and involves architects, visual designers, computer scientists, interaction designers, etc. Check out the simple “AR ready products”:http://sketchblog.ecal.ch/variable_environment/archives/2006/07/ar_ready_simple.html, “sample applications”:http://sketchblog.ecal.ch/variable_environment/archives/2006/07/applications_1.html and “mixed reality tests”:http://sketchblog.ecal.ch/variable_environment/archives/2006/01/mixed_reality_t_1.html with “various patterns”:http://sketchblog.ecal.ch/variable_environment/archives/2006/03/test_01_pattern.html.

This seems to be part of a shift in the research community, to publishing ongoing and exploratory work online (championed by the likes of “Nicolas Nova”:http://tecfa.unige.ch/perso/staf/nova/blog/ and “Anne Galloway”:http://www.purselipsquarejaw.org/). Very inspirational.

The address book desk

Posted on Dec 16, 2005 in Interaction design, Project, Research, Ubicomp

Underneath the desk I have stuck a grid of RFID tags, and on the top surface, the same grid of post-it notes. With the standard Nokia Service Discovery application it is possible to call people, send pre-defined SMSes or load URLs by touching the phone to each post-it on the desk. On the post-its themselves I have hand-written the function, message and the recipient. This is somewhat like a cross between a phone-book, to-do list and temporary diary, with notes, scribbles and tea stains alongside names.

Initial ideas were to spraypaint or silkscreen some of the touch icons to the desk surface, and I may well do that at some point. But for quick prototyping it made sense to use address labels or post-it notes that can be stuck, re-positioned and layered with hand-written notes.

This is an initial step in thinking about the use of RFID and mobile phones, a way of thinking through making. In many ways it is proving to be more inconvenient than the small screen (particularly with the occasionally unreliable firmware on this particular cover, I can’t speak for the production version). But it has highlighted some really interesting issues.

First of all it has brought to the forefront the importance of implicit habits. Initially, it took a real effort to think about the action of using the table as an interface: I would reach for the phone and press names to make a call, instead of placing it on the desk. But for some functions, such as sending an SMS, it has become more habitual.

SMSes have become more like ‘pings’ when very little effort is made to send them. At the same time they are more physically tangible: I rest the phone in a certain position on the desk and wait for it to complete an action. The most useful functions have been “I’m here” or “I’m leaving” messages to close friends.

I have had to consider the ‘negative space’ where the mobile must rest without any action. This space has potential be used for context information; a corner of the table could make my phone silent, another corner could change my presence online. Here it would be interesting to refer to Jan Chipchase’s ideas around centres of gravity and points of reflection, it’s these points that could be most directly mapped to behaviour. I’m thinking about other objects and spaces that might be appropriate for this, and perhaps around the idea of thoughtless acts.

If this was a space without wireless internet, I could also imagine this working very well for URLs: quick access to google searches, local services or number lookups, which is usually very tricky on a small screen. Here it would be interesting to think about how the mobile is used in non-connected places, such as the traditional Norwegian Hytte [pdf].

This process also raised a larger issue around the move towards tangible or physical information, which also implies a move towards the social. As I was making the layout of my address book and associated functions, I realised that maybe these things shouldn’t be explicit, visible, social objects. The arrangement of people within the grid makes personal sense; the placement is a personal preference and maps in certain ways to frequency and type of contact. But I wonder how it appears to other people when this pattern is exposed. Will people be offended by my layout? What if I don’t include a rarely called contact? Are there numbers I want to keep secret, hidden behind acronyms in the ‘Names’ menu?

It will be interesting to see how this plays out and changes over time, particularly in the reaction of others. I’ll post more about the use of NFC in other personal contexts in the near future.

h3. The making of…

The desk is made from 20 mm birch ply, surfaced in Linoleum. I stuck a single RFID to the underside, in the place that felt most natural. A 10 cm grid was worked out from that point, and the RFIDs were stuck in that grid, and the same worked out on top. If I were to re-build the desk with this project in mind, the tags should probably be layered close to the surface, between the ply and Linoleum. This would make them slightly more responsive to touch by giving them a larger read/write distance.

p(caption). Rewriteable 512 bit, Philips MiFare UL stickers.

p(caption). 10 cm grid of tags on the underside of the desk.

p(caption). Blank post-it notes on the surface, with the same grid.

More photos at Flickr.

Load More