No to NoUI

Posted on Mar 13, 2013 in Interaction design, Ubicomp

Invisible-design

The best design is invisible‘ is the interaction design phrase of the moment. The images above are from my ever-expanding collection of quotes about how design and technology will ‘disappear‘, become ‘invisible‘ or how the ‘best interface is no interface‘.

The Verge has recently given both Oliver Reichenstein and Golden Krishna a platform to talk about this. This has spawned manifestos, films, talks, books, #NoUI hashtags and some debates about what it might mean. I’ll call this cluster of things ‘invisible design’.

I agree with some of the reasons driving this movement; that design’s current infatuation with touchscreens is really problematic. I’ve spent the last eight years rallying against glowing rectangles, studying our obsession with screens and the ways in which this has become a cultural phenomena. In response I have been researching and inventing interfaces for taking interaction out from under the glass.

But I also take issue with much of the thinking for a few reasons that I’ll outline below.

1. Invisible design propagates the myth of immateriality

We already have plenty of thinking that celebrates the invisibility and seamlessness of technology. We are overloaded with childish mythologies like ‘the cloud'; a soft, fuzzy metaphor for enormous infrastructural projects of undersea cables and power-hungry data farms. This mythology can be harmful and is often just plain wrong. Networks go down, hard disks fail, sensors fail to sense, processors overheat and batteries die.

Computing systems are suffused through and through with the constraints of their materiality. – Jean-François Blanchette

Invisible design propogates the myth that technology will ‘disappear’ or ‘just get out of the way’ rather than addressing the qualities of interface technologies that can make them difficult or delightful.

Intentionally hiding the phenomena and materiality of interfaces, smoothing over the natural edges, seams and transitions that constitute all technical systems, entails a loss of understanding and agency for both designers and users of computing. Lack of understanding leads to uncertainty and folk-theories that hinder our ability to use technical systems, and clouds the critique of technological developments.

As systems increasingly record our personal activity and data, invisibility is exactly the wrong model.

By removing our knowledge of the glue that holds the systems that make up the infrastructure together, it becomes much more difficult, if not impossible, to begin to understand how we are constructed as subjects, what types of systems are brought into place (legal, technical, social, etc.) and where the possibilities for transformation exist. – Matt Ratto (2007)

In other words, as both users and designers of interface technology, we are disenfranchised by the concepts of invisibility and disappearance.

2. Invisible design falls into the natural/intuitive trap

The movement tells us to ‘embrace natural processes’ and talks about the ‘incredibly intuitive’ Mercedes car interface. This language is a trap (we should ban the use of natural and intuitive btw) that doesn’t give us any insight into how complex products might actually become simple or familiar.

Invisible design leads us towards the horrors of Reality Clippy. Does my refrigerator light really go off? Why was my car unlocked this morning? How did my phone go silent all of a sudden? Without highly legible systems for managing and understanding all of this ‘smartness’ we are going to get very lost and highly frustrated. The tricky business of push notifications and the Facebook privacy train wreck is just the tip of the iceberg.

The example of the Nest thermostat invisibly ‘learning’ your habits to control your home temperature is a good one. But the Nest has a highly visible interface that reassures you as to its status, tells you when it is learning, and a large dial for adjusting temperature. Beautiful, legible microinteractions. A Nest without these visual and direct manipulation interfaces would be useless, uncanny and frustrating. Nest wants UI.

The discussion around invisible design often talks about using sensors and tangible interfaces instead of visual interfaces. But these systems are not inherently simpler or more familiar. They have their own material qualities with edges and ‘grain’ that need to be understood and learnt. Their literal invisibility can cause confusion, even fear, and they often increase unpredictability and failure.

In our work with interface technologies such as RFID and computer vision, we’ve discovered that it takes a lot of work to make sense of the technologies as design materials. So it’s not useful to say that UI is ‘disappearing’ into sensing, algorithms and tangible interfaces, when we don’t fully understand them as UI yet.

3. Invisible design ignores interface culture

Interfaces are the dominant cultural form of our time. So much of contemporary culture takes place through interfaces and inside UI. Interfaces are part of cultural expression and participation, skeuomorphism is evidence that interfaces are more than chrome around content, and more than tools to solve problems. To declare interfaces ‘invisible’ is to deny them a cultural form or medium. Could we say ‘the best TV is no TV’, the ‘best typography is no typography’ or ‘the best buildings are no architecture’?

Much of our work at BERG is not just about solving problems, but about cultural invention:

We’re not interested in this idea of the invisible technology in a modernist sense. Tech won’t be visible but only if it’s embedded into the culture that it exists within. By foregrounding the culture, you background the technology. It’s the difference between grinding your way through menus on an old Nokia, trying to do something very simple, and inhabiting the bright bouncy bubbly universe of iOS. The technology is there, of course, but it’s effectively invisible as the culture is foregrounded.” – Jack Schulze (in Domus 965 / January 2013)

We should be able to simultaneously celebrate the fantastic explosion of diversity in UI, and develop healthy critique around the use of interfaces like touch screens. But by calling for UI to disappear altogether so that things can be more efficient, we remain in the same utilitarian and rational mindset that produces inert technological visions like this, rather than seeing interfaces as part of the cultural landscape.

4. Invisible design ignores design and technology history

The movement ignores at least thirty years of thinking in design and technology. A few examples:

Much of the recent invisible design discussions repeat the thinking in Jared Spool’s ‘Great Designs Should Be Experienced and Not Seen‘ and Donald Norman’s ‘Invisible Computer. But a better reference point would be Don Norman’s earlier book, The design of everyday things, where he instead talks about the ‘problems caused by inadequate attention to visibility’ and supporting or managing our mental models of systems. We need a lot more thinking about our mental models of algorithms in particular.

Adam Greenfield has investigated the social and ethical issues around the development of ubiquitous computing systems, and is particularly concerned by its disappearance:

“Ubiquitous systems must contain provisions for immediate and transparent querying of their ownership, use, and capabilities. Everyware must, in other words, be self-disclosing. Whether such disclosures are made graphically, or otherwise, they ensure that you are empowered to make informed decisions as to the level of exposure you wish to entertain.” – Adam Greenfield (2006)

Some designers have talked about the actual qualities they want from ubiquitous computing interfaces, such as polite, pertinent and pretty:

“The vast quantities of information that personal informatics generate need not only to be clear and understandable to create legibility and literacy in this new world, but I’d argue in this first wave also seductive, in order to encourage play, trial and adoption” Matt Jones & Tom Coates (2008)

Matthew Chalmers has, more than anyone else, revealed the history of seamlessness. Seamlessness is ‘the deliberate “making invisible” of the variety of technical systems, artifacts, individuals and organizations that make up an information infrastructure. This work actively disguises the moments of transition and boundary crossing between these various parts in order to present a solid and seemingly coherent interface to users.’ (Ratto 2007). Although Mark Weiser is often thought of as an advocate of seamless systems, Chalmers found that:

Weiser describes seamlessness as a misleading or misguided concept. In his invited talks to UIST94 and USENIX95 he suggested that making things seamless amounts to making everything the same, reducing components, tools and systems to their ‘lowest common denominator’. He advocated seamful systems (with “beautiful seams”) as a goal. Around Xerox PARC, where many researchers worked on document tools, Weiser used an example of seamful integration of a paint tool and a text editor (Weiser, personal communication). He complained that seamless integration of such tools often meant that the user was forced to use only one of them. One tool would be chosen as primary and the others reduced and simplified to conform to it, or they would be crudely patched together with ugly seams. Seamfully integrated tools would maintain the unique characteristics of each tool, through transformations that retained their individual characteristics. This would let the user brush some characters with the paint tool in some artful way, then use the text editor to ‘search and replace’ some of the brushstroked characters, and then paint over the result with colour washes. Interaction would be seamless as the features of each tool were “literally visible, effectively invisible”. Seamful integration is hard, but the quality of interaction can be improved if we let each tool ‘be itself’. – Matthew Chalmers (2003)

Matt Ratto investigates the darker side of this drive towards invisibility, revealing that seamlessness encourages:

“a particular kind of passivity and lack of engagement between people and their actions and between people and their social and material environment” and that we must “critique the clean, orderly, and homogenous future that is at the heart of these modernist visions” – Matt Ratto (2007)

And Anne Galloway suggests that it is in the seams where the design work can be done:

“Although seamlessness may remain a powerful and effective metaphor to guide particular projects, when it comes to actually getting the work done—and the challenges of having to do it with people who can be very different from each other—then I suggest it is in everyone’s best interests to recognise the importance of seams and scars in marking places where interventions can be made, or where potential can be found and acted upon.” – Anne Galloway (2007)

In interaction design we need to look at the long history of Durrell Bishop‘s work, one of the strongest advocates for self-evident design, whether it is physical or virtual, through his teaching and design practice. Durrell’s ‘Platform 12′ in the RCA Design Products course attempts to see design as:

“a celebration of a model for how things work, where once again we can treat function as beauty, instead of merely treating design as form and image.”

Durrell’s work on the Marble Answering Machine (1992) is a brilliant piece of self-evident design, and remains a touchstone for all interaction design work.

Designers also need to look at the first four chapters of ‘Where the action is‘ by Paul Dourish which give a coherent account of the relationships between human abilities and computer interfaces over the last 50-60 years. Dourish shows how interfaces are not becoming invisible, but how they are increasingly social and tangible.

And finally, from a design perspective, there is a long tradition of making complex products legible and understandable. Industrial designer Konstantin Grcic talks about the relationship between the technologies and the use of an object:

“A machine is beautiful when it’s legible, when its form describes how it works. It isn’t simply a matter of covering the technical components with an outer skin, but finding the correct balance between the architecture of the machine… and an expressive approach that is born out of the idea of interaction with those using the object.” – Konstantin Grcic (2007)

And perhaps more famously, Dieter Rams has always talked of honesty and understanding in his product design practice. Making a product understandable is one of his Ten Principles of “Good Design”.

dieter_rams-10principles-04

This drive for understanding needs to go further than physical form (as it has done at Apple) and start to inform the design of systems and UI.

Towards legible, evident interaction

We must abandon invisibility as a goal for interfaces; it’s misleading, unhelpful and ultimately dishonest. It unleashes so much potential for unusable, harmful and frustrating interfaces, and systems that gradually erode users and designers agency. Invisibility might seem an attractive concept at first glance, but it ignores the real, thorny, difficult issues of designing and using complex interfaces and systems.

We might be better off instead taking our language from typography, and for instance talk about legibility and readability without denying that typography can call attention to itself in beautiful and spectacular ways. Our goal should be to ‘place as much control as possible in the hands of the end-user by making interfaces evident‘.

Of course the interfaces we design may become normalised in use, effectively invisible over time, but that will only happen if we design them to be legible, readable, understandable and to foreground culture over technology. To build trust and confidence in an interface in the first place, enough that it can comfortably recede into the background.

Connecting the digital world with print

Posted on Jun 16, 2011 in Interaction design, Ubicomp

A rare piece of writing from Durrell Bishop:

What if there was a generic tool to link the digital and the physical worlds? A way to touch an object, to select and see its digital augmentation? It would just send a message via your chosen device: laptop, mobile, home wifi et al. The equivalent of a digital finger. Passive objects would act as physical buttons to the digital world. All that these objects would require is that we perceive their purpose, and see how to act on them. We would need to have an accessible tool to make our selection, and carry out the link between the two worlds.

Read the whole thing: Connecting the digital world with print.

A phone to save us from our screens?

Posted on Oct 11, 2010 in Film, Interaction design, Mobility, Ubicomp

Microsoft has two new ads, anticipating their upcoming Windows Phone 7 launch. The first is an almost post-apocalyptic vision of humanity stuck with their heads in their mobile devices:

Here’s David Webster, chief strategy officer in Microsoft’s central marketing group, explaining their anti-screen strategy:

“Our sentiment was that if we could have an insight to drive the campaign that flipped the category on its head, then all the dollars that other people are spending glorifying becoming lost in your screen or melding with your phone are actually making our point for us.”

The problem of glowing rectangles is a subject close to my heart, and Matt Jones has been bothered by the increase in mobile glowing attention-wells.

I think Microsoft & Crispin Porter + Bogusky’s advertising strategy stands out in a world full of slick floaty media. The only problem is that without any strategy towards tangible interaction, I’m not sure the ’tiles’ interaction concept is strong enough to actually take people’s attention out of the glass.

CCD and computational photography

Posted on Mar 18, 2010 in Media, Photography, Technology, Ubicomp

A few links on imaging and computation:

I’ve concluded that the promise of RFID was eclipsed by another technology out there that’s poised to become more and more disruptive, not only to RFID, but to a host of technologies, and that’s the CCD.

from CCD by Joe Gregorio. Via BERG.

Cameras might allow a photographer to record a scene and then alter the lighting or shift the point of view, or even insert fictitious objects.

from Computational Photography, American Scientist

The camera as a device you carry has completely disappeared. Image sensors have become part of the literal fabric of everyday life.

from What Photography Will Look Like By 2060

Practising tomorrows

Posted on Mar 2, 2010 in Ubicomp

It takes ubiquitous computing as a significant case study because the future orientation practised in ubiquitous computing research and development is emblematic of the perpetual technological forecasting in which humanity engages.

“Practising tomorrows? Ubiquitous computing and the politics of anticipation” a PhD by Sam Kinsley.

via Anne

Augmentia

Posted on Feb 2, 2010 in Interaction design, Media, Technology, Ubicomp

Anselm lays out the emerging issues with Augmented Reality (AR). In doing so he relates it to a whole host of known and unknown problems associated with ubiquitous computing, semantic publishing and data platforms.

Below are some clippings of bits that seem particularly insightful:

bq. It puts own embodiment at risk. And whomsoever can mitigate that risk while providing reward will probably do well. I believe that organizations such as Apple and Google see this and are pursuing not merely real-time, or hyper-local or crowd-sourced apps but ownership of the “view”.

bq. Everybody wants a part of the lens of reality, the zero-click base layer beneath the beneath. As Gene Becker puts it “The World is the Platform”. And an ecosystem is starting to emerge.

bq. Suddenly game developers are arguing with GIS experts and having to unify their very different ways of describing mirror worlds.

bq. [I]nterfaces move from being heavy and solid with big heavy buttons and knobs and rotary dials to becoming liquid and effortless like the dynamic UI of the iPhone to becoming like air itself.

bq. By making hidden things visible, and visible things cheap, it will make other things possible that we don’t entirely realize yet.

bq. There will be user interface interaction issues. What will be the conventions for hand-swipes, grabs, drags, pulls and other operations to manipulate objects in our field of view.

bq. [AR] is not simply “memory” – it isn’t just a mnemonic that helps bring understanding closer to the surface of consciousness. Clearly we are surrounded by our own memories, signage, advertising, radio, friends voices and an already rich complicated teeming natural landscape loaded with signifiers and cues. But it is another bridge between personal lived experience and the experience of others. It seems to lower costs of knowing, and it seems to provide stronger subjective filters.

bq. Augmented Reality seems to at least offer the possibility that we can punch some holes in the boxes. It seems to offer a bridge between structure and chaos rather than just structure.

Touch

Posted on Sep 4, 2006 in Mobility, Project, Research, Technology, Ubicomp

Augmented reality experiments

I’m really not a fan of the goggle/glasses/helmet variety of AR, where the user wears something in front of their eyes that superimposes 3D objects into the physical world. In my experience this has been slow, inaccurate, cumbersome, headache inducing, the worst of VR plus a lot more problems. But AR is really interesting when it’s just a screen and a video feed, it becomes somehow magical: to see the same space represented twice: once in front of you, and once on screen with magical objects. I can imagine this working really well on mobile phones: the phone screen as magic lens to secret things.

Hand drawing markers

On that afternoon we didn’t have a printer handy for making the AR marks, so we took to drafting them by hand, stencilling them off the screen with a pencil and inking them in. This hand-crafted process led to all sorts of interesting connections between the possibilities of craft and digital information.

AR nail decorations

We had lots of ideas about printing the markers on clothes, painting them on nails, glazing them into ceramics, etc. We confused ARtoolkit by drawing markers in perspective, and tried to get recursive objects by using screen based markers and video feedback.

Confusing ARtoolkit

Now as it turns out there is an entire research programme dedicated to looking at just this topic. “Variable Environment”:http://sketchblog.ecal.ch/variable_environment/ is a research programme involving partners like “ECAL”:http://www.ecal.ch/pages/home_new.asp and “EPFL”:http://www.epfl.ch. The great thing is that they are blogging the entire exploratory (they call it ‘sketch’) phase and curating the results online. The work is multi-disciplinary and involves architects, visual designers, computer scientists, interaction designers, etc. Check out the simple “AR ready products”:http://sketchblog.ecal.ch/variable_environment/archives/2006/07/ar_ready_simple.html, “sample applications”:http://sketchblog.ecal.ch/variable_environment/archives/2006/07/applications_1.html and “mixed reality tests”:http://sketchblog.ecal.ch/variable_environment/archives/2006/01/mixed_reality_t_1.html with “various patterns”:http://sketchblog.ecal.ch/variable_environment/archives/2006/03/test_01_pattern.html.

This seems to be part of a shift in the research community, to publishing ongoing and exploratory work online (championed by the likes of “Nicolas Nova”:http://tecfa.unige.ch/perso/staf/nova/blog/ and “Anne Galloway”:http://www.purselipsquarejaw.org/). Very inspirational.

The address book desk

Posted on Dec 16, 2005 in Interaction design, Project, Research, Ubicomp

Underneath the desk I have stuck a grid of RFID tags, and on the top surface, the same grid of post-it notes. With the standard Nokia Service Discovery application it is possible to call people, send pre-defined SMSes or load URLs by touching the phone to each post-it on the desk. On the post-its themselves I have hand-written the function, message and the recipient. This is somewhat like a cross between a phone-book, to-do list and temporary diary, with notes, scribbles and tea stains alongside names.

Initial ideas were to spraypaint or silkscreen some of the touch icons to the desk surface, and I may well do that at some point. But for quick prototyping it made sense to use address labels or post-it notes that can be stuck, re-positioned and layered with hand-written notes.

This is an initial step in thinking about the use of RFID and mobile phones, a way of thinking through making. In many ways it is proving to be more inconvenient than the small screen (particularly with the occasionally unreliable firmware on this particular cover, I can’t speak for the production version). But it has highlighted some really interesting issues.

First of all it has brought to the forefront the importance of implicit habits. Initially, it took a real effort to think about the action of using the table as an interface: I would reach for the phone and press names to make a call, instead of placing it on the desk. But for some functions, such as sending an SMS, it has become more habitual.

SMSes have become more like ‘pings’ when very little effort is made to send them. At the same time they are more physically tangible: I rest the phone in a certain position on the desk and wait for it to complete an action. The most useful functions have been “I’m here” or “I’m leaving” messages to close friends.

I have had to consider the ‘negative space’ where the mobile must rest without any action. This space has potential be used for context information; a corner of the table could make my phone silent, another corner could change my presence online. Here it would be interesting to refer to Jan Chipchase’s ideas around centres of gravity and points of reflection, it’s these points that could be most directly mapped to behaviour. I’m thinking about other objects and spaces that might be appropriate for this, and perhaps around the idea of thoughtless acts.

If this was a space without wireless internet, I could also imagine this working very well for URLs: quick access to google searches, local services or number lookups, which is usually very tricky on a small screen. Here it would be interesting to think about how the mobile is used in non-connected places, such as the traditional Norwegian Hytte [pdf].

This process also raised a larger issue around the move towards tangible or physical information, which also implies a move towards the social. As I was making the layout of my address book and associated functions, I realised that maybe these things shouldn’t be explicit, visible, social objects. The arrangement of people within the grid makes personal sense; the placement is a personal preference and maps in certain ways to frequency and type of contact. But I wonder how it appears to other people when this pattern is exposed. Will people be offended by my layout? What if I don’t include a rarely called contact? Are there numbers I want to keep secret, hidden behind acronyms in the ‘Names’ menu?

It will be interesting to see how this plays out and changes over time, particularly in the reaction of others. I’ll post more about the use of NFC in other personal contexts in the near future.

h3. The making of…

The desk is made from 20 mm birch ply, surfaced in Linoleum. I stuck a single RFID to the underside, in the place that felt most natural. A 10 cm grid was worked out from that point, and the RFIDs were stuck in that grid, and the same worked out on top. If I were to re-build the desk with this project in mind, the tags should probably be layered close to the surface, between the ply and Linoleum. This would make them slightly more responsive to touch by giving them a larger read/write distance.

p(caption). Rewriteable 512 bit, Philips MiFare UL stickers.

p(caption). 10 cm grid of tags on the underside of the desk.

p(caption). Blank post-it notes on the surface, with the same grid.

More photos at Flickr.

Nokia 3220 with NFC

Posted on Dec 6, 2005 in Interaction design, Mobility, Research, Ubicomp, Usability

h3. First impressions

Overall the interaction between phone and RFID tags has been good. The reader/writer is on the base of the phone, at the bottom. This seems a little awkward to use at first, but slowly becomes natural. When I have given it to others, their immediate reaction is to point the top of the phone to the tag, and nothing happens. There follows a few moments of explaining as the intricacies of RFID and looking at the phone, with it’s Nokia ‘fingerprint’ icon. As phones increasingly become replacements for ‘contactless cards’, it seems likely that this interaction will become more habitual and natural.

Once the ‘service discovery’ application is running, the read time from tags is really quick. The sharp vibrations and flashing lights add to a solid feeling of interacting with something, both in the haptic and visual senses. This should turn out to be a great platform for embodied interaction with information and function.

The ability to read and write to tags makes it potentially adaptive as a platform wider than just advertising or ticketing. As an interaction designer I feel quite enabled by this technology: the three basic functions (making phonecalls, going to URLs, or sending SMSs) are enough to start thinking about tangible interactions without having to go and program any Java midlets or server-side applications.

I’m really happy that Nokia is putting this technology into a ‘low-end’ phone rather than pushing it out in a ‘smartphone’ range. This is where there is potential for wider usage and mass-market applications, especially around gaming and content discovery.

h3. Improvements

I had some problems launching the ‘service discovery’ application. Sometimes it works, sometimes it doesn’t and it’s difficult to tell why this is. It would be great to be able to place the phone on the table, knowing that it will respond to a tag, but it was just a little too unreliable to do that without checking to see that it had responded. The version I have still says it’s a prototype, so this may well be sorted out by the released version.

Overall there is a lack of integration between the service discovery application and the rest of the system: Contacts, SMS archive and service bookmarks etc. At the moment we need to enter the application to write and manage tags, or to give a ‘shortcut’ to another phone, but it seems that, as with bluetooth and IR, this should be part of the contextual menus that appear under ‘Options’ within each area of the phone. There are also some infuriating prompts that appear when interacting with URL, more details below.

h3. Details

p(caption). The phone opens the ‘service discovery’ application whenever it detects a compatible RFID tag near the base of the phone (when the keypad lock is off). This part is a bit obscure: sometimes it doesn’t ‘wake up’ for a tag, and the application needs to be loaded before it will read properly. Once the application is open (about 2-3 seconds) the read time of the tags seems instantaneous.

p(caption). The shortcuts menu gives access to shortcuts. Confusingly, this is different from ‘bookmarks’ and the ‘names’ list on the phone, although names can be searched from within the application. I think tighter integration with the OS is called for.

p(caption). Shortcuts can be added, edited, deleted, etc. in the same way as contacts. They can be ‘Given’ to another phone or ‘Written’ to a tag.

p(caption). There are three kinds of shortcuts: Call, URL or SMS. ‘Call’ will create a call to a pre-defined number, ‘URL’ will load a pre-defined URL, and ‘SMS’ will send a pre-defined SMS to a particular number. This part of the application has the most room for innovative extensions: we should be able to set the state of the phone, change profiles, change themes, download graphics, etc. This can be achieved by loading URLs, but URLs and mobiles don’t mix, so why should we be presented with them, when there could be a more usable layer inbetween? There could also be preferences for prompts: at the moment each action has to be confirmed with a yes or a no, but in some secure environments it would be nice to be able to have a function launched without the extra button push.

p(caption). If a tag contains no data, then we are notified and placed back on the main screen (as happened when I tried to write to my Oyster card).

p(caption). If the tag is writeable we are asked which shortcut to write to the tag.

p(caption). When we touch a tag with a shortcut on it, a prompt appears asking for confirmation. This is a level of UI to prevent mistakes, and a certain level of security, but it also reduces the overall usability of the system. With URL launching, there are two stages of confirmation, which is infuriating. There needs to be some other mode of confirmation, and the ‘service discovery’ app needs to somehow be deeper in the system to avoid these double button presses.

p(caption). Lastly, there is a log of actions. Useful to see if the application has been reading something in your bag or wallet, without you knowing…

Graphic language for touch

This work explores the visual link between information and physical things, specifically around the emerging use of the mobile phone to interact with RFID or NFC. It was a presentation and poster at Design Engaged, Berlin on the 11th November 2005.

Download the icons (PDF, 721KB, Gif preview).

As mobile phones are increasingly able to read and write to RFID tags embedded in the physical world, I am wondering how we will appropriate this for personal and social uses.

I’m interested in the visual link between information and physical things. How do we represent an object that has digital function, information or history beyond it’s physical form? What are the visual clues for this interaction? We shouldn’t rely on a kind of mystery meat navigation (the scourge of the web-design world) where we have to touch everything to find out it’s meaning.

This work doesn’t attempt to be a definitive system for marking physical things, it is an exploratory process to find out how digital/physical interactions might work. It uncovers interesting directions while the technology is still largely out of the hands of everyday users.

h3. Reference to existing work

Visual references

p(caption). Click for larger version.

The inspiration for this is in the marking of public space and existing iconography for interactions with objects: push buttons on pedestrian crossings, contactless cards, signage and instructional diagrams.

This draws heavily on the substantial body of images of visual marking in public space. One of the key findings of this research was that visibility and placement of stickers in public space is an essential part of their use. Current research in ubicomp and ‘locative media’ is not addressing these visibility issues.

There is also a growing collection of existing iconography in contactless payment systems, with a number of interesting graphic treatments in a technology-led, vernacular form. In Japan there are also instances of touch-based interactions being represented by characters, colours and iconography that are abstracted from the action itself.

I have also had great discussions with Ulla-Maaria Mutanen and Jyri Engestr�m who have been doing interesting work with thinglinks and the intricate weaving of RFID into craft products.

h3. Development

rfid_iconography_circles.gif

Sketching and development revealed five initial directions: circles, wireless, card-based, mobile-based and arrows (see the poster for more details). The icons range from being generic (abstracted circles or arrows to indicate function) to specific (mobile phones or cards touching tags).

Arrows might be suitable for specific functions or actions in combinations with other illustrative material. Icons with mobile phones or cards might be helpful in situations where basic usability for a wide range of users is required. Although the ‘wireless’ icons are often found in current card readers, they do not successfully indicate the touch-based interactions inherent in the technology, and may be confused with WiFi or Bluetooth. The circular icons work at the highest level, and might be most suitable for generic labelling.

rfid_iconography_circles.gif

For further investigation I have selected a simple circle, surrounded by an ‘aura’ described by a dashed line. I think this successfully communicates the near field nature of the technology, while describing that the physical object contains something beyond its physical form.

rfid_iconography_2circle.gif

In most current NFC implementations, such as the 3220 from Nokia and many iMode phones, the RFID reader is in the bottom of the phone. This means that the area of ‘activation’ is obscured in many cases by the phone and hand. The circular iconography allows for a space to be marked as ‘active’ by the size of the circle, and we might see it used to mark areas rather than points. Usability may improve when these icons are around the same size as the phone, rather than being a specific point to touch.

h3. Work in progress

This is early days for this technology, and this is work-in-progress. There is more to be done in looking at specific applications, finding suitable uses and extending the language to cover other functions and content.

Until now I have been concerned with generic iconography for a digitally augmented object. But this should develop into a richer language, as the applications for this type of interaction become more specific, and related to the types of objects and information being used. For example it would be interesting to find a graphic treatment that could be applied to a Pokemon sticker offering power-ups as well as a bus stop offering timetable downloads.

I’m also interested in the physical placement of these icons. How large or visible should they be? Are there places that should not be ‘active’? And how will this fit with the natural, centres of gravity of the mobile phone in public and private space.

I’ll expand on these things in a few upcoming projects that explore touch-based interactions in personal spaces.

Feel free to use and modify the icons, I would be very interested to see how they can be applied and extended.

h3. Visual references

Oyster Card, Transport for London.
eNFC, Inside Contactless.
Paypass, Mastercard.
ExpressPay, American Express.
FeliCa, Sony.
MiFare, various vendors.
Suica, JR, East Japan Railway Company.
RFID Field Force Solutions, Nokia.
NFC shell for 3220, Nokia.
ERG Transit Systems payment, Dubai.
Various generic contactless vendors.
Contactless payment symbol, Mastercard.
Open Here, Paul Mijksenaar, Piet Westendorp, Thames and Hudson, 1999.
Understanding Comics, Scott McCloud, Harper, 1994

Design Engaged 2005

Posted on Nov 20, 2005 in Conferences, Interaction design, Place, Travel, Ubicomp

h3. Intro by Andrew Otwell

Restorative feeling of getting together.
The architecture of participation.
Experience design is participation design.

h3. Adam Greenfield

Ethical and social implications of ubicomp
Has been reading far afield “Out of gas” “Dark age ahead” “Collapse
Growth and decline
Models of growth:
1. Health of economy is based on growth
2. Continuous growth: technological singularity: going off the charts
3. Cancer,
Decline: cyclical decline, populations, rise and fall of the nile
But sometime it’s not cyclical: the global production of oil.
Growth and terminal decline: global oil production. Industrial civilisation becomes very hard to maintain.
There is a contingency that designers probably ought to be thinking about.
An implication: things get intensely local
Things:
Power sources, reclamation, working and forging
Textiles dyes colorings
Materials
Methods practices and standards: knowledge: resident in the minds and things around us.
Deployment of finished items
Value: ie copper may be worth more locally as a metal to be worked, than as wiring in an inert grid.
A new suite of design gestures
Lose:
Our near-instantaneous google+wikipedia
Lose Apple-Z
Flexibility and unquestioned connectivity (the social, fluid, soft meeting space)
digital visualization and rapid prototyping
Petrochemical or dependent derived material properties
Precision fabrication
Low material costs
The global community

Will become important
Libraries
Workshops
Adaptive re-use
Modularity, interoperability
Mash-up ethos
Sense of possibility and play
Reclamation of old industrialisation
Re-densification: new urban centres
Urban core may be untenable: bulding higher than 6 floors may be difficult
KH Kunstler: the long emergency

h3. Matt Ward

Has design lost its way.
Design is always ideological, influenced by specific world views, responds to cultural context, design is a product (of the society it originates from).
It’s also world shaping, future oriented, not yet, soon to be, one day, wouldn’t it be cool if. Imagination and production. Cannot claim autonomy from social change.
Utopia, Outopia: no place. Eutopia: good place
The desire to be somewhere different, not about this place not now, free of the problems of this world.
No problem to link utopianism with the act of designing.
Many see design as a key to changing the world. Corbusier.
Design is also linked to control and power.
Archigram: a move away from conservatism. Anti-utopianism, learning from the mistakes of their forefathers.
Inbetween the space of freedom and control: metamorphosis
Utopia of difference: terroristic meta-narratives (Tom Moylan)
Superstudio: allows for different uses and actions to define the architecture of the studio. A critique of the modernists.
Critical utopia: asks questions: asks towards change, towards critical design.
Problem with critical design: forgets to question the position that the designer has
Locative media: a lot of technology removes us from everyday life, we need to remember the dogshit and the chewing gum.
Malcolm: the avant garde is now 2 minutes.
Digital design: change in space and place: information acts within the production of space.

h3. Joshua Kaufman

Book: The Whale and the Reactor (A search for limits in an age of high technology)
Social consequences of design from a historical perspective
Artefacts have political properties: invention, design or arrangement of artifact becomes a way of settling an issue in the affaris of a community. Some articles are inherently political (weaponry)
Political design can be found in architecture and urban planning
Robert Moses low overpasses: discouraged lower-class travel
Barron Haussmann’s thoroughfares: large roads and parks.
Concrete buildings and plazas at universities, to discourage demonstrations
Cyrus McCorrmick II’s molding machines
How design choices affect the relative distribution of power.
Fabio: Relationship between artefact and culture is complex. You can put the baby out into the world, but can’t take responsibility for all it’s actions.

h3. Anne Galloway

Design and the parliament of things
Making the work of a social scientist applicable
Philosophy of science
The difference between an assemblage and an assembly
“To assemble is one thing; to represent to the eyes and ears of those assembled what is at stake is another” Bruno Latour, How to Make Things Public.
The different between objects and things
In design, things are objects: stable and neutral. Even users are objects.
Things: an assembly: to come together
A coming together precicely because we have differences.
From Realpolitics to Thingpolitics.
“Walking is controlled falling” Laurie Andersen.
Representative democracy: body-politic: unifying difference, stabilising and reducing difference.
From representation to re-presentation: all the constituent parts
Re-presenting design: invention and capitalism, speed and mobilisation.
Going from an assemblage of different parts to an assembly.
Speed and competition, focusing on the novel: ideas and materials and practices are being mobilised, getting rid of the obstacles that will slow it down.
This doesn”t help us ‘assemble’. A constant space of speed, not encouraged to slow down and think things through. We don’t have an assembly.
We can’t afford to keep ignoring each other.
“Your concerns are not relevant to our task at hand, you are not enabling my efficiency, etc.”
What is it going to take to design in this parliament of things?
We can start by slowing down.
“That’s where slowing down comes in you can create new habits only by slowing down, because new habits also mean new feelings, new interests, new possibilities” Isabelle Stengers
Matt: voting is a last resort, consensus is reached in different ways.
Anne: not interested in consensus, interested in convergence.
Malcolm: taking into account, not accounting the bottom line: Latour
Matt: strategies and tactics in moving towards slowness? A: get outside, get into the ground.

h3. Michelle Chang

Public by design
Interested in the issue of privacy
Privacy / Publicness
Public library
How the library became a microcosm of the city: a number of services and interfaces to the city
Practices: activities.
Space
What technologies are public goods? Electricity, wireless, etc.
Which technologies must be individually wrought? Laptops, mobiles.
“A public institution must bear the burden of public demands”
Anne: a public space or a commons? A convergence
Use of the internet problematic: an increasing use of “adult terminals” versus “children’s terminals”.
The library becomes a sort of pitstop: getting out of the rain or using the toilet.

h3. Thomas Vander Wal

Clouds, space and black boxes
Information is found and created, but there is little distinction between the two.
Personal info cloud, local infocloud, global infocloud
A clearing house of filters from the flood of information that flows down.
Filtering based on trust and value: aggregation
A ‘firewall’ that filters what information we give to people
A ‘smart’ black box
Some people keep all different bits of their personal data stored underneath different usernames: so as to not keep all data aggregated in one place.

h3. Louise Klinker

Guilty consciences
Using environmental issues. Using P2P software
Crimewire.
Linking music download to criminal activity, but also tracking music downloads according to the amount of money owed to industry and artists.

h3. Malcolm McCullough

Taking into account the notion of civic space: architecture.
Closed laptop, notebook…

h3. Fabio Sergio

The skin of objects
No definitions of interaction design!
Where is the material of interaction design?
Joy Mountfort: “We are designing the skin of objects” reacted strongly against this.
Industrial design: from Olivetti cash register to Apple G5.
Ben: models are the substance of interaction design. The act of creating constraints and systemic understandings.
The visualisation of complexity: making new materials from the mass of information.
Conceptual models affecting the physical models.
Liz: what are models? Fundamentals of interaction design are stories.

h3. Stefan Smagula

Open source media
Project Lightspeed: AT&T, Yahoo and Microsoft.
Video
Big players: Apple, IPTV, Yahoo, Google
Bootstrappers: Open Media Network, DTV (Bittorrent & RSS), Open Source Radio
RSS provides metadata and triggers downloads
Matt Jones: Jerry Cornelius: an open source character: giving away plots and stories.

h3. Chris Heathcote

Personalisation
Panasonic gets personalisation.
Barcodes on skins.
Websites for cover creation.
Harajuku store for expensive personal covers.
Korea: metal depicting war scenes, cloth.
A spectrum of engagement, consume, buy, customise, accessorize, alter, make, design.
Find out how, make stuff, sell stuff. cyworld (half population of Korea?), habbo, myspace, neopets.
Peer production: flips traditional production economics.
physical peer production: toolmakers, manufacturer, designer, aggregator, printer, user.
Internet fabrication: electronic, materials
Home fabrication: 3d scanners, milling and printing, 2d printing and cutting
Local fabrication: access to machine shops, equipment
Is physical better than digital?
Is this sustainable?
How do we design for casual craft?
Malcolm: biggest criticism of digital ground is the longevity of the tools, the tools are not consistent or stable enough.
Mike: people don’t like to do something they do for a living in their spare time: this might be a reaction away digital work environments.
John: scrapbooking industry is huge in the States: manifesting a collection of pieces: in Target is a whole section for scrapbooking.
Slowness is the point of craft. The process is key.
Anne: when you work in a sweatshop you don’t knit for relaxation.
Liz: … as long as it’s pink. Modern idea of design is strongly opposed to the Victorian ‘crafted’ fussy environments.

h3. Jyri Engstrm & Ulla-maaria Mutanen

Social objects, invisible tail and free product codes.
A map of relationships, with nodes as individuals.
But look at interactions from an ethnographic pov, we see interaction through social objects.
Michael Tomasello: children learn about intentional affordances based on understanding the intentional relations to other people.
Online these things are links, photos, etc.
Developing novel kinds of interactions through objects.
Photos, bookmarks, blogs, products.
Tim O’Reilly: Amazon also introduced their own proprietary identifier: the ASIN: works for objects that don’t have ISBN. A basic necessity for anything to become a social object online.
UPC, EPC, ISBN, ASIN for artefacts of mass production.
What might be a last.fm for physical things?
History, present, makers, materials. Making those relations visible: indicating social value.
Using RFID as a personal tool.
To-do list
Recommendation-based markets
Free thing identifiers
Available wireless access
Simple terminal devices

h3. Matt Webb

Forms of address: how we talk to computers, and how we should.
URLs and point and click are bad.
Desktop search and recent items: help a lot
Global URLs: they are portable, excellent.
They let you see ideas within their structure, not other ideas.
Doug Engelbart: online system
Delicious encodes a certain type of behaviour with more persistence and history.
Recent call lists on mobile phone
Implicature.
Using the Ning database is more like a conversation than Spotlight, it makes assumptions around structured data.
Patterns of behaviour: ‘the things I’ve done in the last 30 seconds can be referenced as an object”.
Bookmarking a conversation.
Mr Messy
“I’m only a first time user once”
Mike: Predictive shell.

h3. Nurri Kim

Tokyo Blues: The city as seen through one material
Blue tarpaulen: used in many different ways across different cities.
Most extensive use of blue tarps in Japan: anything that is passing into or out of existence.
Blue tarps with a modular system: model number is higher with durability
Most popular model number 3000, conforms to the tatami mat size.
Aerial photo: everything is being built, or rebuilt, historical renovation.
Used as shelter for homeless, mat for picnics, covers for temporary objects, street vendors, covering floors, spreading tarps to indicate zones: becomes an indoor space: taking off shoes to enter an ‘indoor zone’. Concealing / covering. Stack of things.

h3. Jack Schulze

Screen based button.
Working with manufacturing engineers
Physical buddies
The Invisibles
Desolation Jones
Will Burtin
What we can get away with in domestic manufacture.
Phones: manufacture, style/form: self reinforcement
Manufacture: iterating this relationship towards cheaper, assume this is better.
Manufacture: Worskshops : craft, local, milling, rfp
Style: malice, opportunism, dissassembly, individual, cults, play, luxury
Metal: Lens 117, casting metal in wax.
Casting phones in metal.

h3. Rgine Debatty

Artists or designers using RFID technology.
RFID was initially frightening.
Some consumer focused uses
RFID sushi: underneath each plate is a tag, conveyer belt. RFID reader scans a stack of plates.
For tracking cows.
In toys: a doll that gets sick, each accessory is a tag, in the toy is a reader.
Privacy
Zapped: a machine that lets you know about readers in the space. Gave a cockroach with RFIDs to every participant: release in the nearest walmart to taint the database.
How artists and designers are using it
A1 lounge: re-materialisation, prada store.
Digital wardrobe: making sure she uses things that people don’t see more than once.
RFID habitat: tables in two parts of the world, shows presence
RFID bootleg objects: mp3 player based on a vinyl turntable, using the old covers as interface to mp3s.
Deal me in: cards for poker: blackjack mat that plugs into the computer: seeing and printing pictures using playing cards, a traditional interface.
Moo-pong
With hidden numbers: traditional fashion objects, by passing these objects through a reader you can play music and play samples
Junkie helper: medicine linked to a chatroom: where people in a chatroom can see your action.
Go-dance: projecting videos in nightclubs, but can DJ using embodied interactions, with RFIDs connected to clothing
Used.co.at
Peripheral needs: Velcro is used to turn appliances in the home on and off. The paper tags can be used to keep track of what’s going on in the home.
Urban eyes.
The living room: Telling stories with rfid

h3. Liz Goodman

Physical fitness
Ecosystem enabling.
Dance Dance revolution incorporating fitness into gaming.
The ‘Pay per play’ model
Arcades are designed to be really fun.

h3. Eric Rodenbeck, Michal Migurski, Stamen

Dumb Gestures
Vito Acconci: our work has to sound exciting over the phone
Dumb: easy explanations: eg. Flickr
Meet the audience more than halfway
Dumb is difficult
Map of the market: martin wattenburg
Wordcount.org, babyname voyager, visualising the way we use language
Vox Delicii
Newsmap
Graffitiarchaeology
Tenbyten.org
Acconci studio: website based on tags: can build dynamic presentations on the fly.
Cabspotting: live taxi visualisation
Molly: data fetishism: what types of things can be done with this as a tool?
Thomas: difficulty with time on the web, nothing that deals with it well: everything stuck on a calendar.
David Gelertner: lifestreams
How can we make this more ugly: graphic ugliness is equated with truth, if anything is to be taken seriously.
Malcolm: too much visualisation as wallpaper

h3. Molly Steenson

With re-unification, an entire class of everyday markers was wiped out.

h3. John Poisson

Seeking nirvana in design for the tiny
Four mobile truths
The mobile world is in a mess: earliest days of the PC.
Intense and competing constraints.
How to design for a form factor that is so limited.
Implementation is so limited by standards. Onerous restrictions and suffering with dealing with carriers. Onerous business models.
But it’s a tremendous opportunity, billions of potential users.
To assuage anxieties over all this: constraints can lead to a level of purity. A mobile app is a bit like a haiku.
A zen approach to the design and implementation of these things.
Refinement towards a very specific set of constraints.

h3. Ben Cerveny

Hyperdimensional hopscotch
Ideas put forward around the possibilites of dystopian futures.
Structure and architecture loom large in our minds (even though our titles are not that enymore)
Information structures are starting to play out in the spaces we inhabit: in the construction of the community.
In the process of game design: commodification of play, play creates interfaces in a richer interface than the traditional process of interface design. (through play finding the boundaries of the self and the other).
Use of symbolic systems and projection of language onto the world.
Game vs play: game is projecting a model into play. Language formalises meaning, games formalise play.
Hopskotch has no real ingredients, apart from a method of making marks and a space (plus an agreement to play the game)
This agreement is where the power is.
Consensual modelling capabilities: emergent social properties.
People have a growing literacy of what is possible.
Chess: a sculpture of behaviour of the pieces.
Aware of the tools to make this kind of models.
Goal directed learning: role-playing is more like improvisation: the roles involved are not defined in silos, but interwoven. Multiplications of capabilities. Certain tasks can only be achieved through a sequence of collaborative actions. Cumulative actions.
The techniques of role-play have made it into the workplace.
Neverendingness: a distributed conversation about the building of ‘what can happen’. Submitting ideas into a collaborative space.
The hopskotch that we are playing, is a directed but temporary structure.
The smooth space that we are standing on, that we demarcate with some markings.
Building models that have a linkable state.
The game we are playing is re-projectable onto different circumstances. (in case of dire future possibilities).
Games gone native: in specific environments.
Temporary autonomous games (tag). Chalk on sidewalk, but chalk is washed away. Temporary collaborative structures of meaning: playing out actions in that space, the product is meaningful. Focus taxonomy onto a specific process, that has traction, but can then discard the rule-set, once we have a product.
In betweening (Aldo van Eyck). Became uncomfortable with the top-down projections of authority (high modernism). Built 700 playgrounds in Amsterdam.
Projecting the possibility of play onto the environment. Filled all of the gaps in Amsterdam with play. A sandbox here, a pole there, etc.
A divergence from the monolithic flows of high modernism.
Collaborative making of urban experience.
‘We are all here now’, feeling. Take different roles in the game that is more empowering towards our ability to make things in the city.
Beaurau of spatial organisational.
Before we can activate a city (with a platform) we need to begin to compose our personal, fragmented understandings, how things can be represented, things that we can broadcast to each other.
These things can be connected to the city, a living model of the city. Building the hopscotch squares from one to the other.
The game model has the flexibility to support the kind of collaborative activity that will support these things.
Matt: how to manage the politicization of artefacts. What are the political consequences of these ideologies, structures, games. A game about the making of the game.
Adam: John Zorn’s cobra: improvisational combat jazz. Disruptor.
Mike: open ended, play-based perspective: changing and following rules is very complex, and leads down paths that are very damaging. But by sticking to one set of rules might end with a better result.
Collaborative idealism is a constant flow of articulation and re-articulation. Laminar flow and turbulence. Turn up the flow of collaboration we also get ‘stop and go’ effects.
Game: flex: so many rulecards have been played that it’s impossible.

Load More