h3. First impressions
Overall the interaction between phone and RFID tags has been good. The reader/writer is on the base of the phone, at the bottom. This seems a little awkward to use at first, but slowly becomes natural. When I have given it to others, their immediate reaction is to point the top of the phone to the tag, and nothing happens. There follows a few moments of explaining as the intricacies of RFID and looking at the phone, with it’s Nokia ‘fingerprint’ icon. As phones increasingly become replacements for ‘contactless cards’, it seems likely that this interaction will become more habitual and natural.
Once the ‘service discovery’ application is running, the read time from tags is really quick. The sharp vibrations and flashing lights add to a solid feeling of interacting with something, both in the haptic and visual senses. This should turn out to be a great platform for embodied interaction with information and function.
The ability to read and write to tags makes it potentially adaptive as a platform wider than just advertising or ticketing. As an interaction designer I feel quite enabled by this technology: the three basic functions (making phonecalls, going to URLs, or sending SMSs) are enough to start thinking about tangible interactions without having to go and program any Java midlets or server-side applications.
I’m really happy that Nokia is putting this technology into a ‘low-end’ phone rather than pushing it out in a ‘smartphone’ range. This is where there is potential for wider usage and mass-market applications, especially around gaming and content discovery.
I had some problems launching the ‘service discovery’ application. Sometimes it works, sometimes it doesn’t and it’s difficult to tell why this is. It would be great to be able to place the phone on the table, knowing that it will respond to a tag, but it was just a little too unreliable to do that without checking to see that it had responded. The version I have still says it’s a prototype, so this may well be sorted out by the released version.
Overall there is a lack of integration between the service discovery application and the rest of the system: Contacts, SMS archive and service bookmarks etc. At the moment we need to enter the application to write and manage tags, or to give a ‘shortcut’ to another phone, but it seems that, as with bluetooth and IR, this should be part of the contextual menus that appear under ‘Options’ within each area of the phone. There are also some infuriating prompts that appear when interacting with URL, more details below.
p(caption). The phone opens the ‘service discovery’ application whenever it detects a compatible RFID tag near the base of the phone (when the keypad lock is off). This part is a bit obscure: sometimes it doesn’t ‘wake up’ for a tag, and the application needs to be loaded before it will read properly. Once the application is open (about 2-3 seconds) the read time of the tags seems instantaneous.
p(caption). The shortcuts menu gives access to shortcuts. Confusingly, this is different from ‘bookmarks’ and the ‘names’ list on the phone, although names can be searched from within the application. I think tighter integration with the OS is called for.
p(caption). Shortcuts can be added, edited, deleted, etc. in the same way as contacts. They can be ‘Given’ to another phone or ‘Written’ to a tag.
p(caption). There are three kinds of shortcuts: Call, URL or SMS. ‘Call’ will create a call to a pre-defined number, ‘URL’ will load a pre-defined URL, and ‘SMS’ will send a pre-defined SMS to a particular number. This part of the application has the most room for innovative extensions: we should be able to set the state of the phone, change profiles, change themes, download graphics, etc. This can be achieved by loading URLs, but URLs and mobiles don’t mix, so why should we be presented with them, when there could be a more usable layer inbetween? There could also be preferences for prompts: at the moment each action has to be confirmed with a yes or a no, but in some secure environments it would be nice to be able to have a function launched without the extra button push.
p(caption). If a tag contains no data, then we are notified and placed back on the main screen (as happened when I tried to write to my Oyster card).
p(caption). If the tag is writeable we are asked which shortcut to write to the tag.
p(caption). When we touch a tag with a shortcut on it, a prompt appears asking for confirmation. This is a level of UI to prevent mistakes, and a certain level of security, but it also reduces the overall usability of the system. With URL launching, there are two stages of confirmation, which is infuriating. There needs to be some other mode of confirmation, and the ‘service discovery’ app needs to somehow be deeper in the system to avoid these double button presses.
p(caption). Lastly, there is a log of actions. Useful to see if the application has been reading something in your bag or wallet, without you knowing…
This work explores the visual link between information and physical things, specifically around the emerging use of the mobile phone to interact with RFID or NFC. It was a presentation and poster at Design Engaged, Berlin on the 11th November 2005.
As mobile phones are increasingly able to read and write to RFID tags embedded in the physical world, I am wondering how we will appropriate this for personal and social uses.
I’m interested in the visual link between information and physical things. How do we represent an object that has digital function, information or history beyond it’s physical form? What are the visual clues for this interaction? We shouldn’t rely on a kind of mystery meat navigation (the scourge of the web-design world) where we have to touch everything to find out it’s meaning.
This work doesn’t attempt to be a definitive system for marking physical things, it is an exploratory process to find out how digital/physical interactions might work. It uncovers interesting directions while the technology is still largely out of the hands of everyday users.
h3. Reference to existing work
p(caption). Click for larger version.
The inspiration for this is in the marking of public space and existing iconography for interactions with objects: push buttons on pedestrian crossings, contactless cards, signage and instructional diagrams.
This draws heavily on the substantial body of images of visual marking in public space. One of the key findings of this research was that visibility and placement of stickers in public space is an essential part of their use. Current research in ubicomp and ‘locative media’ is not addressing these visibility issues.
There is also a growing collection of existing iconography in contactless payment systems, with a number of interesting graphic treatments in a technology-led, vernacular form. In Japan there are also instances of touch-based interactions being represented by characters, colours and iconography that are abstracted from the action itself.
Sketching and development revealed five initial directions: circles, wireless, card-based, mobile-based and arrows (see the poster for more details). The icons range from being generic (abstracted circles or arrows to indicate function) to specific (mobile phones or cards touching tags).
Arrows might be suitable for specific functions or actions in combinations with other illustrative material. Icons with mobile phones or cards might be helpful in situations where basic usability for a wide range of users is required. Although the ‘wireless’ icons are often found in current card readers, they do not successfully indicate the touch-based interactions inherent in the technology, and may be confused with WiFi or Bluetooth. The circular icons work at the highest level, and might be most suitable for generic labelling.
For further investigation I have selected a simple circle, surrounded by an ‘aura’ described by a dashed line. I think this successfully communicates the near field nature of the technology, while describing that the physical object contains something beyond its physical form.
In most current NFC implementations, such as the 3220 from Nokia and many iMode phones, the RFID reader is in the bottom of the phone. This means that the area of ‘activation’ is obscured in many cases by the phone and hand. The circular iconography allows for a space to be marked as ‘active’ by the size of the circle, and we might see it used to mark areas rather than points. Usability may improve when these icons are around the same size as the phone, rather than being a specific point to touch.
h3. Work in progress
This is early days for this technology, and this is work-in-progress. There is more to be done in looking at specific applications, finding suitable uses and extending the language to cover other functions and content.
Until now I have been concerned with generic iconography for a digitally augmented object. But this should develop into a richer language, as the applications for this type of interaction become more specific, and related to the types of objects and information being used. For example it would be interesting to find a graphic treatment that could be applied to a Pokemon sticker offering power-ups as well as a bus stop offering timetable downloads.
I’m also interested in the physical placement of these icons. How large or visible should they be? Are there places that should not be ‘active’? And how will this fit with the natural, centres of gravity of the mobile phone in public and private space.
I’ll expand on these things in a few upcoming projects that explore touch-based interactions in personal spaces.
Feel free to use and modify the icons, I would be very interested to see how they can be applied and extended.
h3. Visual references
Oyster Card, Transport for London.
eNFC, Inside Contactless.
ExpressPay, American Express.
MiFare, various vendors.
Suica, JR, East Japan Railway Company.
RFID Field Force Solutions, Nokia.
NFC shell for 3220, Nokia.
ERG Transit Systems payment, Dubai.
Various generic contactless vendors.
Contactless payment symbol, Mastercard.
Open Here, Paul Mijksenaar, Piet Westendorp, Thames and Hudson, 1999.
Understanding Comics, Scott McCloud, Harper, 1994
h3. Intro by Andrew Otwell
Restorative feeling of getting together.
The architecture of participation.
Experience design is participation design.
h3. Adam Greenfield
Ethical and social implications of ubicomp
Has been reading far afield “Out of gas” “Dark age ahead” “Collapse
Growth and decline
Models of growth:
1. Health of economy is based on growth
2. Continuous growth: technological singularity: going off the charts
Decline: cyclical decline, populations, rise and fall of the nile
But sometime it’s not cyclical: the global production of oil.
Growth and terminal decline: global oil production. Industrial civilisation becomes very hard to maintain.
There is a contingency that designers probably ought to be thinking about.
An implication: things get intensely local
Power sources, reclamation, working and forging
Textiles dyes colorings
Methods practices and standards: knowledge: resident in the minds and things around us.
Deployment of finished items
Value: ie copper may be worth more locally as a metal to be worked, than as wiring in an inert grid.
A new suite of design gestures
Our near-instantaneous google+wikipedia
Flexibility and unquestioned connectivity (the social, fluid, soft meeting space)
digital visualization and rapid prototyping
Petrochemical or dependent derived material properties
Low material costs
The global community
Will become important
Sense of possibility and play
Reclamation of old industrialisation
Re-densification: new urban centres
Urban core may be untenable: bulding higher than 6 floors may be difficult
KH Kunstler: the long emergency
h3. Matt Ward
Has design lost its way.
Design is always ideological, influenced by specific world views, responds to cultural context, design is a product (of the society it originates from).
It’s also world shaping, future oriented, not yet, soon to be, one day, wouldn’t it be cool if. Imagination and production. Cannot claim autonomy from social change.
Utopia, Outopia: no place. Eutopia: good place
The desire to be somewhere different, not about this place not now, free of the problems of this world.
No problem to link utopianism with the act of designing.
Many see design as a key to changing the world. Corbusier.
Design is also linked to control and power.
Archigram: a move away from conservatism. Anti-utopianism, learning from the mistakes of their forefathers.
Inbetween the space of freedom and control: metamorphosis
Utopia of difference: terroristic meta-narratives (Tom Moylan)
Superstudio: allows for different uses and actions to define the architecture of the studio. A critique of the modernists.
Critical utopia: asks questions: asks towards change, towards critical design.
Problem with critical design: forgets to question the position that the designer has
Locative media: a lot of technology removes us from everyday life, we need to remember the dogshit and the chewing gum.
Malcolm: the avant garde is now 2 minutes.
Digital design: change in space and place: information acts within the production of space.
h3. Joshua Kaufman
Book: The Whale and the Reactor (A search for limits in an age of high technology)
Social consequences of design from a historical perspective
Artefacts have political properties: invention, design or arrangement of artifact becomes a way of settling an issue in the affaris of a community. Some articles are inherently political (weaponry)
Political design can be found in architecture and urban planning
Robert Moses low overpasses: discouraged lower-class travel
Barron Haussmann’s thoroughfares: large roads and parks.
Concrete buildings and plazas at universities, to discourage demonstrations
Cyrus McCorrmick II’s molding machines
How design choices affect the relative distribution of power.
Fabio: Relationship between artefact and culture is complex. You can put the baby out into the world, but can’t take responsibility for all it’s actions.
h3. Anne Galloway
Design and the parliament of things
Making the work of a social scientist applicable
Philosophy of science
The difference between an assemblage and an assembly
“To assemble is one thing; to represent to the eyes and ears of those assembled what is at stake is another” Bruno Latour, How to Make Things Public.
The different between objects and things
In design, things are objects: stable and neutral. Even users are objects.
Things: an assembly: to come together
A coming together precicely because we have differences.
From Realpolitics to Thingpolitics.
“Walking is controlled falling” Laurie Andersen.
Representative democracy: body-politic: unifying difference, stabilising and reducing difference.
From representation to re-presentation: all the constituent parts
Re-presenting design: invention and capitalism, speed and mobilisation.
Going from an assemblage of different parts to an assembly.
Speed and competition, focusing on the novel: ideas and materials and practices are being mobilised, getting rid of the obstacles that will slow it down.
This doesn”t help us ‘assemble’. A constant space of speed, not encouraged to slow down and think things through. We don’t have an assembly.
We can’t afford to keep ignoring each other.
“Your concerns are not relevant to our task at hand, you are not enabling my efficiency, etc.”
What is it going to take to design in this parliament of things?
We can start by slowing down.
“That’s where slowing down comes in – you can create new habits only by slowing down, because new habits also mean new feelings, new interests, new possibilities” – Isabelle Stengers
Matt: voting is a last resort, consensus is reached in different ways.
Anne: not interested in consensus, interested in convergence.
Malcolm: taking into account, not accounting the bottom line: Latour
Matt: strategies and tactics in moving towards slowness? A: get outside, get into the ground.
h3. Michelle Chang
Public by design
Interested in the issue of privacy
Privacy / Publicness
How the library became a microcosm of the city: a number of services and interfaces to the city
What technologies are public goods? Electricity, wireless, etc.
Which technologies must be individually wrought? Laptops, mobiles.
“A public institution must bear the burden of public demands”
Anne: a public space or a commons? A convergence
Use of the internet problematic: an increasing use of “adult terminals” versus “children’s terminals”.
The library becomes a sort of pitstop: getting out of the rain or using the toilet.
h3. Thomas Vander Wal
Clouds, space and black boxes
Information is found and created, but there is little distinction between the two.
Personal info cloud, local infocloud, global infocloud
A clearing house of filters from the flood of information that flows down.
Filtering based on trust and value: aggregation
A ‘firewall’ that filters what information we give to people
A ‘smart’ black box
Some people keep all different bits of their personal data stored underneath different usernames: so as to not keep all data aggregated in one place.
h3. Louise Klinker
Using environmental issues. Using P2P software
Linking music download to criminal activity, but also tracking music downloads according to the amount of money owed to industry and artists.
h3. Malcolm McCullough
Taking into account the notion of civic space: architecture.
Closed laptop, notebook…
h3. Fabio Sergio
The skin of objects
No definitions of interaction design!
Where is the material of interaction design?
Joy Mountfort: “We are designing the skin of objects” reacted strongly against this.
Industrial design: from Olivetti cash register to Apple G5.
Ben: models are the substance of interaction design. The act of creating constraints and systemic understandings.
The visualisation of complexity: making new materials from the mass of information.
Conceptual models affecting the physical models.
Liz: what are models? Fundamentals of interaction design are stories.
h3. Stefan Smagula
Open source media
Project Lightspeed: AT&T, Yahoo and Microsoft.
Big players: Apple, IPTV, Yahoo, Google
Bootstrappers: Open Media Network, DTV (Bittorrent & RSS), Open Source Radio
RSS provides metadata and triggers downloads
Matt Jones: Jerry Cornelius: an open source character: giving away plots and stories.
h3. Chris Heathcote
Panasonic gets personalisation.
Barcodes on skins.
Websites for cover creation.
Harajuku store for expensive personal covers.
Korea: metal depicting war scenes, cloth.
A spectrum of engagement, consume, buy, customise, accessorize, alter, make, design.
Find out how, make stuff, sell stuff. cyworld (half population of Korea?), habbo, myspace, neopets.
Peer production: flips traditional production economics.
physical peer production: toolmakers, manufacturer, designer, aggregator, printer, user.
Internet fabrication: electronic, materials
Home fabrication: 3d scanners, milling and printing, 2d printing and cutting
Local fabrication: access to machine shops, equipment
Is physical better than digital?
Is this sustainable?
How do we design for casual craft?
Malcolm: biggest criticism of digital ground is the longevity of the tools, the tools are not consistent or stable enough.
Mike: people don’t like to do something they do for a living in their spare time: this might be a reaction away digital work environments.
John: scrapbooking industry is huge in the States: manifesting a collection of pieces: in Target is a whole section for scrapbooking.
Slowness is the point of craft. The process is key.
Anne: when you work in a sweatshop you don’t knit for relaxation.
Liz: … as long as it’s pink. Modern idea of design is strongly opposed to the Victorian ‘crafted’ fussy environments.
h3. Jyri Engström & Ulla-maaria Mutanen
Social objects, invisible tail and free product codes.
A map of relationships, with nodes as individuals.
But look at interactions from an ethnographic pov, we see interaction through social objects.
Michael Tomasello: children learn about intentional affordances based on understanding the intentional relations to other people.
Online these things are links, photos, etc.
Developing novel kinds of interactions through objects.
Photos, bookmarks, blogs, products.
Tim O’Reilly: Amazon also introduced their own proprietary identifier: the ASIN: works for objects that don’t have ISBN. A basic necessity for anything to become a social object online.
UPC, EPC, ISBN, ASIN for artefacts of mass production.
What might be a last.fm for physical things?
History, present, makers, materials. Making those relations visible: indicating social value.
Using RFID as a personal tool.
• Recommendation-based markets
• Free thing identifiers
• Available wireless access
• Simple terminal devices
h3. Matt Webb
Forms of address: how we talk to computers, and how we should.
URLs and point and click are bad.
Desktop search and recent items: help a lot
Global URLs: they are portable, excellent.
They let you see ideas within their structure, not other ideas.
Doug Engelbart: online system
Delicious encodes a certain type of behaviour with more persistence and history.
Recent call lists on mobile phone
Using the Ning database is more like a conversation than Spotlight, it makes assumptions around structured data.
Patterns of behaviour: ‘the things I’ve done in the last 30 seconds can be referenced as an object”.
Bookmarking a conversation.
“I’m only a first time user once”
Mike: Predictive shell.
h3. Nurri Kim
Tokyo Blues: The city as seen through one material
Blue tarpaulen: used in many different ways across different cities.
Most extensive use of blue tarps in Japan: anything that is passing into or out of existence.
Blue tarps with a modular system: model number is higher with durability
Most popular model number 3000, conforms to the tatami mat size.
Aerial photo: everything is being built, or rebuilt, historical renovation.
Used as shelter for homeless, mat for picnics, covers for temporary objects, street vendors, covering floors, spreading tarps to indicate zones: becomes an indoor space: taking off shoes to enter an ‘indoor zone’. Concealing / covering. Stack of things.
h3. Jack Schulze
Screen based button.
Working with manufacturing engineers
What we can get away with in domestic manufacture.
Phones: manufacture, style/form: self reinforcement
Manufacture: iterating this relationship towards cheaper, assume this is better.
Manufacture: Worskshops : craft, local, milling, rfp
Style: malice, opportunism, dissassembly, individual, cults, play, luxury
Metal: Lens 117, casting metal in wax.
Casting phones in metal.
h3. Régine Debatty
Artists or designers using RFID technology.
RFID was initially frightening.
Some consumer focused uses
RFID sushi: underneath each plate is a tag, conveyer belt. RFID reader scans a stack of plates.
For tracking cows.
In toys: a doll that gets sick, each accessory is a tag, in the toy is a reader.
Zapped: a machine that lets you know about readers in the space. Gave a cockroach with RFIDs to every participant: release in the nearest walmart to taint the database.
How artists and designers are using it
A1 lounge: re-materialisation, prada store.
Digital wardrobe: making sure she uses things that people don’t see more than once.
RFID habitat: tables in two parts of the world, shows presence
RFID bootleg objects: mp3 player based on a vinyl turntable, using the old covers as interface to mp3s.
Deal me in: cards for poker: blackjack mat that plugs into the computer: seeing and printing pictures using playing cards, a traditional interface.
With hidden numbers: traditional fashion objects, by passing these objects through a reader you can play music and play samples
Junkie helper: medicine linked to a chatroom: where people in a chatroom can see your action.
Go-dance: projecting videos in nightclubs, but can DJ using embodied interactions, with RFIDs connected to clothing
Peripheral needs: Velcro is used to turn appliances in the home on and off. The paper tags can be used to keep track of what’s going on in the home.
The living room: Telling stories with rfid
h3. Liz Goodman
Dance Dance revolution incorporating fitness into gaming.
The ‘Pay per play’ model
Arcades are designed to be really fun.
h3. Eric Rodenbeck, Michal Migurski, Stamen
Vito Acconci: our work has to sound exciting over the phone
Dumb: easy explanations: eg. Flickr
Meet the audience more than halfway
Dumb is difficult
Map of the market: martin wattenburg
Wordcount.org, babyname voyager, visualising the way we use language
Acconci studio: website based on tags: can build dynamic presentations on the fly.
Cabspotting: live taxi visualisation
Molly: data fetishism: what types of things can be done with this as a tool?
Thomas: difficulty with time on the web, nothing that deals with it well: everything stuck on a calendar.
David Gelertner: lifestreams
How can we make this more ugly: graphic ugliness is equated with truth, if anything is to be taken seriously.
Malcolm: too much visualisation as wallpaper
h3. Molly Steenson
With re-unification, an entire class of everyday markers was wiped out.
h3. John Poisson
Seeking nirvana in design for the tiny
Four mobile truths
The mobile world is in a mess: earliest days of the PC.
Intense and competing constraints.
How to design for a form factor that is so limited.
Implementation is so limited by standards. Onerous restrictions and suffering with dealing with carriers. Onerous business models.
But it’s a tremendous opportunity, billions of potential users.
To assuage anxieties over all this: constraints can lead to a level of purity. A mobile app is a bit like a haiku.
A zen approach to the design and implementation of these things.
Refinement towards a very specific set of constraints.
h3. Ben Cerveny
Ideas put forward around the possibilites of dystopian futures.
Structure and architecture loom large in our minds (even though our titles are not that enymore)
Information structures are starting to play out in the spaces we inhabit: in the construction of the community.
In the process of game design: commodification of play, play creates interfaces in a richer interface than the traditional process of interface design. (through play finding the boundaries of the self and the other).
Use of symbolic systems and projection of language onto the world.
Game vs play: game is projecting a model into play. Language formalises meaning, games formalise play.
Hopskotch has no real ingredients, apart from a method of making marks and a space (plus an agreement to play the game)
This agreement is where the power is.
Consensual modelling capabilities: emergent social properties.
People have a growing literacy of what is possible.
Chess: a sculpture of behaviour of the pieces.
Aware of the tools to make this kind of models.
Goal directed learning: role-playing is more like improvisation: the roles involved are not defined in silos, but interwoven. Multiplications of capabilities. Certain tasks can only be achieved through a sequence of collaborative actions. Cumulative actions.
The techniques of role-play have made it into the workplace.
Neverendingness: a distributed conversation about the building of ‘what can happen’. Submitting ideas into a collaborative space.
The hopskotch that we are playing, is a directed but temporary structure.
The smooth space that we are standing on, that we demarcate with some markings.
Building models that have a linkable state.
The game we are playing is re-projectable onto different circumstances. (in case of dire future possibilities).
Games gone native: in specific environments.
Temporary autonomous games (tag). Chalk on sidewalk, but chalk is washed away. Temporary collaborative structures of meaning: playing out actions in that space, the product is meaningful. Focus taxonomy onto a specific process, that has traction, but can then discard the rule-set, once we have a product.
In betweening (Aldo van Eyck). Became uncomfortable with the top-down projections of authority (high modernism). Built 700 playgrounds in Amsterdam.
Projecting the possibility of play onto the environment. Filled all of the gaps in Amsterdam with play. A sandbox here, a pole there, etc.
A divergence from the monolithic flows of high modernism.
Collaborative making of urban experience.
‘We are all here now’, feeling. Take different roles in the game that is more empowering towards our ability to make things in the city.
Beaurau of spatial organisational.
Before we can activate a city (with a platform) we need to begin to compose our personal, fragmented understandings, how things can be represented, things that we can broadcast to each other.
These things can be connected to the city, a living model of the city. Building the hopscotch squares from one to the other.
The game model has the flexibility to support the kind of collaborative activity that will support these things.
Matt: how to manage the politicization of artefacts. What are the political consequences of these ideologies, structures, games. A game about the making of the game.
Adam: John Zorn’s cobra: improvisational combat jazz. Disruptor.
Mike: open ended, play-based perspective: changing and following rules is very complex, and leads down paths that are very damaging. But by sticking to one set of rules might end with a better result.
Collaborative idealism is a constant flow of articulation and re-articulation. Laminar flow and turbulence. Turn up the flow of collaboration we also get ‘stop and go’ effects.
Game: flex: so many rulecards have been played that it’s impossible.
I too have “ditched”:http://interconnected.org/home/2005/04/12/my_40gb_ipod_has my large iPod for the “iPod Shuffle”:http://www.apple.com/ipodshuffle/, finding that “I love the white-knuckle ride of random listening”:http://www.cityofsound.com/blog/2005/01/the_rise_and_ri.html. But that doesn’t exclude the need for a better small-screen-based music experience.
The pseudo-analogue interface of the iPod clickwheel doesn’t cut it. It can be difficult to control when accessing huge alphabetically ordered lists, and the acceleration or inertia of the view can be really frustrating. The combinations of interactions: clicking into deeper lists, scrolling, clicking deeper, turn into long and tortuous experiences if you are engaged in any simultaneous activity. Plus its difficult to use through clothing, or with gloves.
h3. Music and language
My first thought was something “Jack”:http://www.jackschulze.co.uk and I discussed a long time ago, using a phone keypad to type the first few letters of a artist, album or genre and seeing the results in real-time, much like “iTunes”:http://www.apple.com/itunes/jukebox.html does on a desktop. I find myself using this a lot in iTunes rather than browsing lists.
“Predictive text input”:http://www.t9.com/ would be very effective here, when limited to the dictionary of your own music library. (I wonder if “QIX search”:http://www.christianlindholm.com/christianlindholm/2005/02/qix_from_zi_cor.html would do this for a music library on a mobile?)
Maybe now is the time to look at this as we see “mobile”:http://www.sonyericsson.com/spg.jsp?cc=gb&lc=en&ver=4000&template=pp1_loader&php=php1_10245&zone=pp&lm=pp1&pid=10245 “phone”:http://www.nokia.com/n91/ “music convergence”:http://www.engadget.com/entry/1234000540040867/.
h3. Navigating through movement
Since scrolling is inevitable to some degree, even within fine search results, what about using simple movement or tilt to control the search results? One of the problems with using movement for input is context: when is movement intended? And when is movement the result of walking or a bump in the road?
One solution could be a “squeeze and shake” quasi-mode: squeezing the device puts it into a receptive state.
Another could be more reliance on the 3 axes of tilt, which are less sensitive to larger movements of walking or transport.
I’m not sure about gestural interfaces, most of the prototypes I have seen are difficult to learn, and require a certain level of performativity that I’m not sure everyone wants to be doing in public space. But having accelerometers inside these devices should, and would, allow for the hacking together other personal, adaptive gestural interfaces that would perhaps access higher level functions of the device.
One gesture I think could be simple and effective would be covering the ear to switch tracks. To try this out we could add a light or capacitive touch sensor to each earbud.
With this I think we would have trouble with interference from other objects, like resting the head against a wall. But there’s something nicely personal and intimate about putting the hand next to the ear, as if to listen more intently.
h3. More knobs
Things that are truly analogue, like volume and time, should be mapped to analogue controls. I think one of the greatest unexplored areas in digital music is real-time audio-scrubbing, currently not well supported on any device, probably because of technical constraints. But scrubbing through an entire album, with a directly mapped input, would be a great way of finding the track you wanted.
Research projects like the “DJammer”:http://www.hpl.hp.com/research/mmsl/projects/djammer/ are starting to look at this, specifically for DJs. But since music is inherently time-based there is more work to be done here for everyday players and devices. Let’s skip the interaction design habits we’ve learnt from the CD era and go back to vinyl :)
h3. Evolution of the display
Where displays are required, I hope we can be free of small, fuzzy, low-contrast LCDs. With new displays being printable on paper, textiles and other surfaces there’s the possibility of improving the usability, readability and “glanceability” of the display.
We are beginning to see signs of this with this OLED display on this “Sony Network Walkman”:http://dapreview.net/comment.php?comment.news.1086 where the display is under the surface of the product material, without a separate “glass” area.
For the white surface of an iPod, the high-contrast, “paper-like surfaces”:http://www.polymervision.com/New-Center/Downloads/Index.html of technologies like e-ink would make great, highly readable displays.
So I really need to get prototyping with accelerometers and display technologies, to understand simple movement and gesture in navigating music libraries. There are other questions to answer: I’m wondering if using movement to scroll through search results would create the appearance of a large screen space, through the lens of a small screen. As with “bumptunes”:http://interconnected.org/home/2005/03/04/apples_powerbook, I think many more opportunities will emerge as we make these things.
h3. More reading
“Designing for Shuffling”:http://www.cityofsound.com/blog/2005/04/designing_for_s.html
“Thoughts on the iPod Shuffle”:http://interconnected.org/home/2005/04/22/there_are_two
“On the body”:http://people.interaction-ivrea.it/b.negrillo/onthebody/
These are some of my notes from Mikael FernstrÃ¶m‘s lecture at AHO.
The aim of the “Soundobject”:http://www.soundobject.org/ research is to liberate interaction design from visual dominance, to free up our eyes, and to do what small displays don’t do well.
Reasons for focusing on sound:
* Sound is currently under-utilised in interaction design
* Vision is overloaded and our auditory senses are seldom engaged
* In the world we are used to hearing a lot
* Adding sound to existing, optimised visual interfaces does not add much to usability
Sound is very good at attracting our attention, so we have alarms and notification systems that successfully use sound in communication and interaction. We talked about using ‘caller groups’ on mobile phones where people in an address book can be assigned different ringtones, and how effective it was in changing our relationship with our phones. In fact it’s possible to sleep through unimportant calls: our brains are processing and evaluating sound while we sleep.
One fascinating thing that I hadn’t considered is that sound is our fastest sense: it has an extremely high temporal resolution (ten times faster than vision), so for instance our ears can hear pulses at a much higher rate than our eyes can watch a flashing light.
h3. Disadvantages of sound objects
Sound is not good for continuous representation because we cannot shut out sound in the way we can divert our visual attention. It’s also not good for absolute display: pitch, loudness and timbre are relative to most people, even people that have absolute pitch can be affected by contextual sounds. And context is a big issue: loud or quiet environments affect the way that sound must be used in interfaces: libraries and airplanes for example.
There are also big problems with spatial representation in sound, techniques that mimic the position of sound based on binaural differences are inaccessible by about a fifth of the population. This perception of space in sound is also intricately linked with the position and movement of the head. “Some Google searches on spatial representation of sound”:http://www.google.com/search?&q=spatial+representation+of+sound. See also “Psychophysical Scaling of Sonification Mappings [pdf]”:http://sonify.psych.gatech.edu/publications/pdfs/2000ICAD-Scaling-WalkerKramerLane.pdf
‘Filling a bottle with water’ is a sound that could work as part of an interface, representing actions such as downloading, uploading or in replacement of progress bars. The sound can be abstracted into a ‘cartoonification’ that works more effectively: the abstraction separates simulated sounds from everyday sounds.
Mikael cites inspiration from “foley artists”:http://en.wikipedia.org/wiki/Foley_artist working on film sound design, that are experienced in emphasising and simplifying sound actions, and in creating dynamic sound environments, especially in animation.
A side effect of this ‘cartoonification’ is that sounds can be generated in simpler ways: reducing processing and memory overhead in mobile devices. In fact all of the soundobject experiments rely on parametric sound synthesis using “PureData”:http://www.puredata.org/: generated on the fly rather than using sampled sound files, resulting in small, fast, adaptive interface environments (sound files and the PD files used to generate the sounds can be found at the “Soundobject”:http://www.soundobject.org/ site).
One exciting and pragmatic idea that Mikael mentioned was simulating ‘peas in a tin’ to hear how much battery is left in a mobile device. Something that seems quite possible, reduced to mere software, with the accelerometer in the “Nokia 3220″:http://www.nokia.com/phones/3220. Imagine one ‘pea’ rattling about, instead of one ‘bar’ on a visual display…
h3. Research conclusions
The most advanced prototype of a working sound interface was a box that responded to touch, and had invisible soft-buttons on it’s surface that could only be heard through sound. The synthesised sounds responded to the movement of the fingertips across a large touchpad like device (I think it was a “tactex”:http://www.tactex.com/ device). These soft-buttons used a simplified sound model that synthesised _impact_, _friction_ and _deformation_. See “Human-Computer Interaction Design based on Interactive Sonification [pdf]”:http://richie.idc.ul.ie/eoin/research/Actions_And_Agents_04.pdf
The testing involved asking users to feel and hear their way around a number of different patterns of soft-buttons, and to draw the objects they found. See “these slides”:http://www.flickr.com/photos/timo/tags/soundobjects/ for some of the results.
The conclusions were that users were almost as good at using sound interfaces as with normal soft-button interfaces and that auditory displays are certainly a viable option for ubiquitous, especially wearable, computing.
h3. More reading
“Gesture Controlled Audio Systems”:http://www.cost287.org/
Notes on two related projects:
h2. 1. Time that land forgot
* A “project”:http://www.elasticspace.com/timeland/ in collaboration with Even Westvang
* Made in 10 days at the Icelandic locative media workshop, summer 2004
* Had the intention of making photo archives and gps trails more useful/expressive
* Looked at patterns in my photography: 5 months, 8000 photos, visualised them by date / time of day. Fantastic resource for me: late night parties, early morning flights, holidays and the effect of midnight sun is visible.
* Looking now to make it useful as part of more pragmatic interface, to try other approaches less about the abstracted visualisation
* “info, details, research and source code”:http://www.elasticspace.com/2004/07/timeland
* “time visualisation”:http://www.elasticspace.com/images/photomap_times_large.gif
h2. 2. Marking in urban public space
I’ve also been mapping stickering, stencilling and flyposting: walking around with the camera+gps and “photographing examples of marking”:http://www.flickr.com/photos/timo/sets/8380/ (not painted graffiti).
This research looks at the marking of public space by investigating the physical annotation of the city: stickering, stencilling, tagging and flyposting. It attempts to find patterns in this marking practice, looking at visibility, techniques, process, location, content and audience. It proposes ways in which this marking could be a layer between the physical city and digital spatial annotation.
h3. Some attributes of sticker design
* *Visibility*: contrast, monochromatic, patterns, bold shapes, repetition
* *Patina*: history, time, decay, degredation, relevance, filtering, social effects
* *Physicality*: residue of physical objects: interesting because these could easily contain digital info
* *Adaptation and layout*: layout is usually respectful, innovative use of dtp and photocopiers, adaptive use of sticker patina to make new messages on top of old
Layers of information build on top of each other, as with graffiti, stickers show their age through fading and patina, flyposters become unstuck, torn and covered in fresh material. Viewed from a distance the patina is evident, new work tends to respect old, and even commercial flyposting respects existing graffiti work.
Techniques vary from strapping zip-ties through cardboard and around lampposts for large posters, to simple hand-written notes stapled to trees, and short-run printed stickers. One of the most fascinating and interactive techniques is the poster offering strips of tear-off information. These are widely used, even in remote areas.
Initial findings show that stickers don’t relate to local space, that they are less about specific locations than about finding popular locations, “cool neighbourhoods” or just ensuring repeat exposure. This is opposite to my expectations, and perhaps sheds some light on current success/failure of spatial annotation projects.
I am particularly interested in the urban environment as an interface to information and an interaction layer for functionality, using our spatial and navigational senses to access local and situated information.
There is concern that in a dense spatially annotated city we might have an overload of information, what about filtering and fore-grounding of relevant, important information? Given that current technologies have very short ranges (10-30mm), we might be able to use our existing spatial skills to navigate overlapping information. We could shift some of the burden of information retrieval from information architecture to physical space.
I finished by showing this animation by Kriss Salmanis, a young Latvian artist. Amazing re-mediation of urban space through stencilling, animation and photography. (“Un ar reizi naks tas bridis” roughly translates as “And in time the moment will come”.
p(footnote). Graffiti Archaeology, Cassidy Curtis
p(footnote). Street Memes, collaborative project
p(footnote). Spatial annotation projects list
p(footnote). Nokia RFID kit for 5140
p(footnote). Spotcodes, High Energy Magic
p(footnote). ?Mystery Meat navigation?, Vincent Flanders
p(footnote). RDF as barcodes, Chris Heathcote
p(footnote). Implementation: spatial literature
p(footnote). Yellow Arrow
The workshop was organised by “Erich Berger”:http://randomseed.org/ (of “7 Mile Boots”:http://randomseed.org/sevenmileboots/ fame) who brought in Helen Evans & Heiko Hansen of “HeHe”:http://www.hehe.org/ to give context and direction to the technical process.
My intention was to avoid the screen for the duration of the workshop, to concentrate on simple interactions between sensors and outputs entirely independent of a desktop computer. But I ended up staring at microprocessor programming languages like “PBasic”:http://stage.itp.nyu.edu/~tigoe/pcomp/stamp/stamp-programming.shtml and “JAL”:http://sourceforge.net/projects/jal while making lots of LEDs blink.
A lot of it brought back memories of school; circuit diagrams, resistance calculations, it was great to refresh the memory. We spent a lot of time translating circuit diagrams onto breadboards, and programming both “PIC”:http://stage.itp.nyu.edu/~tigoe/pcomp/pic/index.shtml and “Basic Stamp”:http://stage.itp.nyu.edu/~tigoe/pcomp/stamp/index.shtml microprocessors.
Erich is now setting up a Physcomp lab at Atelier Nord to support art/design projects in Oslo, maybe alongside some regular meetings (entitled Atelier Nerd :). There are many projects that I would like to pursue, this should be a great resource.
Outside In is a forum for involving new voices, media and practices in a discourse about the use and design of public space. It took place from 14 – 15 June 2004.
Roda Sten is amazing, below a suspension bridge, with huge concrete creations. Really windy, but calm inside the lecture space. Here are my notes and a few pictures.
h2. Day 1
h3. Session 2: Hacking the streets (I missed the 1st workshop)
h3. Space Hijackers
* Putting memories in spaces: spaces arent the same after having been disrupted. after ‘reclaim the streets’ or a ‘circle line party’ you can’t see the space in the same way.
* Distinction between public and private. What is it?
* Public space doesn’t exist anymore.
* Ken’s new city hall is half private half public (private investment was involved in the building, so protests cannot happen outside)
* Do we need institutions in order to do events, is that the only way to do it legally?
* What’s stopping people from doing these things is not necessarily capitalism, but the fear of looking like a pillock: self-regulation is a big factor. Can spark things to let down inhibitions or shackles. Uses example of the scooter, became a kids toy and then it wasn’t cool anymore.
* What’s the connection between anarchism and these spontaneous events. Emergent order is interesting, so much control over actions, and the ways people move through the city. How does this relate to anarchy? Is this anarchy?
* The city is a workshop: not just walls to tag
* Shadows of urban furniture: really good
* Visual kidknapping: Lavazza woman gets cut out of the frame
* Big poster with bleeding eyes
* Uses a high pressure water jet to clean the city, but also write at the same time.
* Digs at the notion of authorship, a site where people find work on the streets
* The work is anonymous, but there is the projection of authorial control behind it, its individual and definitely authored
* Would be interesting to explore more about Graffiti authorship: how do public artists want to be recognised?
* Managing the mystique around the work and the author.
* Difference between author/instigator
* “Visual kidknapping”:http://www.visual-kidnapping.org/
h3. 3D bombing: Akim
* Polystyrene models, matched to fit specific city spaces
* City of names: what if the writers are the ones who build the houses?
h2. Day 2
h3. Session 3: Network experience
h3. “Jonah Brucker Cohen”:http://coin-operated.com/
* Wants to deconstruct network context
* Context: physical and social situation in which computation sits
* How does the network affect the output and experience
* Companies are claiming ownership of space because of signal
strength: strengthening signals to drown out free competion
* WiFihog: saps out all wifi bandwidth
* LAN party versus Flash Mob
* Simpletext: collaborative sms image searching on large screens
* re-mapping and changing the context of interfaces: what about
shifting consequences: changing the input/output relationship.
* Simpletext project: assigns an image search to inputted text
messages, and displays via jitter/max on a large screen.
* Steven Levy quote on hackers
h3. “Katherine Moriwaki”:http://kakirine.com/
* Altering space by altering the body
* character of a space
* remnants of things, people, individuals
* put magnets on wrists and fingers and bodies to reveal the proximity of electronic devices: unexpected connections to other people and lampposts. Nice.
h3. Data Climates: Pedro Sepúlveda Sandoval
* Living in a scanscape city
* electronic space, synthetic city
* Congestion charge as walled city, in electronic space
* London: highest density of cctv in the world
* will we decide to travel to areas based on the quality of electronic space
* A new architectural language for electronic space
* Houses without windows, just cameras. Can start to control life inside. Can also choose to use the weather channel as windows
* Pay a fee for personal surveillance: ask them to watch you all the way to the supermarket.
* The city of Yokohama was brought down by the coming of age party for 40,000 teenagers: the networks were overloaded with messages, because the teenagers didn’t want to talk face to face.
* Palm trees as cell towers (seen in south africa)
* Looked at a community in Hackney that were campaigning to not have a cell phone tower.
* Designed a house for them that would shield them from the signals, but they would have to give up cell phone connectivity. Designed it so that windows would open and close based on calls being made, or would give them 10 minute windows in which to make calls every 2 hours.
* Digital shelter: stand inside the line
h3. Round up
* These presentations all use the strategy of showing ‘hypothetical products’ that are really non-products. They are doing this, rather than providing platforms or design methodologies, or distributing resources and infrastructures for people to design their own systems. I understand the need for designers as visionaries, but this could be made more valuable and useful.
* specialists in electronic space could be similar to lighting design specialists in the ’70s. Will grow into a general field of understanding.
* Platforms and inftrastructure for technology is beyond architects, but understanding of the use and consequences is really important.
h3. Session 4
h3. Jocko Weyland
* Skateboarding as adaptive design: difference between skate parks and the street, skate parks become designed over time to mimic certain aspects of streets, but also according to innate, human skaters needs. A combination of factors go into making a good skateboarding space: free, alcohol, quality, location.
* New to NY: wanted to work outside gallery space, was inspired by collage of city streets. Not from a graffiti background, being a female, can do certain things outside the norms of graffiti.
* Changes billboards during the day, looks official.
* Open democratic visual space
* a visual direct democracy…
* Cuba used to have street art as a means of free expression, but outlawed by dictatorship
* Makes lightboxes with imagined cities, and mounts on the reverse side of construction site walls, with peepholes ‘peer here’
* Interesting mix of opportunism and ‘designed intervention’
* Sometimes driven purely by visual interest.
h3. “Michael Rakowitz”:http://www.possibleutopia.com/mike/
* Mike Davis: Public is phantom
* Bedouin as a model of sustainable nomadic communities
* Homeless use waste air from air conditioning (airvac exhaust ports) to stay warm and dry
* Homeless have receded to the peripheral vision of the public. Want to see and be seen.
* Seeing is important for living nomadically in the city.
* Started to map the heat and the power of the exhaust fans in the city. Found a high one at MIT plasma lab.
* Re-routed smell from from a bakery to an art gallery, to subvert a ‘high art’ re-appropriation of space
h2. Workshop ‘Loop City’
* “Dietmar Offenhuber”:http://residence.aec.at/wegzeit/ & Sara Hodges
* Showed Rybczynski’s film “New Book”:http://www.microcinema.com/titleResults.php?content_id=1190 using 9 frames: a good way of mapping space in the city. Starts off and the viewer is not sure if each frame is occurring synchronously, or in the same space, but a bus passes between all of the frames and the spatial link is made immediately. There is also a point where a plane flies overhead and all the actors look up: showing time synchronicity too.
h3. Looking at the city
* as a set of repeated actions
* as a playground: situationists
* as a balance of social as well as physical architectures