Tangible and social interaction

Posted on Mar 18, 2005 in Interaction design, Research, Social, Technology, Ubicomp

Brief history of interaction

(Based on Dourish, see reading recommendations, below)

Each successive development in computer history has made greater use of human skills:

  • electrical: required a thorough understanding of electrical design
  • symbolic: required a thorough understanding of the manipulation of abstract languages
  • textual: text dialogue with the computer: set the standards of interaction we still we live with today
  • graphic: graphical dialogue with the computer, using our spatial skills, pattern recognition, and motion memory with a mouse and keyboard

    We have become stuck in this last model.

    Interaction with computers has remained largely the same: desk, screen, input devices, etc. Even entirely new fields like mobile and iTV have followed these interaction patterns.

    Definitions:

  • Tangible: physical: having substance or material existence; perceptible to the senses
  • Social: human and collaborative abilities, or ‘software that’s better because there’s people there’ (Definition from Matt Jones and Matt Webb)

    Examples

    Dourish notes in the first few chapters of his book that as interaction with computers moves out into the world, it becomes part of our social world too. The social and the tangible are intricately linked as part of “being in the world”.

    What follows are examples of products or services we can use or buy right now. I’m specifically interested in the ways that these theories of ubiquitous computing and tangible interaction are moving out into the world, and the way that we can see the trends in currently available products.

    I’m aware that there are also terrifically interesting things happening in research (eg the Tangible Media Group) but right now I’m interested in the emergent things that start to happen effects of millions of people using things (like Flickr, weblogs, Nintendo DS, and mobile social software).

    Social trends on the web

    On the web the current trend is building simple platforms that support complex social/human behaviour

  • Weblogs, newsreaders and RSS: simple platform that has changed the way the web works, and supported simple social interaction (the basic building blocks of dialogue, or conversation)
  • Flickr: a simple platform for media/photo sharing: turned into a thriving community: works well with the web by allowing syndicated photos, bases the social network on top of a defined funciton
  • Others include del.icio.us, world of warcraft, etc.

    Social mobile computing

    On mobile platforms most of the exciting stuff is happening around presence, context and location

  • Familiar strangers: stores a list of all the phones that you have been near in places that you inhabit, and then visualises the space around you according to who you have met before. More mobile social software
  • Mogi: location based game, but most interestingly supports different contexts of use: both at home in front of a big screen, and out on a small mobile screen.

    Social games

    Interesting that games are moving away from pure immersive 3D worlds, and starting to devote equal attention to their situated, social context

  • Nintendo DS: PictoChat, local wireless networks that can be adapted for gameplay or communication (picture chatting included as standard)
  • Sissyfight: very simple social game structure, encourages human behaviour, insults
  • Habbohotel: simple interaction structures, (and fantastic attention to detail in iconic representations) support human desires. Now a very large company, in over 12 countries, based on the sales of virtual furniture
  • Singstar: entirely social game, about breaking social barriers and mutual humiliation: realtime analysis/visualisation of your voice actually makes you sing worse!

    Tangible games

  • Eyetoy: Brings the viewer into the screen, creates a performative and social space, and allows communication via PS2
  • Dance Dance Revolution: taking the television into physical space
  • Nokia wave-messaging: puts information back into space, and creates social and performative opportunities (Photo thanks to Matt Webb)

Sound objects

Posted on Feb 25, 2005 in Interaction design, Research, Sound, Technology, Ubicomp

These are some of my notes from Mikael Fernström’s lecture at AHO.

The aim of the Soundobject research is to liberate interaction design from visual dominance, to free up our eyes, and to do what small displays don’t do well.

Reasons for focusing on sound:

  • Sound is currently under-utilised in interaction design
  • Vision is overloaded and our auditory senses are seldom engaged
  • In the world we are used to hearing a lot
  • Adding sound to existing, optimised visual interfaces does not add much to usability

    Sound is very good at attracting our attention, so we have alarms and notification systems that successfully use sound in communication and interaction. We talked about using ‘caller groups’ on mobile phones where people in an address book can be assigned different ringtones, and how effective it was in changing our relationship with our phones. In fact it’s possible to sleep through unimportant calls: our brains are processing and evaluating sound while we sleep.

    One fascinating thing that I hadn’t considered is that sound is our fastest sense: it has an extremely high temporal resolution (ten times faster than vision), so for instance our ears can hear pulses at a much higher rate than our eyes can watch a flashing light.

    Disadvantages of sound objects

    Sound is not good for continuous representation because we cannot shut out sound in the way we can divert our visual attention. It’s also not good for absolute display: pitch, loudness and timbre are relative to most people, even people that have absolute pitch can be affected by contextual sounds. And context is a big issue: loud or quiet environments affect the way that sound must be used in interfaces: libraries and airplanes for example.

    There are also big problems with spatial representation in sound, techniques that mimic the position of sound based on binaural differences are inaccessible by about a fifth of the population. This perception of space in sound is also intricately linked with the position and movement of the head. Some Google searches on spatial representation of sound. See also Psychophysical Scaling of Sonification Mappings [pdf]

    Cartoonification

    ‘Filling a bottle with water’ is a sound that could work as part of an interface, representing actions such as downloading, uploading or in replacement of progress bars. The sound can be abstracted into a ‘cartoonification’ that works more effectively: the abstraction separates simulated sounds from everyday sounds.

    Mikael cites inspiration from foley artists working on film sound design, that are experienced in emphasising and simplifying sound actions, and in creating dynamic sound environments, especially in animation.

    A side effect of this ‘cartoonification’ is that sounds can be generated in simpler ways: reducing processing and memory overhead in mobile devices. In fact all of the soundobject experiments rely on parametric sound synthesis using PureData: generated on the fly rather than using sampled sound files, resulting in small, fast, adaptive interface environments (sound files and the PD files used to generate the sounds can be found at the Soundobject site).

    One exciting and pragmatic idea that Mikael mentioned was simulating ‘peas in a tin’ to hear how much battery is left in a mobile device. Something that seems quite possible, reduced to mere software, with the accelerometer in the Nokia 3220. Imagine one ‘pea’ rattling about, instead of one ‘bar’ on a visual display…

    Research conclusions

    The most advanced prototype of a working sound interface was a box that responded to touch, and had invisible soft-buttons on it’s surface that could only be heard through sound. The synthesised sounds responded to the movement of the fingertips across a large touchpad like device (I think it was a tactex device). These soft-buttons used a simplified sound model that synthesised impact, friction and deformation. See Human-Computer Interaction Design based on Interactive Sonification [pdf]

    The testing involved asking users to feel and hear their way around a number of different patterns of soft-buttons, and to draw the objects they found. See these slides for some of the results.

    The conclusions were that users were almost as good at using sound interfaces as with normal soft-button interfaces and that auditory displays are certainly a viable option for ubiquitous, especially wearable, computing.

    More reading

    Soundobject
    Gesture Controlled Audio Systems
    ICAD

Spatial memory at Design Engaged 2004

Notes on two related projects:

1. Time that land forgot

  • A project in collaboration with Even Westvang
  • Made in 10 days at the Icelandic locative media workshop, summer 2004
  • Had the intention of making photo archives and gps trails more useful/expressive
  • Looked at patterns in my photography: 5 months, 8000 photos, visualised them by date / time of day. Fantastic resource for me: late night parties, early morning flights, holidays and the effect of midnight sun is visible.
  • time visualisation

    2. Marking in urban public space

    I’ve also been mapping stickering, stencilling and flyposting: walking around with the camera+gps and photographing examples of marking (not painted graffiti).

    This research looks at the marking of public space by investigating the physical annotation of the city: stickering, stencilling, tagging and flyposting. It attempts to find patterns in this marking practice, looking at visibility, techniques, process, location, content and audience. It proposes ways in which this marking could be a layer between the physical city and digital spatial annotation.

    Some attributes of sticker design

  • Visibility: contrast, monochromatic, patterns, bold shapes, repetition
  • Patina: history, time, decay, degredation, relevance, filtering, social effects
  • Physicality: residue of physical objects: interesting because these could easily contain digital info
  • Adaptation and layout: layout is usually respectful, innovative use of dtp and photocopiers, adaptive use of sticker patina to make new messages on top of old

    Layers of information build on top of each other, as with graffiti, stickers show their age through fading and patina, flyposters become unstuck, torn and covered in fresh material. Viewed from a distance the patina is evident, new work tends to respect old, and even commercial flyposting respects existing graffiti work.

    Techniques vary from strapping zip-ties through cardboard and around lampposts for large posters, to simple hand-written notes stapled to trees, and short-run printed stickers. One of the most fascinating and interactive techniques is the poster offering strips of tear-off information. These are widely used, even in remote areas.

    Initial findings show that stickers don’t relate to local space, that they are less about specific locations than about finding popular locations, “cool neighbourhoods” or just ensuring repeat exposure. This is opposite to my expectations, and perhaps sheds some light on current success/failure of spatial annotation projects.

    I am particularly interested in the urban environment as an interface to information and an interaction layer for functionality, using our spatial and navigational senses to access local and situated information.

    There is concern that in a dense spatially annotated city we might have an overload of information, what about filtering and fore-grounding of relevant, important information? Given that current technologies have very short ranges (10-30mm), we might be able to use our existing spatial skills to navigate overlapping information. We could shift some of the burden of information retrieval from information architecture to physical space.

    I finished by showing this animation by Kriss Salmanis, a young Latvian artist. Amazing re-mediation of urban space through stencilling, animation and photography. (“Un ar reizi naks tas bridis” roughly translates as “And in time the moment will come”.

    Footnotes/references

    Graffiti Archaeology, Cassidy Curtis
    otherthings.com/grafarc

    Street Memes, collaborative project
    streetmemes.com

    Spatial annotation projects list
    elasticspace.com/2004/06/spatial-annotation

    Nokia RFID kit for 5140
    nokia.com/nokia/0,,55739,00.html

    Spotcodes, High Energy Magic
    highenergymagic.com/spotcode

    ?Mystery Meat navigation?, Vincent Flanders
    fixingyourwebsite.com/mysterymeat.html

    RDF as barcodes, Chris Heathcote
    undergroundlondon.com/antimega/archives/2004_02.html

    Implementation: spatial literature
    nickm.com/implementation

    Yellow Arrow
    yellowarrow.org

Load More