# Embodied interaction in music > A set of sketches from Easter 2005 on navigating music on portable devices: predictive text input, squeeze-and-shake movement, audio scrubbing and gestures like covering an ear. - Page: https://www.elasticspace.com/2005/04/embodied-interaction-music/ - Author: Timo Arnall (https://www.elasticspace.com/) - Section: Projects (https://www.elasticspace.com/project/) - Published: 2005-04-28 - Updated: 2026-07-04 - Tags: Gestural Interaction, Interaction Design, IxD Theory, Media, Mobile, Mobility, Research, Sound, Sound Design, Usability - Site map for agents: https://www.elasticspace.com/llms.txt --- One gesture I keep coming back to: covering your ear to switch tracks. The hand next to the ear, as if to listen more intently. A light or capacitive touch sensor in each earbud could pick it up; the interface would feel intimate rather than performative. *A set of sketches from Easter 2005 on navigating music on portable devices. Written after switching from a 40GB iPod to the [iPod Shuffle](https://web.archive.org/web/20050630234548/http://www.apple.com:80/ipodshuffle/) and finding that [the white-knuckle ride of random listening](https://web.archive.org/web/20050722082015/http://www.cityofsound.com:80/blog/2005/01/the_rise_and_ri.html) is genuinely enjoyable, but that the need for a better small-screen music experience hasn’t gone away. The clickwheel doesn’t cut it. These are sketches, not prototypes.* The pseudo-analogue clickwheel is difficult to control when scrolling huge alphabetical lists, and the view’s acceleration can be frustrating. Clicking into deeper lists, scrolling, clicking deeper, long tortuous experiences if you’re doing anything else at the time. It’s also difficult to use through clothing, or with gloves. ### Music and language ![Sketch of a mobile phone keypad used to type the first few letters of an artist or album, with iTunes-style realtime results](https://www.elasticspace.com/images/embodied-interaction-music/embodied_music_search.jpg) *Sketch of a mobile phone keypad used to type the first few letters of an artist or album, with iTunes-style realtime results* My first thought was something [Jack Schulze](https://web.archive.org/web/20050520085811/http://www.jackschulze.co.uk:80/) and I discussed a long time ago, using a phone keypad to type the first few letters of an artist, album or genre, and seeing results in real-time, the way [iTunes](https://web.archive.org/web/20050701014924/http://www.apple.com:80/itunes/jukebox.html) does on a desktop. I find myself using this in iTunes more than browsing lists. [Predictive text input](https://web.archive.org/web/20050702030924/http://www.t9.com:80/) would be very effective here, limited to the dictionary of your own music library. [QIX search](https://web.archive.org/web/20050721004927/http://www.christianlindholm.com:80/christianlindholm/2005/02/qix_from_zi_cor.html) could do this for a music library on a mobile. Mobile-phone music convergence makes this the right time, with [mobile](https://web.archive.org/web/20051228023248/http://www.sonyericsson.com:80/spg.jsp?cc=gb&lc=en&ver=4000&template=pp1_loader&php=php1_10245&zone=pp&lm=pp1&pid=10245) [phone](http://www.nokia.com/n91/) [music convergence](https://web.archive.org/web/20160702043310/http://www.engadget.com/entry/1234000540040867/). ### Navigating through movement ![Sketch of squeezing the device to put it into a receptive state for motion input](https://www.elasticspace.com/images/embodied-interaction-music/embodied_music_squeeze.jpg) *Sketch of squeezing the device to put it into a receptive state for motion input* Since scrolling is inevitable to some degree, even within fine search results, what about using simple movement or tilt to control the search? The problem with movement as input is context: when is movement intended, and when is it the result of walking or a bump in the road? ![Sketch of a squeeze-and-shake quasi-mode for gestural control](https://www.elasticspace.com/images/embodied-interaction-music/embodied_music_squeeze2.jpg) *Sketch of a squeeze-and-shake quasi-mode for gestural control* One solution: a squeeze-and-shake quasi-mode. Squeezing the device puts it into a receptive state. ![Sketch of three-axis tilt input, less sensitive to larger movements of walking or transport](https://www.elasticspace.com/images/embodied-interaction-music/embodied_music_tilt.jpg) *Sketch of three-axis tilt input, less sensitive to larger movements of walking or transport* Another: more reliance on the three axes of tilt, which are less sensitive to larger movements of walking or transport. ### Gestures ![Sketch of various gestural inputs for music navigation](https://www.elasticspace.com/images/embodied-interaction-music/embodied_music_gestures.jpg) *Sketch of various gestural inputs for music navigation* Gestural interfaces are a hard sell. Most prototypes I have seen are difficult to learn and require a level of performativity not everyone wants to do in public space. But accelerometers inside these devices should allow for hacking together personal, adaptive gestural interfaces for higher-level functions. ![Sketch of a capacitive touch sensor in each earbud, detecting when the ear is covered](https://www.elasticspace.com/images/embodied-interaction-music/embodied_music_earbuds.jpg) *Sketch of a capacitive touch sensor in each earbud, detecting when the ear is covered* One gesture that could be simple and effective is the one I opened with: covering the ear to switch tracks. A capacitive touch sensor in each earbud. The hand next to the ear, as if to listen more intently. Interference from other objects, resting the head against a wall, for instance, would be a concern, but there’s something nicely personal about the gesture. ### More knobs ![Sketch of analogue controls for truly analogue functions like volume and time](https://www.elasticspace.com/images/embodied-interaction-music/embodied_music_knobs.jpg) *Sketch of analogue controls for truly analogue functions like volume and time* Things that are truly analogue, like volume and time, should be mapped to analogue controls. One of the great unexplored areas in digital music is real-time audio scrubbing, currently not well supported on any device, probably because of technical constraints. Scrubbing through an entire album with a directly mapped input would be a good way of finding the track you wanted. Research projects like the [DJammer](https://web.archive.org/web/20050829021639/http://www.hpl.hp.com:80/research/mmsl/projects/djammer/) are starting to look at this, specifically for DJs. But since music is inherently time-based there is more work to be done here for everyday players and devices. Let’s skip the interaction design habits we’ve learnt from the CD era and go back to vinyl. ### Evolution of the display ![Sketch of an OLED display integrated under the surface of a device, without a separate glass area](https://www.elasticspace.com/images/embodied-interaction-music/embodied_music_display.jpg) *Sketch of an OLED display integrated under the surface of a device, without a separate glass area* Where displays are required, I hope we can be free of small, fuzzy, low-contrast LCDs. With new displays being printable on paper, textiles and other surfaces, there’s the possibility of improving the usability, readability and ‘glanceability’ of the display. The [Sony Network Walkman](https://web.archive.org/web/20050830115552/http://dapreview.net:80/comment.php?comment.news.1086) shows signs of this, the OLED display is under the surface of the product material, without a separate ‘glass’ area. ![Sketch of an e-ink display integrated into the white surface of a music player](https://www.elasticspace.com/images/embodied-interaction-music/embodied_music_display2.jpg) *Sketch of an e-ink display integrated into the white surface of a music player* For a white iPod surface, the high-contrast, [paper-like surfaces](http://www.polymervision.com/New-Center/Downloads/Index.html) of e-ink would make great, highly readable displays. ### Prototyping ![Sketch of a hardware prototype with accelerometer and small display for testing embodied music navigation](https://www.elasticspace.com/images/embodied-interaction-music/embodied_music_prototype.jpg) *Sketch of a hardware prototype with accelerometer and small display for testing embodied music navigation* So I need to get prototyping with accelerometers and display technologies, to understand simple movement and gesture in navigating music libraries. There are open questions: whether movement-based scrolling through search results would create the appearance of a large screen space, through the lens of a small screen. As with [bumptunes](http://interconnected.org/home/2005/03/04/apples_powerbook), many more opportunities will emerge in the making. ### More reading - [Designing for Shuffling](https://web.archive.org/web/20050428203737/http://www.cityofsound.com:80/blog/2005/04/designing_for_s.html) - [Thoughts on the iPod Shuffle](http://interconnected.org/home/2005/04/22/there_are_two) - [Bumptunes](http://interconnected.org/home/2005/03/04/apples_powerbook) - [Audioclouds / gestural interaction](http://www.dcs.gla.ac.uk/~jhw/audioclouds/) - [Sound objects](https://www.elasticspace.com/2005/02/sound-objects/) - [DJammer](https://web.archive.org/web/20050829021639/http://www.hpl.hp.com:80/research/mmsl/projects/djammer/) - [On the body](https://web.archive.org/web/20050828001233/http://people.interaction-ivrea.it:80/b.negrillo/onthebody/) - [Runster](https://web.archive.org/web/20050312022600/http://communications.siemens.com:80/cds/frontdoor/0,2241,hq_en_0_91525_rArNrNrNrN,00.html)