Embodied interaction in music
One gesture I keep coming back to: covering your ear to switch tracks. The hand next to the ear, as if to listen more intently. A light or capacitive touch sensor in each earbud could pick it up; the interface would feel intimate rather than performative.
A set of sketches from Easter 2005 on navigating music on portable devices. Written after switching from a 40GB iPod to the iPod Shuffle and finding that the white-knuckle ride of random listening is genuinely enjoyable, but that the need for a better small-screen music experience hasn’t gone away. The clickwheel doesn’t cut it. These are sketches, not prototypes.
The pseudo-analogue clickwheel is difficult to control when scrolling huge alphabetical lists, and the view’s acceleration can be frustrating. Clicking into deeper lists, scrolling, clicking deeper, long tortuous experiences if you’re doing anything else at the time. It’s also difficult to use through clothing, or with gloves.
Music and language
My first thought was something Jack Schulze and I discussed a long time ago, using a phone keypad to type the first few letters of an artist, album or genre, and seeing results in real-time, the way iTunes does on a desktop. I find myself using this in iTunes more than browsing lists.
Predictive text input would be very effective here, limited to the dictionary of your own music library. QIX search could do this for a music library on a mobile.
Mobile-phone music convergence makes this the right time, with mobile phone music convergence.
Navigating through movement
Since scrolling is inevitable to some degree, even within fine search results, what about using simple movement or tilt to control the search? The problem with movement as input is context: when is movement intended, and when is it the result of walking or a bump in the road?
One solution: a squeeze-and-shake quasi-mode. Squeezing the device puts it into a receptive state.
Another: more reliance on the three axes of tilt, which are less sensitive to larger movements of walking or transport.
Gestures
Gestural interfaces are a hard sell. Most prototypes I have seen are difficult to learn and require a level of performativity not everyone wants to do in public space. But accelerometers inside these devices should allow for hacking together personal, adaptive gestural interfaces for higher-level functions.
One gesture that could be simple and effective is the one I opened with: covering the ear to switch tracks. A capacitive touch sensor in each earbud. The hand next to the ear, as if to listen more intently.
Interference from other objects, resting the head against a wall, for instance, would be a concern, but there’s something nicely personal about the gesture.
More knobs
Things that are truly analogue, like volume and time, should be mapped to analogue controls. One of the great unexplored areas in digital music is real-time audio scrubbing, currently not well supported on any device, probably because of technical constraints. Scrubbing through an entire album with a directly mapped input would be a good way of finding the track you wanted.
Research projects like the DJammer are starting to look at this, specifically for DJs. But since music is inherently time-based there is more work to be done here for everyday players and devices. Let’s skip the interaction design habits we’ve learnt from the CD era and go back to vinyl.
Evolution of the display
Where displays are required, I hope we can be free of small, fuzzy, low-contrast LCDs. With new displays being printable on paper, textiles and other surfaces, there’s the possibility of improving the usability, readability and ‘glanceability’ of the display.
The Sony Network Walkman shows signs of this, the OLED display is under the surface of the product material, without a separate ‘glass’ area.
For a white iPod surface, the high-contrast, paper-like surfaces of e-ink would make great, highly readable displays.
Prototyping
So I need to get prototyping with accelerometers and display technologies, to understand simple movement and gesture in navigating music libraries. There are open questions: whether movement-based scrolling through search results would create the appearance of a large screen space, through the lens of a small screen. As with bumptunes, many more opportunities will emerge in the making.