Since I last wrote, everything I see or read seems to inspire my half-baked ideas about the better ways we can browse our unmanageably large music libraries. After telling a friend about these ideas, he said:

Yeah, it’s actually really frustrating. I intentionally keep the number of artists on my iPod small so I don’t have to sort to find things I’m currently into.

Me too.

Then there are the people who are doing a lot of (real) work towards novel interfaces like the (hypothetical) ones I’m describing; Last.fm’s “Islands of Music” (explained here) demonstrates the kind of artist-similarity topology that would make browsing your library a more pleasant experience; Lee Byron explains in more detail how he developed that Last Graph infovis; necimal releases a Music Recommendations extension for Songbird that promises to use Last.fm’s data to find within your library artists similar to the one playing; and the Aurora project, part of the Mozilla Labs concept browser series, depicts a radical three-dimensional view of files and data with auto-clustering, which, if applied to a music library, would be nothing short of incredible.

I’ve also thrown together a pitiful little mock-up of what Songbird might look like when you start it up with the kind(s) of extensions I’m hoping for:

The two core components depicted are the Start Page and the Timeline View. The Start Page I feel would be seriously valuable, one of the ideas behind all these blatherings of course being that one doesn’t always have a destination in mind when opening their music library. The Start Page would offer a number of convenient “jumping-off” points, pulling you into your library to explore it further — by artist similarity, maybe, or by play history proximity, after just a couple clicks.

The Timeline View is a zoomable timeline, shown here zoomed to a daily view. Zooming out could show you albums played within recent weeks; then months, quarters, etc. These albums might be sorted by Periodical Impact, something I explained in depth here; essentially they would be sorted not by the raw number of times they were played within any given period, but by how distinct they were to that period.

Even these meager ideas are leagues ahead of what’s available, and I’m not even a data analyst. Just imagine how a library’s play history data could be exploited by somebody trained in these things.

Blueboy’s Unisex, on the other hand, will probably always remind me of the winter of 2006-7, as I listened to it for the first time that season, nine additional times within that season (racking up about 150 tracks listened, according to Last.fm), and virtually never again once spring hit.

Ever since I began submitting listening data to Last.fm in November of 2004, I’ve wondered whether I’d ever enjoy direct access to all those numbers. Then came Last.fm Extra Stats, mercifully collecting all my listening data for me in a tab-separated file that can be pulled into Excel and manipulated to my heart’s content. Here, as a small example of the data, are my top ten artists (by tracks listened) from winter 2006-7, along with total listens for each artist (since November 2004) (now that I’m finally getting around to publishing this post, all the following data is very old):

Now for some methodology. We only have two numbers available to us here; let’s call them S (seasonal total) and T (total total). As I said, the psychological impact of a band/record over a given time period depends on both the amount of listening during that period (S) and the percentage of total listening within that period (S/T). Neither figure is good enough on its own; I may have listened to Destroyer 102 times this summer (I did), but that’s insignificant given my 1,500 overall listens. And I may have only listened to The Ladybug Transistor this summer (yielding a 100% concentration), but 3 listens is hardly enough to make an impact.

But, since both figures are positive indicators of impact, we can just multiply them (S²/T). This would give Ladybug Transistor a summer 2007 impact index of 3, and Destroyer, 6.92.

Let’s see how this multiplication affects winter’s top ten artists by tracks played:

Most notably, Ratatat is knocked down a few notches (on account of how consistently I listen to them), The Moldy Peaches go up a few spots — and if we reorder the entire list (of 78 artists) by impact index, a couple more eke into the top ten:

This last table is what we’re really after, theoretically giving us the ten artists who will most strongly remind me of the winter of 2006-7. It seems a bit boring at this point, since it’s pretty close in its results to the standard top-ten-by-listens chart; but the methodology is sound, and under more extreme listening conditions (e.g., a Destroyer obsession), it will prove to be a reliable indicator of impact.

And don’t forget, as time passes and I listen to these artists more, their season-specific impacts will drop. For instance, as things turned out, I listened to Television Personalities heavily over the spring, increasing their T value, and reducing their impact in the above table (which uses old data). As a result of my TVP spring listening, their winter index dropped dramatically from 35 to 7.95 — their spring index, as of October 5, 2007, is 91.95.

Depicting this analysis visually could go several ways; somehow I want to see a pie chart. Making one from every artist would be impractical, leaving a circle with a hundred or more slivers. Taking the top ten artists feels insensitive to the quirks of any given season; what if I don’t even listen to ten artists that season, for instance? What if I listen to a thousand? I could select only those artists who have indices greater than, say, 20; but this also feels arbitrary and vulnerable to seasonal quirks.

I think a pretty safe threshold above which to choose artists is at 50% of the sum of all impact values. In other words: the sum of the impact indices of winter’s artists is 783. If I take artists off the top of the list until I have half that, I stop after the fourth artist, The Moldy Peaches. Admittedly, a four-sector pie chart is kinda dull, but I need to be consistent in the creation of these charts so that impacts are accurately represented from season to season.

So, roughly…here are what the last three seasons of 2007 “meant” to me:

What I like about these is that they are not only internally consistent, but consistent with each other; Ariel Pink will remind me of the spring of 2007 more than M.I.A. will remind me of the summer, as represented by a larger sector in his season’s pie. I could similarly generate these with a focus on albums, which would be slightly more specific and alter the results dramatically. I know, for instance, that all my summer 2007 listening to M.I.A. was off her new record, songs that necessarily didn’t contribute to any prior M.I.A. listening — so, Kala, were it to have a sector in an album pie, would have one much larger than the less specific “M.I.A.” one.

Granted, there are more than a few problems with this whole endeavor, and here are the major three:

1. Last.fm does not track things that I listen to on CD/vinyl. This is a serious problem, as the things I purchase tend to be the things I like the most, and go totally unaccounted for.
2. The number of tracks you’ve listened to by an artist is only kind of related to how much time you’ve spent listening to them. 4 tracks can fill 2 CDs (Lift Your Skinny Fists), or you can cram 91 tracks into 40 minutes of music (Irrevocably Overdriven).
3. iPod listens are not properly documented. Even if I remember to report them to Last.fm with AudioPod, multiple listens on a single track only get counted as one — this is particularly damaging on long trips, for instance.

The (S²/T) formula may have further applications, as I’ll touch on in a follow-up to my original post about alphabetization and digital music library navigation.

Advantages of sorted lists include:

• one can easily find the first n elements (e.g. the 5 smallest countries) and the last n elements (e.g. the 3 largest countries)
• one can easily find the elements in a given range (e.g. countries with an area between .. and .. square km)
• one can easily search for an element, and conclude whether it is in the list

The first two advantages are things you almost never need to do with music libraries. And the third has been supplanted by now-ubiquitous search boxes: if you know what you’re looking for, you search; and if you don’t, an alphabetized list is not the way to find it.

Web visionary Ted Nelson (<mst3k>Dr. Ted Nelson?</mst3k>) has been paraphrased as pointing out that “electronic documents have been designed to mimic their paper antecedents,” and that “this is where everything went wrong: electronic documents could and should behave entirely differently from paper ones.” If the folder metaphor is inadequate for digital documents, no wonder it’s so pitiful at handling music. The proximity between pieces of music in a library should least of all be based on the first letter in a band’s name—it’s as arbitrary as sorting them by the vocalist’s month of birth—yet this is how it’s universally done.

Music library organization needs to be re-thought from the ground up. We need to consider how it is that people used to listen to music before it was all on their iTunes. How are your CDs organized (or disorganized) on your shelf? How are they organized in your head? What is it that prompts you to listen to what you listen to when you listen to it? And how can we use computers to adopt and enhance these ways of thinking, rather than forcing us to think like computers?

The most natural method for organizing music (if you can escape alphabetical thinking for a moment) is by similarity. Last.fm does this, and it is invaluable. When you are at an artist’s page at Last.fm, you feel that you are in that artist’s “neighborhood,” with links to similar bands, tags, listeners, and related groups. The Last.fm architecture was designed to manifest organic, bottom-up communities around bands and genres. This is an experience that cannot currently be replicated in any music player, at least not easily (with the possible exception of Amarok). But because Last.fm’s data is extraordinarily accessible, there are virtually no obstacles to incorporating this sense of “musical neighborhoods” into a piece of software.

If you don’t already have an artist in mind whose neighborhood you’d like to browse, you probably at least have some idea of the kind of mood you’re after, and there are several approaches here. One (perhaps the least viable) is using Last.fm’s tags. These are actually less often concerned with mood than they are with genre, a taxonomy well-known as being inconsistent and, I would argue, misguided: When I want to listen to Lullatone, it’s not because I want to listen to electronic music; it’s because I want to listen to “whimsical,” “delicate,” “innocent,” “sparkling” music. Who cares what genre it is?

AllMediaGuide began a project called Tapestry some time ago, an application of their vast mood/situation/genre dataset. It is an ideal solution for browsing music, and its integration into desktop software would be hugely rewarding. It’s possible to simulate Tapestry with foobar2000 and some elbow grease, but the results are not as robust as they could be.

Again, if you already know what you’re looking for, it would be difficult to find it through these channels; but this is what search is for.

We also need to consider the less objective and more personal reasons that music becomes relevant in specific contexts, analogously to the way in which CDs become disordered on one’s shelf. I, for instance, usually have about 20 albums littering the top of my receiver and speakers. These include, roughly, (a) stuff I just bought, (b) stuff I just listened to, and (c) stuff I haven’t bothered putting away because I know I’ll listen to it again soon.

The main obstacle to browsing in this way is a prevalent shortcoming whose symptoms are far-reaching: the fact that music players “think” in terms of songs, not in terms of albums—or even in terms of artists, for that matter: My music software doesn’t know that these 38 songs are all by Electrelane; it just knows that their artist metadata is alphabetically adjacent. Sure, you can sort iTunes libraries by data such as “last listened” and “added”; and you can use CoverFlow to simulate a pale approximation of a flesh-and-blood record collection; but the only way you can sort albums or artists is alphabetically. I’ve written about the ways in which MP3Toys addresses this problem, and it remains a commendable pioneer in music management, but its difficulties (a steep learning curve, a buggy interface, a rapid release schedule) outweigh its advantages.

Browsing your own music library is a very impersonal experience, despite enormous potential for personalization. Rich info visualization “toys” such as Last.fm Extra Stats and LastGraph are seen as novelties, but would, in fact, be revolutionary as library browsing environments. There is nothing to prevent this from development, either; even users who are not plugged into Last.fm could have their listening history stored locally by their music software, which could then be used to generate small, cached, infinite-resolution SVG histograms, browsable by zooming, panning, and clicking. Far from being cumbersome and CPU-intensive, it would actually be rather elegant.

Continuing on the theme of chronology, what about a simple calendar charts view, with varying granularity by day, week, month, quarter? Presentationally, these charts could even be made easily to resemble vertical stacks of CDs, with spine art generated from a cropped cover image and overlaid text. This would arguably be eye-candy, of course, but just imagine how it would “feel” to see your music this way. If there’s one thing Apple’s been consistently right about, it’s that functionality is not at odds with a pleasant user experience, but rather that they are meant to be mutually supportive.

When I was helping to establish some playstamp tagging standards with the foobar community in 2005, it was jokingly suggested that when songs are played they be tagged with the current weather. Despite the sarcasm, I couldn’t help but think, “What a great idea!” I know my listening habits are affected by the weather, and I can’t imagine other people are not the same. There is, after all, a frequently-used “rainy day” tag at Last.fm.

What about a histogram based not on play count, but on hotness values over time? What about artist similarity webs based not on Last.fm data, but on proximity of play times within your personal history? What about taking lessons from the DONTCLICK.IT project, BumpTop, and the pile metaphor for unprecedentedly fluid user interfaces?

All the ingredients are there. Everyone is rapidly moving towards an exclusively digital music collection, and the technology is embarrassingly outmoded. Music has become a major component of computing, at levels once reserved for word processing and gaming. Our relationship with our digital music collections is poised for reinvention, a looming difficulty that has been made invisible by custom and habit. Digital music management is hell, and users have complacently accepted this.

The obvious solution at this point is Songbird. Songbird’s media views (present since 0.5) allow more easily than ever for custom browsing environments. Previously the only way to alter your music browsing environment was to switch programs entirely; besides which, nearly all available programs simply mimic the well-known disk/directory views or iTunes’ browser pane view (which is just a glorification of a disk/directory view anyway). Songbird, on the other hand, boasts an unprecedented extensibility, coupled with media attention, ease-of-use, and the Mozilla platform, for which people have been developing extensions for ages (in computer years).

The prospects are thrilling and the potential for innovation is virtually limitless. Promisingly, there are some glimpses of where things might be headed for Songbird media views: Catalogue View demonstrates a novel visual presentation of your library, though it doesn’t do much in the way of organizational presentation; and Metrics Media Page is the beginnings of the kind of infovis view that could be (but currently isn’t) adapted to allow for actual navigation. Nevertheless, I have a bad feeling that this opportunity will be missed, as the status quo continues to obfuscate these possibilities.

Update: Part II

Very early in 2005, in deep snow, I was temping in Cleveland, long outstaying my expected tenure at Heinen’s corporate office, distributing and replacing preferred customer cards. I was becoming obsessed with foobar and listening mostly to The Mollusk and Guero. I had recently converted my html-based dated-entries Beigetower site to the Blogger platform. I became really sick, sicker than I’ve been since, and had to call into work to recover in bed. Temps, of course, aren’t allowed the luxury of health, so they let me go within the hour, and, admittedly, the last week or two of my employment there was more of a favor than anything else; they liked me.

Later that same week I picked up a brief assignment somewhere else, a place whose name escapes me now, but whose offices I remember vividly — small, dreary, windowless, me stuck in a very bare cube with a very uncomfortable chair. At this time I was still extremely sick, plowing through packs of tissues and Throat Coat® tea. I had just downloaded The Clientele’s Suburban Light from Blair, and with the iPod that my entire family had contributed toward that Christmas, I listened to the album repeatedly my first day there. Just over and over, it was mindless data entry work, so nobody cared. I hardly spoke a word during those three days.

The music, the illness, the desperate joblessness, the tea. I remember that day better than most other days of the last two years, and I find myself wanting to know as much as possible about my life during that two-week period. Fortunately it was about that time that Last.fm began archiving weekly charts for future reference, and you can see what I was listening to here, though it actually looked something more like THIS. Between that, Gmail’s almost limitless storage capacity, several forums I visit regularly, and, to an extent, kbps, I can piece together a pretty good picture of what was going on.

Of course, the perfect tool for logging things and stages in my life is WordPress, which is right here. I’ve never really used it much for this purpose, because I used to adamantly hate personal blogs, but since I don’t record the mundane details of my life anywhere else, I have no choice but to do it here, which accounts for 

http://www.library.utoronto.ca/utel/criticism/hutchinp.html

http://www.kilobitspersecond.com/topalbumart.php?user=

Just tack your last.fm username onto the end of that url to generate the cover art for your most listened-to album of last week. Pretty cool.

Bands and albums with ampersands don’t work at the moment. urlencode() doesn’t seem to do the trick. Any theories?

http://ws.audioscrobbler.com/1.0/user/topdownjimmy/weeklyalbumchart.xml

Unfortunately, this won’t reflect your listening accurately if you’re in the habit of listening to leaked albums. For what are certainly legal issues, last.fm plays dumb that these albums even exist, failing to report them in charts even though the track and artist counts are updated accordingly.

My next step is to use the url embedded in the feed to scrape the Amazonian cover art from each album’s last.fm page. This would be cool to do even for the recent track feed, come to think of it.

last.fm is great, and it gets better every single day. Part of its appeal is voyeurism. I love being able to see what my friends are listening to, but that usually requires going to the “What are my friends listening to?” page, which is still too much effort; I’m not that curious. But still, if somebody I know starts listening to something, I’d like to be alerted with a totally passive system.

There are, of course, RSS feeds for all kinds of things from last.fm. But there is no feed consisting of all your friends’ recent tracks, which is surprising because it’s such an intuitive idea. So implementing the ones that are available is ostensibly possible, but nevertheless tricky. I mean, logging into Bloglines or Google’s new reader still requires an active request for this information. And while there are some web services that will merge multiple feeds into a single one for you, I don’t like relying on a third party like that, one that may go down any day and that might insert advertisements into my feed.

It seems to me that there should be a very, very small program that sits in your system tray, checking multiple feeds regularly, then popping up a native Windows balloon with a link to the “article” every time there’s an update. This would be ideal for watching your last.fm friends. There are programs that do this, but they’re all full applications that only have this as an auxiliary feature. I can’t afford the memory.

So, finally, I found infoRSS. It’s a Firefox extension that adds a little ticker to the statusbar. Initially I wasn’t hopeful, as its default presentation is ugly and therefore indicative of poor programming:

The writer of this extension isn’t a native speaker of English, and there’s very little help available anyway. I spent a long time studying its many confusing features, confident that it could be made to do what I want. The result (shown at the top of this post) isn’t perfect, but is better than I had expected or hoped. There’s a nice little Audioscrobbler logo on the left; each entry is marked with the user’s avatar, which is far more efficient than if their name were displayed; and the listening status of everyone is constantly on display for me. Here’s how to do this:

First, in the options for the extension, click on the “Advanced” tab. Under “Default values,” check “# of news” and constrain it to 1 so that you only get the most recent item (I’m not sure that this is effectual, but in case it is, it would be helpful). Check “# of char/news” and constrain it to 20 or 30 (depending on how many friends you add later, things could get pretty cluttered). Check “Refresh Time” and change it to 2 minutes (last.fm will ban your IP if you make requests on a feed more than once per minute).

Now go to the “Main menu” section, still in Advanced. Make sure “Synchronize main icon with current group” is on, and “Flashing icon” is off (trust me, it’s really annoying).

Go to the “Basic” tab. Here’s where you’ll add feeds. First get rid of all the presets. Select them from the drop-down menu at the top of the window and click the “Remove” button at the lower-right. Now click the “New group” button and name it last.fm. In the “Icon” box, enter http://static.last.fm/matt/favicon.ico and click “Test.” Now, to add individual users’ feeds, click “New feed” and enter this URL: http://ws.audioscrobbler.com/1.0/user/USERNAME/recenttracks.rss. Replace “USERNAME” with the person’s username of course. In the “Icon” box, paste the URL for their last.fm avatar. Repeat for each user you want to add.

There’s more.

In the “General” section, turn off all the options under “General” (leave “InfoRSS activity” on), and set “Tooltip on headlines” to “Full headline.”

In the “Headlines area” section, “Location” dictates where the ticker will go (I prefer bottom). Turn off “Scrolling headlines”; the “Fade in/out” feature seems nice, but it’s buggy. I’m not sure what the “Cycling feed/group” options are for. Also, uncheck all the boxes in the bottom section.

Okay, in the “Headlines style” section, make those first three options On, Off, Off. The next three should be “Automatic,” “Auto,” “Default.” Also change the settings in the bottom box to “Auto” and “Default,” and uncheck “Italic” and “Bold.”

Go back to the “Feed/Group” section, and select the last.fm group from the drop-down. Click the check box next to the words “Check all” to add all the feeds to this group, and click the “Make current” button to set the last.fm group to be displayed.

Now, the most important step: click on the “Filter” tab. Under “Filter policy,” choose “Use active filters from both.” Set “Match” to “All.” Check the box next to the first criterion, and change it to “Headline # Equals 1.” This ensures that it only shows the most recent track for each person. Click the plus sign to add another criterion, and make the new one “Published date Less than 30 minutes.” This ensures that tracks are only shown if they’ve been played in the last half hour (or however many minutes you want). This must be the last thing you do before you finalize changes by clicking “OK.” If you change any other options after this, the criteria will be mangled by a bug in the program, and you will be forced to remove all the criteria, click “OK,” and go back into options again to reset the criteria.

That should do it. The Audioscrobbler icon at the far left will turn into a user’s avatar when it checks their feed. If for any reason you can’t get this to work for you, let me know.