1. The Decemberists
2. Destroyer
3. Pants Yell!
4. Brian Eno
5. Belle and Sebastian
6. The Clientele
7. Electrelane
8. Pastels/Tenniscoats
9. Pale Saints
10. Broadcast and The Focus Group

I blame Rhythmbox, Banshee, and Songbird.

Since I last wrote, everything I see or read seems to inspire my half-baked ideas about the better ways we can browse our unmanageably large music libraries. After telling a friend about these ideas, he said:

Yeah, it’s actually really frustrating. I intentionally keep the number of artists on my iPod small so I don’t have to sort to find things I’m currently into.

Me too.

Then there are the people who are doing a lot of (real) work towards novel interfaces like the (hypothetical) ones I’m describing; Last.fm’s “Islands of Music” (explained here) demonstrates the kind of artist-similarity topology that would make browsing your library a more pleasant experience; Lee Byron explains in more detail how he developed that Last Graph infovis; necimal releases a Music Recommendations extension for Songbird that promises to use Last.fm’s data to find within your library artists similar to the one playing; and the Aurora project, part of the Mozilla Labs concept browser series, depicts a radical three-dimensional view of files and data with auto-clustering, which, if applied to a music library, would be nothing short of incredible.

I’ve also thrown together a pitiful little mock-up of what Songbird might look like when you start it up with the kind(s) of extensions I’m hoping for:

The two core components depicted are the Start Page and the Timeline View. The Start Page I feel would be seriously valuable, one of the ideas behind all these blatherings of course being that one doesn’t always have a destination in mind when opening their music library. The Start Page would offer a number of convenient “jumping-off” points, pulling you into your library to explore it further — by artist similarity, maybe, or by play history proximity, after just a couple clicks.

The Timeline View is a zoomable timeline, shown here zoomed to a daily view. Zooming out could show you albums played within recent weeks; then months, quarters, etc. These albums might be sorted by Periodical Impact, something I explained in depth here; essentially they would be sorted not by the raw number of times they were played within any given period, but by how distinct they were to that period.

Even these meager ideas are leagues ahead of what’s available, and I’m not even a data analyst. Just imagine how a library’s play history data could be exploited by somebody trained in these things.

Blueboy’s Unisex, on the other hand, will probably always remind me of the winter of 2006–7, as I listened to it for the first time that season, nine additional times within that season (racking up about 150 tracks listened, according to Last.fm), and virtually never again once spring hit.

Ever since I began submitting listening data to Last.fm in November of 2004, I’ve wondered whether I’d ever enjoy direct access to all those numbers. Then came Last.fm Extra Stats, mercifully collecting all my listening data for me in a tab-separated file that can be pulled into Excel and manipulated to my heart’s content. Here, as a small example of the data, are my top ten artists (by tracks listened) from winter 2006–7, along with total listens for each artist (since November 2004) (now that I’m finally getting around to publishing this post, all the following data is very old):

Now for some methodology. We only have two numbers available to us here; let’s call them S (seasonal total) and T (total total). As I said, the psychological impact of a band/record over a given time period depends on both the amount of listening during that period (S) and the percentage of total listening within that period (S/T). Neither figure is good enough on its own; I may have listened to Destroyer 102 times this summer (I did), but that’s insignificant given my 1,500 overall listens. And I may have only listened to The Ladybug Transistor this summer (yielding a 100% concentration), but 3 listens is hardly enough to make an impact.

But, since both figures are positive indicators of impact, we can just multiply them (S²/T). This would give Ladybug Transistor a summer 2007 impact index of 3, and Destroyer, 6.92.

Let’s see how this multiplication affects winter’s top ten artists by tracks played:

Most notably, Ratatat is knocked down a few notches (on account of how consistently I listen to them), The Moldy Peaches go up a few spots — and if we reorder the entire list (of 78 artists) by impact index, a couple more eke into the top ten:

This last table is what we’re really after, theoretically giving us the ten artists who will most strongly remind me of the winter of 2006–7. It seems a bit boring at this point, since it’s pretty close in its results to the standard top-ten-by-listens chart; but the methodology is sound, and under more extreme listening conditions (e.g., a Destroyer obsession), it will prove to be a reliable indicator of impact.

And don’t forget, as time passes and I listen to these artists more, their season-specific impacts will drop. For instance, as things turned out, I listened to Television Personalities heavily over the spring, increasing their T value, and reducing their impact in the above table (which uses old data). As a result of my TVP spring listening, their winter index dropped dramatically from 35 to 7.95 — their spring index, as of October 5, 2007, is 91.95.

Depicting this analysis visually could go several ways; somehow I want to see a pie chart. Making one from every artist would be impractical, leaving a circle with a hundred or more slivers. Taking the top ten artists feels insensitive to the quirks of any given season; what if I don’t even listen to ten artists that season, for instance? What if I listen to a thousand? I could select only those artists who have indices greater than, say, 20; but this also feels arbitrary and vulnerable to seasonal quirks.

I think a pretty safe threshold above which to choose artists is at 50% of the sum of all impact values. In other words: the sum of the impact indices of winter’s artists is 783. If I take artists off the top of the list until I have half that, I stop after the fourth artist, The Moldy Peaches. Admittedly, a four-sector pie chart is kinda dull, but I need to be consistent in the creation of these charts so that impacts are accurately represented from season to season.

So, roughly…here are what the last three seasons of 2007 “meant” to me:

What I like about these is that they are not only internally consistent, but consistent with each other; Ariel Pink will remind me of the spring of 2007 more than M.I.A. will remind me of the summer, as represented by a larger sector in his season’s pie. I could similarly generate these with a focus on albums, which would be slightly more specific and alter the results dramatically. I know, for instance, that all my summer 2007 listening to M.I.A. was off her new record, songs that necessarily didn’t contribute to any prior M.I.A. listening — so, Kala, were it to have a sector in an album pie, would have one much larger than the less specific “M.I.A.” one.

Granted, there are more than a few problems with this whole endeavor, and here are the major three:

1. Last.fm does not track things that I listen to on CD/vinyl. This is a serious problem, as the things I purchase tend to be the things I like the most, and go totally unaccounted for.
2. The number of tracks you’ve listened to by an artist is only kind of related to how much time you’ve spent listening to them. 4 tracks can fill 2 CDs (Lift Your Skinny Fists), or you can cram 91 tracks into 40 minutes of music (Irrevocably Overdriven).
3. iPod listens are not properly documented. Even if I remember to report them to Last.fm with AudioPod, multiple listens on a single track only get counted as one — this is particularly damaging on long trips, for instance.

The (S²/T) formula may have further applications, as I’ll touch on in a follow-up to my original post about alphabetization and digital music library navigation.

Advantages of sorted lists include:

• one can easily find the first n elements (e.g. the 5 smallest countries) and the last n elements (e.g. the 3 largest countries)
• one can easily find the elements in a given range (e.g. countries with an area between .. and .. square km)
• one can easily search for an element, and conclude whether it is in the list

The first two advantages are things you almost never need to do with music libraries. And the third has been supplanted by now-ubiquitous search boxes: if you know what you’re looking for, you search; and if you don’t, an alphabetized list is not the way to find it.

Web visionary Ted Nelson (<mst3k>Dr. Ted Nelson?</mst3k>) has been paraphrased as pointing out that “electronic documents have been designed to mimic their paper antecedents,” and that “this is where everything went wrong: electronic documents could and should behave entirely differently from paper ones.” If the folder metaphor is inadequate for digital documents, no wonder it’s so pitiful at handling music. The proximity between pieces of music in a library should least of all be based on the first letter in a band’s name – it’s as arbitrary as sorting them by the vocalist’s month of birth – yet this is how it’s universally done.

Music library organization needs to be re-thought from the ground up. We need to consider how it is that people used to listen to music before it was all on their iTunes. How are your CDs organized (or disorganized) on your shelf? How are they organized in your head? What is it that prompts you to listen to what you listen to when you listen to it? And how can we use computers to adopt and enhance these ways of thinking, rather than forcing us to think like computers?

Multi-Dimensional Sorting

The most natural method for organizing music (if you can escape alphabetical thinking for a moment) is by similarity. Last.fm does this, and it is invaluable. When you are at an artist’s page at Last.fm, you feel that you are in that artist’s “neighborhood,” with links to similar bands, tags, listeners, and related groups. The Last.fm architecture was designed to manifest organic, bottom-up communities around bands and genres. This is an experience that cannot currently be replicated in any music player, at least not easily (with the possible exception of Amarok). But because Last.fm’s data is extraordinarily accessible, there are virtually no obstacles to incorporating this sense of “musical neighborhoods” into a piece of software.

If you don’t already have an artist in mind whose neighborhood you’d like to browse, you probably at least have some idea of the kind of mood you’re after, and there are several approaches here. One (perhaps the least viable) is using Last.fm’s tags. These are actually less often concerned with mood than they are with genre, a taxonomy well-known as being inconsistent and, I would argue, misguided: When I want to listen to Lullatone, it’s not because I want to listen to electronic music; it’s because I want to listen to “whimsical,” “delicate,” “innocent,” “sparkling” music. Who cares what genre it is?

AllMediaGuide began a project called Tapestry some time ago, an application of their vast mood/situation/genre dataset. It is an ideal solution for browsing music, and its integration into desktop software would be hugely rewarding. It’s possible to simulate Tapestry with foobar2000 and some elbow grease, but the results are not as robust as they could be.

Again, if you already know what you’re looking for, it would be difficult to find it through these channels; but this is what search is for.

Personalization

We also need to consider the less objective and more personal reasons that music becomes relevant in specific contexts, analogously to the way in which CDs become disordered on one’s shelf. I, for instance, usually have about 20 albums littering the top of my receiver and speakers. These include, roughly, (a) stuff I just bought, (b) stuff I just listened to, and © stuff I haven’t bothered putting away because I know I’ll listen to it again soon.

The main obstacle to browsing in this way is a prevalent shortcoming whose symptoms are far-reaching: the fact that music players “think” in terms of songs, not in terms of albums—or even in terms of artists, for that matter: My music software doesn’t know that these 38 songs are all by Electrelane; it just knows that their artist metadata is alphabetically adjacent. Sure, you can sort iTunes libraries by data such as “last listened” and “added”; and you can use CoverFlow to simulate a pale approximation of a flesh-and-blood record collection; but the only way you can sort albums or artists is alphabetically. I’ve written about the ways in which MP3Toys addresses this problem, and it remains a commendable pioneer in music management, but its difficulties (a steep learning curve, a buggy interface, a rapid release schedule) outweigh its advantages.

Browsing your own music library is a very impersonal experience, despite enormous potential for personalization. Rich info visualization “toys” such as Last.fm Extra Stats and LastGraph are seen as novelties, but would, in fact, be revolutionary as library browsing environments. There is nothing to prevent this from development, either; even users who are not plugged into Last.fm could have their listening history stored locally by their music software, which could then be used to generate small, cached, infinite-resolution SVG histograms, browsable by zooming, panning, and clicking. Far from being cumbersome and CPU-intensive, it would actually be rather elegant.

Continuing on the theme of chronology, what about a simple calendar charts view, with varying granularity by day, week, month, quarter? Presentationally, these charts could even be made easily to resemble vertical stacks of CDs, with spine art generated from a cropped cover image and overlaid text. This would arguably be eye-candy, of course, but just imagine how it would “feel” to see your music this way. If there’s one thing Apple’s been consistently right about, it’s that functionality is not at odds with a pleasant user experience, but rather that they are meant to be mutually supportive.

When I was helping to establish some playstamp tagging standards with the foobar community in 2005, it was jokingly suggested that when songs are played they be tagged with the current weather. Despite the sarcasm, I couldn’t help but think, “What a great idea!” I know my listening habits are affected by the weather, and I can’t imagine other people are not the same. There is, after all, a frequently-used “rainy day” tag at Last.fm.

What about a histogram based not on play count, but on hotness values over time? What about artist similarity webs based not on Last.fm data, but on proximity of play times within your personal history? What about taking lessons from the DONTCLICK.IT project, BumpTop, and the pile metaphor for unprecedentedly fluid user interfaces?

What Now?

All the ingredients are there. Everyone is rapidly moving towards an exclusively digital music collection, and the technology is embarrassingly outmoded. Music has become a major component of computing, at levels once reserved for word processing and gaming. Our relationship with our digital music collections is poised for reinvention, a looming difficulty that has been made invisible by custom and habit. Digital music management is hell, and users have complacently accepted this.

The obvious solution at this point is Songbird. Songbird’s media views (present since 0.5) allow more easily than ever for custom browsing environments. Previously the only way to alter your music browsing environment was to switch programs entirely; besides which, nearly all available programs simply mimic the well-known disk/directory views or iTunes’ browser pane view (which is just a glorification of a disk/directory view anyway). Songbird, on the other hand, boasts an unprecedented extensibility, coupled with media attention, ease-of-use, and the Mozilla platform, for which people have been developing extensions for ages (in computer years).

The prospects are thrilling and the potential for innovation is virtually limitless. Promisingly, there are some glimpses of where things might be headed for Songbird media views: Catalogue View demonstrates a novel visual presentation of your library, though it doesn’t do much in the way of organizational presentation; and Metrics Media Page is the beginnings of the kind of infovis view that could be (but currently isn’t) adapted to allow for actual navigation. Nevertheless, I have a bad feeling that this opportunity will be missed, as the status quo continues to obfuscate these possibilities.

Update: Part II

Anyway I used to use MediaCoder to convert to flash video, but when it gave me errors, and refused to tell me the specifics of those errors, I took it old school to the command prompt with FFmpeg (which MediaCoder uses anyway). This gives you a lot of useful info about the source file you’re encoding, such as audio sampling rate, frame rate, etc.

Wanting to find a balance between picture quality and streamability, I began encoding a short length of AVI video at different compression levels. FFmpeg calls this “qscale” (a way of representing variable bitrate qualities, much like LAME’s –V parameter), and the lower the qscale value, the better the quality. The available qscale values range from 1 (highest quality) to 31 (lowest quality). Going worse than a 13 qscale produces unacceptably poor quality, so that’s as low as I went for the purposes of this test.

I encoded 3:14 minutes of an AVI, resizing it to 500×374 pixels, and encoding the audio at 96kbps and 44.1KHz, which sounds fine, and is a negligible part of the ultimate file size, so going lower wouldn’t be very beneficial. Plus I find that good audio can create the illusion that the whole thing is of higher quality. Poor audio just makes it sound like “web video.”

Here are the results, courtesy of Google Spreadsheets:

The filesize, of course, goes down as quality goes down. And the loss in filesize also decreases, not just in amount, but in percentage as well, as indicated by the red line. For instance, the value of the red line at qscale 3 is 33.97%, which means that in going from qscale 2 to qscale 3, 33.97% of the filesize is shaved off.

However, because these losses are not perfectly exponential, I knew that there had to be qscale values that were more “efficient,” in a sense, than others — values that, despite being high, and causing a lower change in filesize than the previous step in qscale, still caused a comparably large change in filesize. For instance, still looking at the red line, you’ll notice that going from 2 to 3, as I said, shaves off 33.97% of the filesize, while going from 3 to 4 only shaves off 23.93% of the filesize; and that is a 29.56% decrease in change-in-filesize, which is a relatively large cost. We want the change-in-filesize to remain as high as possible for as long as possible.

Now, if you follow the red line from 4 to 5, you’ll see that that’s a 20.32% loss in filesize, which is pretty close to our previous 23.93% loss in filesize in going from 3 to 4. In fact, we’ve only lost 15.09% of change-in-filesize from the previous step. So these are the values we really want to examine: change in change-in-filesize, represented by the orange line.

This is nowhere close to exponential, nor does it follow any predictable decline. It darts around, seemingly at random. And we want to catch it at its lowest values, at points that represent changes in qscale that were nearly as efficient as the previous change in qscale. So the most desirable qscale values become, quite obviously, 5, 9, and 11.

What this means is that if quality is your primary concern (and you’re not crazy enough to encode at qscale 1), go with 5. qscale 5 turns 3:14 minutes of video into 30.62MB, which requires a download rate of 157.84KB/s to stream smoothly. qscale 11 will give you about half the filesize, and require a download rate of 77.37KB/s. But, because that’s the level at which picture quality really begins to suffer, and because most people don’t really mind buffering for a few seconds initially, I’m probably going to stick with qscale 9, whose videos take up 91.58 kilobytes per second, and which is by far the most efficient qscale anyway, with only a 4.92% change in change-in-filesize.

One caveat: This whole examination presupposes (as far as I can tell) that if it were possible to measure and chart the changes in the actual perceived visual quality of videos encoded at these qscale values, the curve would be perfectly geometric or exponential, with no aberrations similar to those above, and with all extrapolated delta curves showing no aberrations either. Given that, it might be easier to believe that every step you take through the qscale is of equal relative cost, and that there are no “objectively preferable” qscale values. But that is a lot more boring.

My foray into MP3Toys was ultimately short-lived, brought to a halt when I found what people were doing with Single Column Playlist for foobar, particularly the playlist-embedded album art. Back in the foobar saddle, I also gave in and tried out the “official” Play Count component, which I had avoided for so long because it didn’t support %FIRST_PLAYED%, and because I wasn’t sure I wanted my playback statistics only kept in the database — even though writing them to the files posed a lot of trouble as well. Turns out, playback statistics stored by the official component are less sensitive to changes to the files it’s keeping track of than the unofficial one, which means I only have to be a little careful to keep all my stats intact, while being able to play and track files that I’m still seeding.

This, along with the invaluable $cwb_datediff() function provided by Bowron’s new foo_cwb_hooks component, called for a rewrite to the hotness code, which had been stagnating in some marginally compatible 1.5 version since May. After severely trimming the code down and robusting things up, I thought of a new and totally non-arbitrary way to soften the blow hotness scores receive when songs are played. I hated seeing them leap to 100 every time, and this new softening method makes so much sense, utilizing existing baseline calibrations to keep things a lot more interesting. How anybody tolerated the old method is beyond me.

Anyway, here it is.

I also dug up a lot of old screenshots this week and I’m planning a nostalgia-fueled retrospective in the near future.

It’s pretty cool. So, uh, here it is.