Perhaps that’s somewhere to look for inspiration in the design. On the surface a system like Google Adsense seems unrelated, but their systems do the sort of large scale data mining and heuristic processing.

Ideally under a system like this you will find The Beatles next to The Beach Boys and next to Oasis. Or, if we want to get really tricky, all three of those bands will have the same X and Y values, but Oasis may have a different Z value as a result of their being influenced by ’60s pop, and not a part of it themselves. A 3D mapping like this makes things a little more complicated than is practical, but it’s worth thinking about.

Last.fm may be recommending bands to you that you don’t like, but if the Last.fm similarity data were applied to your own library, then you would only be presented with things you presumably like (since you own them). And, although Last.fm may not be perfect, it’s probably the most thorough and available dataset out there — and, in my opinion, it’s pretty damn good.

If you prefer to listen on a song-by-song basis, then yes, I admit that this becomes a more difficult problem to handle. AllMusic and Pandora do possess mood-centric metadata at the song level, but don’t make it publicly available. Without their cooperation, personal tools might be the only way to attack this problem, such as Moody for iTunes or (the now defunct) MoodLogic.

On the album level we only have mood-centric metadata from AllMusic. This is subjective, of course, and even dealing with full albums on a mood-based level is methodologically questionable, since many albums are all over the place in terms of mood. So we either (a) just accept that this is a fundamental problem with describing albums by mood, or (b) devise some algorithm to apply the moods of every song on an album to the album as a whole, which would make an album’s inclusion under a particular mood not binary — i.e., either it is or is not “sad” — but on a gradient — i.e., it is 80% “sad.”

Personally, with my hack-ish implementation of these moods on the album level in foobar2000, I find that I get very good results. And though there are albums that are emotionally all over the map, there are also many that are emotionally pretty stable; think about Loveless or Slanted and Enchanted. Furthermore, even if every song on an album is not perfectly described by the moods used to describe the album, I would argue that in most cases, albums as a whole, even if they vary in mood, either do or do not complement the mood of a situation. AllMusic will often describe albums that are emotionally erratic as “cerebral” or something, rather than strictly “bleak.” Check out the full list of AllMusic’s mood descriptors and you’ll see that it’s actually an admirably nuanced system.

On the artist level, this is where mood breaks down. Although summing up an album by mood is a dodgy but viable endeavor, the same can’t be said for artists. Browsing artists by mood would be fruitless, so I don’t really propose it. This is why a 2D genre/style map is in my opinion the best way to organize artists. Last.fm already does this with their “Islands of Music” experiment, which, as an interface to a desktop music application, would be bliss.

Grandmaster Flash is described by AllMusic as being: Boisterous, Brash, Party/Celebratory, Confident, Bravado, Playful, Visceral, Freewheeling, Energetic, Gritty, Intense, Exuberant, Ominous, Provocative, Aggressive, Rousing, Somber, Confrontational, Cathartic, Dramatic, Searching.

Louis Armstrong is described by AllMusic as being: Warm, Lively, Freewheeling, Carefree, Amiable/Good-Natured, Earthy, Cheerful, Joyous, Playful, Boisterous, Earnest, Romantic, Gleeful, Rousing, Energetic, Fun, Confident, Whimsical, Exuberant, Elegant, Rollicking.

Where would this put these artists in AllMusic’s more general mood categories like “Fun/Good-Natured” and “Slick/Smooth” from their Tapestry project? I don’t know — that’s too much data to extrapolate a quick guess from. But even if they were neighbors in mood, so what? Like I said, if you have a decent mood categorization system, genre becomes irrelevant. Hell, right next to each other in this image I have cLOUDDEAD (experimental hip-hop) and Cyann & Ben (psychedelic folk-rock) under Bleak/Cold. And I can tell you, yes, both of those albums are pretty bleak and pretty cold, and if one would complement my mood at a given moment, there’s a good chance the other would as well, despite the two being from vastly different genres.

This conversation too has strayed from another major point I was trying to make in the article, which is that your own personal play history within your music player provides a very rich dataset to be exploited. I myself have spent some time developing an algorithm to describe what I call “Hotness,” which looks at how frequently and how recently you’ve listened to any given song to give you a snapshot of the artists that are more or less your “current favorites.” And despite being admittedly rudimentary, it still produces vastly more interesting results than simple “last played” and “number of plays” sorting or “smart playlists” (“smart” — hah!). And I’m sure somebody smarter than myself could invent far more compelling ways to look at your library that are based on your play history.

There are a lot of ways this could all go, and none of them is perfect. But I think it’s important to start thinking about this, and for people to start at least attempting to create a richer, more dynamic interface to our libraries. I’m not an expert programmer by any means, and am just now trying to verse myself more thoroughly in Javascript and XUL to start working with Songbird. But I was hoping to at least inspire conversation with this post, and to potentially reach the right person who both shares my ideas about what these solutions might look like, and knows enough to implement them somehow.

If you associate artists, how do you link bands like (just taking the first one I think of) The Beatles? Are they close to the Beach Boys, the Status Quo, ELO, the Monkees, and even Britpop like Oasis? Are they close only to some? The Beatles started out with Please Please Me and Help!, but finished with Sargent Pepper’s… If one considers the Beatles as a whole, they become linked to so many bands there’s little point in linking them in such a way… I only like their Psychedelia albums, and I definitely don’t like their early pop tunes. I have no interest in the Monkees, but I like the Status Quo and the Beach Boys. To go back to the author comparison I made earlier, who is Wilde close to? The Portrait of Dorian Gray and the Ballad of Reading Gaol are so dissimilar (and then there are all his plays that are once more a different tone) that he can be linked with far too many people to make any links useful to someone who is only interested in the Ballad of Reading Gaol (or the Portrait of Dorian Gray).

So what about albums? That’s already a better proposal, since it would make an artist’s evolution clearer. “Help!” and “Sargent Pepper’s” can be linked because it’s the same band, but at least groups like the Monkees will be kept away from Sergeant Pepper’s.

Unlike you, I listen more to just one track of an album at a time. Create playlists, play them. So I wouldn’t be very interested by grouping music by album. After all, I feel that between Lucy in the Sky with Diamonds, She’s Leaving Home, A Day in the Life and Sergeant Pepper’s Hearts Club Band, there are differences so great I don’t always feel like listening to all of them one after another (or in such a short time frame). So I’d be a proponent of linking each song… I think you can see where this is going. It’s going to be a nightmare to link songs individually, and even harder to recognise if said link is worth it (how many hundred of thousand of songs imitated Lucy in the Sky With Diamonds’ sound, and how many were any good?).

The problem is that liking or disliking (or being indifferent) to a track seems to me as something very subjective. I think a lot of people (or at the very least myself) like music that is very diverse. From Joy Division to the Happy Mondays, Dizzy Gillespie to Bill Haley, and countless others, with each time tracks that I like more and others that I like less… What does “bleak/cold” mean when talking about Louis Armstrong? Is it the same as for A.I.R and Grandmaster Flash? All these appreciations are ultimately extremely personal. Last.fm suggest I listen to Alain Souchon, Luke, Renan Luce, Cali, BB Brunes, Francis Cabrel and Bruel… Except I already know all of them and I dislike them. Yes, they sound like some artists I listen to (funnily enough, not artists I listen to a lot, so perhaps last.fm’s ranking system needs to be re-tuned), and lots of them have fans who are also fans of bands I like. But when my top fifteen only has three french-language artists (according to last.fm), why suggest so many french bands, and –NO– english-language bands (nine of those fifteen)?

These ideas are very interesting, but I think I’ll stick to making my own playlists, and every now and then either going straight to a track/album/artist I want to listen to, or just putting on the shuffle mode.

I’ve drifted away from the book comparison I started with, but I do find it similar. We don’t rank books by the author’s name, and organising music by the name of the composer/performer is also not the best. However, once the topic is broadly defined (novels, refence books, psychology, philosophy or rock, classical, electronic) it becomes slightly futile to attempt to continue classifying them since any further division will likely prove more confusing for the user than helpful. If Victor Hugo’s books were separated more than between “plays”, “novels” and “poetry”, would it be useful? In my opinion, no. (However, I still find these different “organisations” mightily interesting, and perhaps one day I’ll feel compelled to use them and find them brilliant!)

I dual boot, and like running very stripped down music players. I detest iTunes. On my Linux partition I like XMMS. Windows…I’ll admit I still love Winamp. I haven’t broken free from that interface. I know there are all sorts of library based music players for Linux, such as Amarok. I know there are all sorts of fancy library based players for Windows…but I don’t know that I’ll ever get past straightforward music players.

The groundwork has sort of been laid for that already. There are many music services that offer tailored experiences…rate a song up and you get it more often (sometimes even similar songs more often). Rate it down and you never hear it again.

You could create a personalized system where you identify your mood, or what sort of music you’re interested, or what color socks you’re wearing…then rate songs up or down. In theory, with time, it could simply train itself to serve music based upon those factors on just your hardware.

To do it well would be expensive, a central database could combine the data from people, perhaps sort it by overall preferences (for instance, sometimes I like The Distillers…so perhaps my opinion is held to be less relevant on when I want to hear Paul Simon for someone who just listens to light rock).

I think the problem is that a truly intelligent, easy to use, service would take a massive amount of infrastructure. It would have to deal with music on a song, album, and artist level…tying all of those to musical taste and emotional description. It would either have to be based upon a sales model, or be done by a company with enough market penetration (Read: Google) to mine data at a very massive rate.

As far as the “ton of work to create and maintain” goes, Adam’s right in that there’s metadata everywhere. He mentions Pandora, which is kind of a top-down approach to classifying music: “experts” meticulously categorizing everything they can by musical “genome.” I prefer Last.fm, however, with its empirical, bottom-up approach to classification, based on tracking actual listening habits. That, and they have a long history of making their metadata free and accessible with public XML services for just about everything. So, in short, you wouldn’t have to do anything; your music player would know which artists should be grouped together automatically.

I suppose when it comes to a mouseless/keyboardless interface like the PS3 (or a portable player, come to think of it), alphabetical lists do make searching easier than having to “type” artists’ names; so in those cases, I’d have to concede that you’re right, Me. And I suppose what I’m describing will be useful only to people with large libraries like myself. But for those of us who have those large libraries, mining our collections is incredibly frustrating, especially when you realize that there are so many opportunities to create browsing environments that are not only more intuitive and functional, but fun as well.

Adam’s also right that these kinds of innovations should be “effortless,” yet it seems as though nobody’s innovating. Nobody has been thinking beyond the alphabetized list until recently, with Apple’s “Genius” feature and Microsoft’s “MixView” — and both of these seem to be efforts to sell music, not primarily to make your library more dynamic.

All it would take is a room of user-interface experts experimenting with different models to “push” your music to you in interesting new ways, based on artist similarity, based on your listening history, based on mood, based on time of day, based on time of year — hell, based on the weather and on news feeds. It wouldn’t merely be flashy; it would, I think, profoundly change the way you interact with your music library.

We need to complement existing interfaces with new accessibility and use the hardware at our disposal to do so. For example, the iphone has audio out and in, so how about analysing the “sound” of your music collection to fit the ambient sound level, or to give clues to the “mood” of where you are? Everything is net-enabled these days, and we’re not short of metadata to describe music. So these kind of connections should be effortless.

Technologically, we’re at a wonderful point in time where all these things are just becoming possible. We just need to get on with it and start making them. It’s almost as if music is on the verge of some audio equivalent of photosynth[1] being invented, and with the kind of metadata being formed by services like pandora.com[2] there are some exciting and unexpected music applications ahead.

[1] photosynth demo
[2] music genome

To a large degree that same classification can be done by good tags and a solid media player, plus it has the added advantage of being easy for someone else to browse and deal with on all sorts of players. If my wife is trying to play something on from my Tversity server on the PS3 she doesn’t need to know that I’ve classified The Violent Femmes as punk rather than folk, and in that case searching is inconvenient. There is the problem of knowing whether they’d be filed under T or V, but that’s more minor.

And even beyond the bands that don’t neatly fit, do you handle musical style by band, album, or song? If it’s just by band something like On Every Street by Dire Straits will end up massively misclassified. If it’s by album how do you handle Sticky Fingers by The Rolling Stones? And if it’s by song you then need a whole system of playlists to recreate the albums themselves.

There’s one simple answer to your question: If you want to listen to The Who, you put your mouse in that little search box in the corner and type “The Who.” If you’re using an alphabetical list to find artists and albums also, then that’s two tools redundantly serving the same purpose.

I’m not talking about making mixes, although that’s valuable to some people too. I, like you, am “tied to that archaic creation” as well — almost all of my listening is by full album. In fact I even mention in the article that one of the shortcomings of the most popular media players is that they don’t “think” in terms of albums.

What I’m trying to address is the problem of having too much music, and too diverse taste. I have 1,664 albums in my library, and I’m sure there are other people with thousands, if not tens of thousands, more. I don’t always know what I want to listen to; in fact, I’d wager that most of the time, I don’t know what I want to listen to. I know I want to listen to something, and that that something is in there somewhere, but I don’t know what it is.

If you asked me to list, from memory, all 1,664 albums in my library, I’d probably never make it past a couple hundred, and that would take days. If I’m in a situation where I have to figure out what to listen to, only the artists who are most familiar to me will spring to mind first, but that leaves me trapped listening to the same things over and over. What I’m after is a system that will know better than me what I want to listen to, based on my listening habits, artist similarity, etc., and which can help me to explore the things I have that won’t spring to mind.

A short story: yesterday I was perusing my physical CD collection (gasp!) to find a specific album I wanted to listen to while I walked to work. While looking for that one album, I found another album that I realized was more appropriate for my mood, for the weather, for the time of day, for how long it had been since I last listened to it…etc. Now, why didn’t I know that’s what I wanted to hear when I first approached my CD collection? It’s because my mind is very flawed and very finite, and can only know so much about what my library looks like.

The Who is a very well-known band, and one that I’m sure you’ve listened to a lot, so it’s easy for you to know that you want to listen to them. But wouldn’t it be a better system if, when you’re browsing (not searching) your library for The Who, you’d find them adjacent to The Rolling Stones, The Kinks, and Cream, rather than Wilson Phillips, Wilderness, and Wesley Willis?

It’s like I said in the article: If you know what you’re looking for, you search. If you don’t, then an alphabetized list is not the way to find it.

And I think your laundry analogy actually works in my favor. Shirts go together because they are of the same type of clothing, just as The Who and The Rolling Stones are the same type of music. I would never sort my clothes by their brand, for instance, yet this is analogous to sorting a music library by artist. I don’t care who made the clothes, I care that they are appropriate for the weather, that they match, that I haven’t worn them recently, etc.

Because many of us are tied to that archaic creation, the album. And many of us listen to, and judge, musicians based upon that criteria. I may be an old man at 26, but when I decide I want to hear The Who I put on The Who Sell Out, or whatever album I want to hear. I don’t care that it’s most closely associated with Britney Spears and then next.

There’s some value in organizing a music library with tags to easily create playlists based upon mood. That can be helpful, if you want mixed music. But why would I organize my music in any way but alphabetically? You provide an argument, but no evidence to back it up. I want to be able to find a band/album/song easily. Anarchy belongs in the streets, not in a library or music library.

When you do your laundry do you put your clothing away by % cotton, or do you put it away by type of clothing. I’m sure it will be very useful when you’re trying to find that specific pair of pants and have to dig through 20 shirts that happen to have a similar fiber makeup.