Thoughts about a news algorithm

Amazon Recommendation system based on your personal profile
Last week I was reading a Wired article (March edition) about how the video rental service Netflix is awarding $1.000.000 to the person or group who can improve its recommendation algorithm by 10%.

Todays popular websites use smart algorithms to determine what we want or might like. Google is famous for its mix and so is the Amazon recommendation system. Your actions tell these systems about your behavior. And your actions make these services better in recommending you information. For example Google tracks what results people click. If most users click the second search result they make this the first result.

I love news selection
I really like how news websites, blogs and other person driven websites make a selection. Most often this works best if there is a sharp focus. A popular blog can’t be about everything. It has to be about a person or a subject to keep the blog interesting.

In the future this fragmentation might be happening to news websites as well. The traditional newspaper told you everything. It was your primary source of information. With websites we see a different pattern. People don’t just read one news website, they read many. They might have a favorite, but it is no such thing as exclusive readership. Will we see focus in newspaper websites as well? Although media operates independent it is almost always marked as ‘left’ or ‘right’ by the type of stories they focus on.

The news algorithm
Why wouldn’t news sorting be captured in algorithms? There is nothing that makes this impossible. Stories are written as closed interchangeable containers. News websites might make a selection on the frontpage, they also provide lists and rss-feeds where they sort the same information on time or popularity.

Journalists have multiple tasks, they create stories and they sort them on relevance. Maybe with this sorting we can experiment and create a more personal version as well?

Sorting news by machines
Sorting news is not just making a selection on popularity. Sorting news by systems is difficult. The presentation of what you like consists out a complex set of variables.

  • What do you like (personal interest)
  • What you might like (if you like a subject you might like to read about)
  • What do you need to know (because it is important to you, and it will dominate the media landscape for a while)
  • What everyone needs to know (breaking news)
  • What do you officially don’t like, but occasionally read (the stories everyone says they don’t read but always seem to get the highest click-through rates)
  • What do your friends (colleagues) read (news creates conversation and small-talk)
  • What do your friends recommend (you trust your network)
  • What you don’t want to know (things that really bore you and are irrelevant in any way)
  • Where do you like to know more about (if you are an expert in something you don’t want another article that explains it all again. You would prefer analysis and background articles)
  • What is your (current) location (for large groups of people location based information has extra value)
  • Surprises (they change your interests and habbits)

* If I forgot something please ad your thoughts in the comments

These are the variables that construct personal relevance of a news website. It’s a complex set, but if you can manage a good balance you are able to create a website that sorts news by personal relevance on another level than we are used to.

I don’t know if an algorithm can create a better news experience and what it should look like. I do think there is value in tracking and learning form your users behavior and return new or additional value to the reader.

Update: Concept Design

What this could look like and how you can keep this simple for the reader. The text is in Dutch. The screens ask for your location, favorite topics, company you work or would like to work and friends.

Share this post
Tweet about this on Twitter0Share on Facebook0Share on Google+0Share on LinkedIn0Email this to someone
  • http://www.ingejanse.nl Inge

    I think Google News does the basics of this already in their personalized news section (suggested reading for [account name]). I’m not certain how many parameters they include, but I assume only click throughs while being on their site.

    As much as like the idea (more relevant news, less skimming), I also resent the proposal to only show me news that I might like. I just started reading a news paper, and it amazes me how much I like all the stories that I normally would never click on (and that also don’t match a pattern of related news that I read). I don’t think any system for proposing news can substitute for that experience of stumbling into something genuinely new and unexpected.

    Another example: last.fm tells you about a great deal of new artists that perfectly match your listening profile. However, since I don’t listen to, let’s say, dixieland (or anything related), I will never get the opportunity to learn about it (unless I start searching for it myself). Hence, the last bullet you mention (surprises) seems as a dramatically important one for me.

  • http://www.designvsart.com/blog/2008/04/10/should-a-website-design-show-its-precious-algorithm/ Alexis Brion

    Hi Wilbert, I like your article a lot. It’s interesting to me because I’ve recently written a similar one but with a more “web design” point of view. Amazon algorithm is great, right?

    If you want to have a look at my article you can get it here:

    http://www.designvsart.com/blog/2008/04/10/should-a-website-design-show-its-precious-algorithm/

    Alex

  • http://www.hypernarrative.com Wilbert

    @Inge True, a human (editor, friends) selection is special and I don’t know if it can ever be replaced. I don’t think we only want relevant news, but more relevant might be something that could be interesting for news websites. People read local newspapers or newspaper sections (sports, art) because it is more relevant.

    The ‘stumble upon’ articles you are referring to are probably the extra and unique stories the newspaper writes about. This works great on paper, but would you also read these articles on the web? Where you allow yourself less time to discover it?

    For an online news service that only focus on ‘the now’, like Google News. Personal relevance might be something that makes it more valuable.

    What I wonder about is what are the things that construct a good news experience?

    - Being surprised is definitely an important one.

  • http://www.hypernarrative.com Wilbert

    @Alex, thanks for the overview, nice link.

  • Pingback: Automatically generated profiles? on hypernarrative.com, a weblog by Wilbert Baan about Art, Media and Technology

  • Pingback: Blogroll » Automatically generated profiles?