Parallax : Calculating Technology's Future

Trada Customer: PosterBrain

Trada's rocking along and growing quickly. Just wanted to give some blog-love out to one of our customers: PosterBrain. Here's their blurb:

www.PosterBrain.com is a local Boulder/Denver company that makes it addictively easy  to turn any picture into a poster overnight. Its a super simple web design, same-day shipping, incredible print quality, and great customer service. 

We have one of their signs on our front door. If you're promoting your business using PPC (like PosterBrain does) you should drop Trada a line and find out more about what we're doing: [email protected].

The Death of Newspapers (and the Rebirth of the Editor)

TechCrunch had an interesting link today to this article written by a newspaper industry pundit on the "Link Economy". His basic point is that the value of inbound links to news sites is overshadowed by the ability of site scrapers to use them and disintermediate the newspaper site from their readers.

Why won't newspapers realize their long term fate is sealed in the same way that the music industry's fate was sealed - if your business model has been based on controlled distribution and bundling of variable quality media units (e.g. songs on an album or stories in a newspaper) into one purchasable SKU, you'd better hope that you don't loose distribution control. Whoops!

Why won't the newspapers and the music labels realize the same basic thing: what they have on their side is the editorial (or curatorial or AR or whatever you want to call it) function. That's their bag. Monetize that! The music industry is figuring out how to do that with merch and live shows -  newspapers just need to do it with their own versions of the same (e.g. Wired and New Yorker's thought leadership conferences).

I wrote about this transformation a long time ago now and i believe my arguments hold even more true today than when i wrote this: 

http://parallax.blogs.com/parallax_calculating_tech/2008/06/editor-20.html

If the NYT won't own the editorial function for news, then I guarantee you someone else will. 

TechStars Global Domination: Step 2

The very successful TechStars mentorship-based incubator program has spread its wings to Boston:

http://www.pitchengine.com/free-release.php?id=4714

Congratulations to Brad Feld, David Cohen, Jared Polis and all the mentors that have made Boulder's first version of this so successful.

I am Being Stalked By the State of Colorado

Dear The State of Colorado,

I'm starting to think you're stalking me. 4 times in the last 2 days (including right now, on the weekend) your automated dialing system has called me resulting in the following back and forth:

IVR: This is the Colorado Secretary of State with a very important message about <one of my company's names>. Press #1 if you are the right person to handle this call.
Me: <pressing 1>
IVR: Thank you, we have an important and urgent message for you. Please press #0 to speak to an agent.
Me: <pressing 0>
IVR: We're sorry, there are no agents available. We will call you again at a later time. <hangs up>

I know I can be pretty intimidating some times but why do you keep hanging up on me when you find out i'm home? Was it you that i gave my number to at the bar last weekend? If so, just leave me yours and i will hit you back - I promise. Maybe i missed Secretaries day or something and you're just wanting an apology? Okay - i really am sorry - you're doing a great job! But can I tell you something? If this was an ex-girlfriend i'd actually be thinking about calling the police right now.

But Colorado, let's not start off on the wrong foot. Maybe you can pass this suggestion on to your IVR engineers: it's not really that complicated, just a little code change to your system

agent = AgentQueue.get_available_agent()

if  !agent.nil?
{
    lock(agent)
    call_tax_payer(agent)
    unlock(agent)
}

Just wrap this in a happy loop that checks the list for any unresolved cases and voila, no more restraining orderer needed.

I do need to get something off my chest though. I was amused the first time I went through this process and i even ignored the imposition that it was on my cell phone. But now i'm getting a little annoyed. If you don't have an agent available at least give me a phone number to call or an email address to write to or something!  If you wanna get ahold of me so bad you can find me in two minutes on Facebook! Maybe you're being very mysterious about this all for a reason and I admit I am a little intrigued. But that aside, this is getting pretty silly and on my personal cell phone I consider it to be at do-not-call spam levels. Not to mention it makes you and the Secretary of State look a little bit like bumbling idiots. Not necessarily the best fist impression to make with me.

Please, if you're going to use technology then at least write some decent requirements beforehand. And if you see me out this weekend just come over and say hi. I won't bite i promise.

Signed,
Baffled in Boulder

Putting the Crowd Horse Before the Cart

In my recent post The Wisdom of Sales Crowds i talked, among other bigger things, about the idea of crowdsourcing your dating choices. It seems like the folks at SideTaker probably agree with me that crowdsourced dating is coming and have already prepared a way to crowdsource the inevitabnle arguments you'll have in those relationships. I see a merger in the future!

The Wisdom of Sales Crowds

I can’t stand reading business books. I think they’ve become an inherent oxymoron; many extolling bold new ways of creating personal or business efficiency while somehow justifying 270 pages of my valuable time to explain it. Bugger!

At first pass, James Surowiecki’s “The Wisdom of Crowds” falls right into this category. The book is split into two parts – the interesting part and the part that makes the book thick enough to charge $14.95. The best business books though are the ones that have psyche-burning memes; those annoying simplicities that stick in your head like the catchy chorus of an 80’s pop song, constantly bubbling into your conscious long after you’ve put the book down. The “Wisdom of the Crowds” is Wham!-like in its ability to do this.

The basic premise of the book is (shockingly) that crowds of people can actually be pretty smart. The unintuitive part of the argument is that in certain situations the crowd is actually smarter than any individual expert in the group. To some extent this is an interesting counter argument to one of my favorite thought pieces, Jaron Lanier’s Digital Maoism. The classic example given is the gumballs-in-a-jar game where a group of people is asked to pick the number of gumballs in a jar. While the individual estimates vary widely, systematically the averaged guess of each person in the crowd ends up approximating the actual number of gumballs more accurately than the best individual guesser. It’s a bit of a trite example as I’m not familiar with many gumball counting experts - but let’s not move too quickly here, we have 269 pages to go.

A more impactful example would be the long term success of fund managers versus market indexes (fund managers rarely outperform the market over a long period of time). Another example would be the Iowa Election Markets where the “crowd” of people buying and selling political outcome futures has historically been quite accurate. “Wisdom of Crowds” is littered with other examples covering politics, economics, gambling, social experiments, and beyond.

Surowiecki’s justification for this phenomenon is that in most groups of people you have a distribution of localized or limited knowledge (which can also be regarded as a bias). By aggregating localized knowledge and bias, the outliers tend to cancel each other out and the resulting aggregate wisdom approximates reality. This is a sort of crowdsourcing version of dropping the high and low score in the Olympics to approximate the real value of any 15-year-old Chinese gymnast’s floor routine. Another way to consider localize phenomena’s affect is to consider that sometimes localized (or biased) information is actually the most influential to the greater outcome. Take for example Palin’s VP nomination. Some people will have a bias that her political earmarking will have a large affect on people’s opinion of her while others will dismiss it as a triviality. The results will show that one of these answers is correct. If you expand this information to include the 100 other accusations that have and will come up, somewhere there is a permutation of specifics that will affect the outcome. 

Surowiecki further argues that in many situations if you add in even a meager financial incentive to be right and allow individuals to refine their guesses over time, you tend to get an even closer approximation of the actual results (bookies evening out bets on both sides of a football game would be a good example of this). In general this tends to work well in situations where new information and developments over time will affect the actual results (e.g. new information such as the weather on a day the football game will be played or the outcome of the 2008 presidential election).

Okay, so enough about the basics – you get it. I just saved you $14.95 so don’t say this blog never did anything for you.

CrowdMatch.com

So I was sitting around trying to think of all the situations where the wisdom of crowds could outperform the experts. And then it came to me as I saw a commercial for Free Communication Weekend on eHarmony: dating!

I have been on very few blind dates in my life (with paltry success). I think the reason is that I don’t trust any one individual in my “dating recommender group” to understand what ends up being a good match for me. Usually an individual’s recommendation is based on what they want for me (or their friend) and not necessarily what I will enjoy. So it dawned on me, why not throw this decision out to the crowd in some kind of crowdsourcing fashion?

Here’s an experiment you can all run. Get 15 of your friends, family members and perhaps even exes (yikes) to sign up for eHarmony’s next free communication weekend. Have them all peruse the dating selections in your local area. Each person in your “crowd” would pick who they think is the best match based on their own biases about you. Perform some sort of reduction in a few rounds of elimination where each person must again pick who they think is your best match, but from the increasingly limited pool of possible choices. Finally you’d whittle it down to one (or a few) people that should theoretically be your best match. Why not outsource your dating choices to the crowd as they should supposedly be smarter than you (the self anointed expert of the crowd)?

The reason this might work is based on Surowiecki’s argument for local bias. Some people in your recommender crowd are going to be more sensitive to a physical match for you (assuming they know what you like), some are going to focus on hobbies you have in common, and some may focus on beliefs or career choices. Through the aggregation of these biases a true pastiche of what might make you happy may emerge. Come to think of it, this would be a really interesting application to build on Facebook. People propose friends from their network that they think you should date and then your friends effectively narrow down the field to the one (or few) people you should go out with.  When CrowdMatch.com turns profitable please just remember you heard it here first!

But Seriously..

While I believe that the results of CrowdMatch.com would have some legs (all puns intended), it dawned on me that there was likely a more valuable business problem in wait of a crowd solution. I think I have found the perfect problem. This situation is one that many of us have found ourselves in countless times; a situation where traditionally the (supposed) expert is listened to over the crowd’s opinion and where the results have historically been very poor.

Sales forecasting in startups should absolutely be determined by the wisdom of the crowd.

Let’s review why:

1) Sales forecasting in startups is generally mediocre at best (so it’s a problem that needs a different solution than the status quo)

2) The forecast is usually compiled by one person who is supposedly the expert (the VP Sales)

3) Every person in the sales ecosystem has only a limited amount of information during the quarter (only certain people have directly interacted with prospects, sales people are regionally focused, there may be limited knowledge of the true deliverability of a product in the quarter, etc..)

4) The expert simply cannot know all the localized information that impacts the outcome

5) New information enters the sales ecosystem constantly throughout the quarter

6) Every person in the sales ecosystem has a bias based on personal experience, style, compensation plan, or management approach

For a long time now I have had the pleasure of working with Wendy Lea. Among many other things, Wendy ingrained in me a fundamental belief that sales is not art, it’s a science. When you find yourself in a pipeline debrief and you notice you’re starting to hear more stories than data you know you’re dealing with a VP of Sales that either has no good news, has little definitive information, or doesn’t fundamentally believe in the theory of sales being a science not art. As a side note, Wendy and I have also worked Seth Levine in a number of board situations and he wrote a fantastic piece cum plea to rationality on his blog about this exact subject. The point of this preamble is that I have come to believe strongly that sales forecasting should be a rational and predictable exercise. So why is it so damn hard to get right? I’ll attempt to argue Surowiecki-style that it’s because we have learned (poorly) to trust the expert and not the crowd.

As background, I have always been close sales but not directly responsible for it. That’s allowed me somewhat unimposingly to have a lot of private conversations with people in the sales ecosystem about their own predictions. Will a certain deal close or not? What will the eventual size of a specific deal be? Will we make the number? Usually their predictions are based on their personal experiences in the sale process either working directly with revenue sources (prospects, existing customers in the pipeline that you plan on upselling) or their own interactions with the sales team (if they are not directly a quota carrying sales person). In hindsight, inevitably somewhere in the cacophony of conversations there have almost always been pretty accurate predictions for my questions. I’ll also note that rarely is it the VP of Sales that, no matter how much he or she talks to their team, has the closest prediction of the final quarterly number during the first couple months of the quarter.

The last point is particularly interesting because it shows a weakness in the expert driven forecasting model. Depending on the dynamic between the VP Sales and your sales team, answers to various questions are going to vary widely depending on who is asking the question (a peer like me or a manager like the VP of Sales). I believe this is a trickle-down of bias phenomenon. CEOs are inclined to stay optimistic to the board and thus the VP of Sales is inclined to stay optimistic to the CEO (even many times in the face of abundant information to the contrary). And thus sales people are inclined to stay optimistic to the VP of Sales. When peers talk about the quarter they predict without this implicit bias.

The Wisdom of Sales Crowds

So this takes us back to the wisdom of the crowds. Why not leverage the crowd to get the best possible forecast of the quarter as it moves along. Theoretically the collective perspective of the crowd would almost always be the best guess at any point in time. That doesn’t mean that the collective guess of the crowd would be correct on the first day of the quarter but it is likely that it would be the most accurate possible prediction given all the information available at the time. As the quarter progresses this guesstimate should get closer and closer to the final number in theory much more accurately than the VP of Sales could individually give. How many board members have been told by a VP of Sales that the quarter will be X only 2 weeks before quarter end and the final number is something dramatically off X (unfortunately almost always lower).

What’s fantastically counter intuitive to the classic expert-driven model, per Surowieki’s analysis, is that the more diversity you have in the crowd of estimators the better. Why not let sales engineers participate in the prediction? What about professional services? What about other executives that are exposed to prospects during sales discovery, or product managers who might have the most current customer conversations at their disposal? This diversity, while it might seem to allow uneducated opinions into the mixture, tends to lead to more accurate results. To compound the problems of a crowd limited to just direct sales people, remember that sales folks are many times driven from a perspective of wanting to hear that there might be a sale. Someone like a sales engineer who might not be comped on making the sale directly might be more inclined to hear the few “not gonna happen this quarter” signals thrown out by the prospect. The diversity should balance out these situations where an overly optimistic sales person might be counterbalanced by an overly pessimistic sales engineer.

What’s interesting about this is that other than informally no one I know does this, or even does anything like it! To some extent this is the result of hierarchical structures in an organization. In the same way the VP of Engineering is looked to for the definitive reality of a release schedule, the sales VP is looked to for the definitive reality of the sales forecast. Come to think of it, the wisdom of crowds here could be as equally well applied to project deadlines as to sales (engineering, professional services implementation, etc..). One unique oddity of sales though is that the VP of Sales, in my experience, has a more inherent need to manage and control expectations (even if they are dead wrong in the end). Perhaps it’s the quarterly pressure of predictions (engineering teams don’t get lambasted if they deliver a product on October 4^th instead of September 28^th) or perhaps it’s just bad old school behavior that we’ve not unlearned to question. While crowd predictions might be broadly applicable, for now let’s just focus on solving all the problems of the sales world.

Sales and Marketplace

So how would you implement this? While poor man’s approaches might work (an email survey here an email survey there), Suroweicki provides a much more elegant approach. Remember that predictions sometimes have more accuracy if there is an incentive to be right. A simple way to engage the crowd would be to set up a sales forecasting prediction market just like the Iowa Election markets. This market would allow anyone in the sales ecosystem to buy and sell shares in “forecasting assets” about quarterly sales outcomes. Consider a marketplace with the following “assets” that could be bought and sold freely between market participants (for real money):

1) Final results of the quarter in increments (say $750,000-1M is an “asset”)

2) Whether a certain deal will close (e.g. “ECOLab will close this quarter” is the asset)

3) The size of a deal (e.g. “ECOLab will be greater than 250K”)

4) If the quarterly goal will be met (e.g. “Quarterly target of 2M will be met”)

In this forecasting market all the participants can constantly buy and sell shares of these assets amongst each other based on their local information. This would provide an incentive for the participants because they could make real money in the transactions. You could also set up the market to payout to the most accurate participant (e.g. the person holding the most shares of the right final answer would get a $1000 bonus).

What would be interesting about this forecasting market is that the current and historical share prices of each assets could be made available to anyone: CEO, VP sales, board members, engineers, etc.. As the quarter progressed you’d see these share prices shift as new information came into the business. One could imagine that you’d see a lot of volatility in the share prices when dramatic new information came in as well. Imagine what you’d see in the forecasting market after a disastrous customer demo? A plunge in share values for the “We will make the quarter” asset. Or a run up in share value when a blue bird deal came into the pipeline. It would also be interesting to note if trading volume was high on an asset which might imply an opinion disparity between buyers and sellers around a forecasted sales outcome. Another interesting scenario would be when there was high transaction volume on polarized assets classes in the market (a lot of people buying “The Quarterly Number Will be High” assets and a lot of people buying “The Quarterly Number Will be Low” assets).

Part of what would make this forecasting market work well would be the anonymity of the buyers and sellers (as it is, with some exceptions, in the general stock market). This anonymity would prevent peer influence (e.g. if you knew all the sales people were buying high revenue assets you might be inclined to buy some as well). This would also solve the practical human problem of sales people not wanting publicly to disagree with the VP of Sales whose inclination might be to manicure forecasting information to the CEO or the Board until better information became available. This market would inevitably drive transparency into the sales forecasting process which would also require all participants to curtail capricious actions (e.g. if a board member sees the high revenue assets in the market tank they need to not get on the phone and start rattling the CEO’s cage).

The goal here would be to transcend the emotional nature of forecasting volatility (the “art”) in favor of better information for all the participants (“the science”). If for example a specific transaction asset dips (e.g. “Company XYZ will close this quarter”) the VP of Sales would have a leading indicator to try and figure out what was going on – a leading indicator they may likely not get from talking to just the account manager for that prospect.

Putting the Force in Salesforce.com

The more I have thought about this the more I am a believer that this is a fundamentally better approach to sales forecasting. I’d love for someone to build a forecasting marketplace application for Salesforce.com. I’d love to see (as a starting point) all Salesforce user accounts for a company be able to buy and sell shares on the market anonymously. Perhaps in the beginning all the forecasting marketplace “assets” for the quarter are automatically created from SFdC data (you can see which opportunities in SFdC are scheduled to close this quarter and what the predicted aggregate quarterly number would be). Or perhaps the VP of Sales can set up the assets each quarter. Eventually you’d like to see anyone participating in the market be able to “float” an asset that they wanted a prediction on. To some extent this approaches the generality of some of the new prediction market players like Predictify. The key here is to structure it as an incented marketplace and not just an opinion poll (which much of Predictify is).

Either way this approach would drive a level of transparency into sales forecasting that is sorely lacking. It’s not that most startup VP of Sales are bad, it’s just that the expert model doesn’t work with such broad diversity of localized information. And hey, and if nothing else you can finally have a way to pacify anyone that  that complains that only sales people make money off of sales.

Editor 2.0 - The Emergence List

As i see (or people point me to) Editor 2.0 types of products, I will keep a running list of them. This announcement got me off my ass to start the list as its pretty much in the sweet spot of what Editor 2.0 predicts:

Aug 1, 2008: Google (YouTube) acquires Omisio

There will be more!

Ruby Developer Survey

To all Ruby developers:

I've been talking to the guys over at Devver recently. They are cooking up some pretty interesting testing tools for Ruby developers. If you code in Ruby they'd love your feedback about the tools you would find most interesting in the pre-check-in phase of your development. Take 10 seconds and fill out the survey if you can.

The survey is here.

Thanks. I'm getting back to blogging soon so don't count my commentary out yet :)

Editor 2.0

It’s not often that I have an epiphany in a book store. Today, however, was the exception. I was lazily wandering around Trident Bookstore (a sort of Boston version of the Tattered Cover but with more humus options on the menu), looking in vain for a present for my mother. Granted I was distracted by the idle bookish banter going back and forth between myself and the friend I was with, but I finally got tired of the hunt, said “forget it!” and walked back up to the front of the store. And there, neatly displayed in convenient impulse-buy format was the book shelf devoted to employees’ picks of the month. From a small set of choices I was quickly able to find something I thought she’d like (The Red Tent). And herein came my epiphany: what the world needs now is not more content but more editors. And to boot, it dawned on me that a lot of the interesting startup ideas I have seen recently are following this trend - reintroducing the editor back into our lives.

You Can’t Make Choices without Bias 

Let’s step back a second and understand what’s happened in the world of media consumption in the last 10 years. Pre-internet (which sort of sounds like “pre-modern-day” now) pretty much everything we consumed was controlled by a limited number of sources. Whether it was classic fourth estate newspapers, Sam Goody and the Wherehouse, or a small offering of cable channels, our access to content was limited. Of course not every newspaper or television channel that leveraged its distribution monopoly into our homes became successful. Those that rose to the top did it through excellent editorial oversight.

Editors essentially have two functions: pick the most important content from the various sources they have at their disposal and manage the brand or positioning that is projected through those choices. That said, the blessing, the curse, and the not so unobvious dirty little secret of said editorial function is that it introduces bias into the equation. In the end though, it’s this bias which leads to consistent editorial choices. 

Slouching Towards Authorgeddon

Take a newspaper editor’s choices of content for example. They have AP feeds, staff writers, freelance writers (stringers), op ed pieces, etc.. all at their disposal when putting together their dailies. On top of that, they have multiple choices about richer media with which to emotionally paint their pieces. Remember the bru-ha-ha [scroll down] over the differing OJ Simpson covers on Time and Newsweek? For every issue, an editor makes 1000s of choices to about what to print and what not to. 

A friend of mine who is the editor of the Atlantic Monthly recently told me he has over 12,000 non-fiction submissions (not including his own staff’s work) to choose from a year. And in the television world, through a friend of mine who has written for CSI, Vanished and Chuck, I have heard about the reams of spec pilot scripts that NBC, CBS, etc.. have to choose from each season. Any way you slice it, the editorial function has been busy at work for us over the last 10 years.

This all changed one day when someone inspirationally stuck the word “citizen” in front of “journalism” and  “media” in every 2002 VC presentation about the changing face of online content. And with that one small step, a giant leap for mediakind was taken. 

With the dawning of user generated content, we have cast aside the editorial function, and in the process migrated from a consumption generation to a participation generation. Blogs, YouTube, Wikipedia, Podcasting, Facebook, Flickr – the alphabet is littered with participation economy portmanteaus and poorly spelled company names that allowed anyone with an internet connection to become a content producer. The last time I checked, Technorati reported 57MM blogs in existence (its probably much more by now). As of 6:00pm EST, MySpace has had 75,195 videos uploaded TODAY. And last year’s 90,000 member strong National November Writing Month [www.nanowrimo] novel-in-a-month club generated 473,201,894 words all slouching further towards Authorgeddon in only 14 days!

None of this goes through an editor. It’s all headlines on the global front page that is the Internet. Unless I get to start using more than 10% of my brain, I think I’m going to have to pull a Gary Larson pretty soon: 

The Production Curve and Community Percolation

Okay, so after a few years generating all this stuff (insert appropriate George Carlin “stuff’ reference here) we clearly have the problem of too much stuff. How do people end up picking what to read and watch and what to ignore. Well, what we’ve seen so far is three emergent solutions to this problem. The first is what I would call the quality over quantity (QoQ) approach, the second I’m going to name “community percolation”, and the third is the authority approach. 

Quality over Quantity (QoQ)

QoQ occurs when content producers decide that the best way to attract consumers to their content is by adding a higher degree of production to it. By pushing up the production value, one can argue there is a direct correlation to moving from the right to the left in the classic long tail curve. The most obvious (and first) examples would be produced blogs. Sites like TMZ, Gawker, TechCrunch, Engadget, AskTheVC, etc.. are all what I would call lightly produced QoQ content. That means that someone (or more likely a team) with actual writing or journalistic skills puts some effort into silly little things like fact checking, correct punctuation and grammar (clearly I don’t strive to elevate myself to this level), editing, and generally well-written or thought out pieces. People go to these blogs because, on the whole, they are guaranteed a more valuable consumer experience (e.g. there are few posts about what movie Mike Arrington watched last night). But wait, you say, this is really just like a newspaper or magazine. Well, as I will explain in a bit, it’s not, because for the most part it’s all sourced by the staff bloggers themselves. Yes, there is an editorial choice about what to write, but the key is that it’s not culled from multiple sources. 

We see the same phenomenon on YouTube and MySpace with the emergence of higher quality content channels (MySpace’s Roommates excepted!) and also with a number of higher production quality independent pod/videocasts like the one from my good friend Dana Brunetti at TriggerStreet.com. In general “production” can mean higher core quality (e.g. using something more than a Motorola Razr to take video at a Tool concert) or thoughtful post-production editing like you see on TriggerStreet or another friend of mine’s Citizen Sports Network.

And to some extent the QoQ approach has worked. Feedburner quotes 768K (yes that’s 768,000) subscribers to TechCrunch. While that’s not quite the New Yorker’s 1M subscriber base, if you include what is probably a much higher secondary and tertiary readership with blogs (most people not using subscriber email or RSS feeds), we’re talking aggregate readership levels that the ABC would register firmly as a blip on their publication radar. 

Community Percolation

The second solution I mentioned is what I call community percolation. Community percolation can really be attributed in origin to Digg (and its multiple successors and automated derivatives like TechMeme and StumbleUpon). Many people call this community filtering but I think that moniker is a bit misleading. Filtering, to me, implies an intentional slicing of a data set using specific criteria. By definition, the masses have no common criteria other than  grunting “this blog good” while hitting a button. Semantics set aside, the idea here is that the wisdom of the masses shall help the cream of the blogosphere rise to the top. If you believe this will solve all of our problems let me remind you of two things: first, Jaron Lanier’s fantastic treatise on lowest common denominators, Digital Maoism and second, Bush’s second term (come on guys, that one was tee ball). While Digg does reduce the overall noise, it serves in reality as a band-pass filter not a high-pass filter (yes, I took one EE course at MIT – and all I got to show for it was one semi-intelligent sentence in a blarticle). 

Authority

The last solution is the authority approach. This has a few derivatives but generally follows the mantra “if X is an authority on Y and I am interested in Y and X recommends something about Y its worth my time to check it out”. This takes place in the form of recommendation engines (Amazon and NetFlix), the Technorati Authority system, and my friend Matt Cutler’s random indie music recommendations (most recent would be Bassnectar and Phantom Planet). In general this can work in limited circumstances (picking a horror movie to add to your NetFlix queue), the problem is that it’s hard to find out who is an authority on anything very quickly on the net. Go ahead, try and find out who is a real authority on Bangladesh in less than 2 minutes. I just tried this and it’s not so easy! 

So after a few years of user generated content, what we’re seeing is that the three solutions I mentioned do help a bit but not in the way that any of us really want. The metric for “really want” is simple – it still frigging annoying to find content you like on the net. And the more people hit the publish button the worse its getting.

The Return of the Editor 

If you consider a spectrum with the New York Times on one side and your average blogger on the other, it’s interesting to consider what the two end points really represent. Some might see that spectrum as simply the continuum from professional journalism to citizen journalism. But I’m sure most people could point out a blogger that writes better than an in-house journalist or a journalist that is more insightful and free of voice than some bloggers. So I don’t think that’s the scale. What I think the scale is really about the editorial function.

On the left we have a tightly controlled editorial function performing both a filtration and assembly mechanism pulling from a number of different sources and also introducing bias to the benefit of the readers. On the right we have an individual voice, someone who by definition has a bias, but generates all content and thus provides no filtering or assembly other than what they choose to write about. So what’s half way between Bill Keller and me? This is what I call Editor 2.0. 

Editor 2.0

If we can have citizen journalist (bloggers), citizen media (YouTube and people on the street uploading live coverage on UStream.tv) we can definitely have citizen Editors. This, in my opinion, is the next step in the future of media. 

Consider for a minute some of the trends that are pointing in this direction. First, let’s consider the editor function of filtration and assembly. Last time I checked, pretty much every single thing on the web was getting delivered by a feed or was designed to share. Flickr, blog entries, YouTube videos, del.icio.us tags, etc.. these are all input sources akin to the Reuters and AP newswire feeds that most daily editors cull from. Second, while blog entries are already published when you find them, such fragmentation in the market means they are probably relatively unread and assembling them into an aggregate format and republishing them to a broader audience probably encroaches little on their readership loyalty by the fact that they might exist elsewhere as well.

Second, consider that many people already play the editor role, if but for themselves. iGoogle, myYahoo, NetVibes, the Daily Me, etc.. they are all really just systems set up to let people become their own editors using the familiar section-based format of a paper with access to all the feeds underlying those sections. If you took this model and you turned it outwards, not inwards, this would be citizen editing in its simplest and roughest form. 

Tabula Rasa

But let’s get a bit more specific about what the next generation of media is going to look like. Let’s take the easy one – blogs. A while ago a friend of mine, Eric Herman, and I came up with an idea we called Tabula Rasa (TR). TR was in essence my first exploration into citizen editing. I was frustrated with the insane growth in blogs and more particularly the breadth of topics covered in most blogs I read (e.g. a blog purportedly about PeopleSoft sometimes has random entries about a vacation spot the blogger went to). I was so overwhelmed culling through the sheer volume of blog entries in my blog reader every day that I simply gave up and took the lazy blogosphere approach (a derivative of the more general Lazy Web). The lazy blogosphere approach relies on me staying in sparse communication with the smarter bloggers and entrepreneurs I know and relying on the fact that if anything really important comes up, someone will send me a link to it. Frankly I think this works pretty well as I just got two links to the same NYT article related to something I am working on within 5 minutes this morning. This, as a side note, was a subconscious default to the authority solution I mentioned above. 

What we really need is a new platform designed for the editorial function. This is what Eric and I envisioned TR to be. A sort of console for taking incoming feeds, finding related media, and cobbling together a front page of the best of the best. If you wanted to add your own blog entries to it, then by all means. Imagine that rather than write a blog about Tennis I was more interested in pulling together the best writing about Tennis on a daily basis. The TR platform we designed would let you do this in 15 minutes.

One of the beauties of the TR concept is that it would publish in a newspaper front page format. What I mean by this is that it disrupts the time based entry format of blogs. This is one of the biggest problems with blogs in my opinion. The best content is seldom at the top of the blog. When I scan my blarticle readership, seldom is the top entry the most read (due to inbound links from Google and emailed links to outside readers). With the front page format, citizen editors could choose to keep important articles and stories on the front page for longer, swapping out side stories that might be more current. In my own blog I’d love to keep some of my better and more timeless pieces prevalent on the blog but I am relegated to making a “best of the best” sidebar set of links. 

The other beauty of the TR concept is that different media types about a subject could be sourced from different places. Maybe there is a fantastic blog entry on Wimbledon but the best picture is from Flickr and a great video of the final match is on YouTube. TR would allow me to assemble these quickly into one seamlessly produced front page.

I know, I know, everyone will say “what about licensing issues”. Well there are a few simple half-solutions (like creative commons licensing) and blurbing (ala Google News) and such, but in general I think the benefits would outweigh the means and you could always include attribution and eventually advertising revenue share to all the content contributors. 

And the same thing is emerging in video. With live video platforms like Ustream.tv and Justin.tv becoming more popular, it won’t be far off when we can skip between multiple live video streams of events. An editor can easily insert themselves into this process and hand direct the best “camera” angles in real time just like TV stations do for sporting events and live TV programming.

Sooner or later someone will construct something like this and the citizen editor will emerge. And once the citizen editor has emerged, community editing will emerge as well. A few like minds can pretty easily keep on top of a topic or two on the net and edit it into one useful front page. 

Swinging the Pendulum

I am starting to think there is a palpable gestalt emerging around this right now. You can even see it in traditional media. Walk into any bookstore and you will note that some of the most popular titles on shelves are edited compendiums. A good example would be the “Best American Writing” series. These things are everywhere and they espouse the value of an editor. If you hunted around you could find all the articles yourself but I can tell you from personal experience it’s worth just buying the book. It’s scary to identify a media trend by pointing at something from the printed media world, but let’s not get too full of our internet selves here. 

Whether traditional media goes first or entrepreneurism comes to the rescue, it’s definitely time the pendulum starts swinging back towards the middle.

Why Enterprise Software is Not Dead

This is part 1 of a multi-part series on the virtualization revolutions no one is really talking about. If you enjoy my longer blarticles, make a cup of coffee; you won't be disappointed by this one.