Basically, Shirky proposes that consequences we normally attribute to an explosion of available information thanks to the internet is actually attributable to a failure of the filters in place to deal with an already abundant amount of information. Since the invention of the printing press, he says, we have lived amidst “information overload”, such that we can no longer look at the phenomenon as a problem, but as a fact. An appropriate response, then, is to build better filters.

Notice that the solution is to build, not to fix. An overload of information has been our oxygen for centuries, but the type and amount of data we deal with now is vastly different than whatever has come before. Shirky ends by separating the types of filters needed into two categories: programming and social. The latter category is pretty nebulous, but “programming” is much more concrete, and already in use today. Digg, Netflix, Google, and every other website run on extrapolating from its users’ actions is utilizing so-called collaborative filters, and these are a pivotal part of the internet’s future.

Collaborative filtering works by ranking content according to prior users’ actions (e.g., Google looks at links, the paragraphs surrounding search terms, other mysterious data), then analyzing your own actions to serve up relevant content. With sites like Digg and Reddit, this involves users upvoting or downvoting submitted sites, and then you seeing the best sites in descending order (with maybe some specializing depending on if you’re at a subpage). With sites like Netflix and Amazon, your own consumption is compared to other consumers in order to serve up recommendations.

As we approach the singularity and the web becomes more ubiquitous, collaborative filtering will become increasingly sophisticated (and accurate). As Jay Adelson mentions, Google’s search rankings have become more powerful simply because of the diversity of its users has increased. Collaborative filtering thrives on multifarious data, and this will come quickly with a larger number of netizens, and more slowly through the growing number of connections between web services. Adelson points to the economic advantages of shared data (read: better advertising targeting), but this is of course of huge value to users and developers as well.

Low-level web curation, from Metafilter to Undress Me Robot, will always have its place online, but the future is definitely in these automated processes which leverage what each user is already doing to provide a highly personalized and more effective experience. However, obviously, collaborative filtering is simply an update of the sort of filter that has been around since the 1500s, moving from the editorial eye of a single person to the gaze of millions. The next big jump in information overload might break down even these strong filters, taking a paradigm shift to get back on top. And who knows where that will take us?

Haughey’s sentiments are summed up in his deriding comments as “just another content management feature available to you on the web.” Though Huaghey hints at the technology being the problem, he actually lays blame on the audience, saying:

I’m starting to think there’s this “new generation” that has grown up online only knowing blogs as having snarky comment areas and never realizing it used to be a personal, intimate space where you’d never say anything in a comment that you wouldn’t say to a friend’s face.

A good scratch at the surface, but the mystery runs deeper, and it takes a few dozen comments to take the argument to further, more insightful ends. The audience’s attitude is important, but that attitude is shaped by the available technology and the context in which that technology is used.

As Haughey points out, technology has enabled comments to be added to a website with no thought at all. This has resulted in almost every blog to have a comment system, and a majority of comments to be of “poor quality”. A lot of blogs are of “poor quality” as well, so it is not surprising that comments are no different. Lowering the barrier of entry means letting in both the intelligentsia and the idiots.

But the technology deal cuts both ways. It is easier than ever to comment, and it is easier than ever to blog. As Brad says, “the big problem with comments is most people who want to engage thoughtfully with blogs and bloggers already have their own outlets (blogs/twitter/facebook, etc).” Your mileage may vary with this argument, but you could easily find examples in big name bloggers like Jason Kottke and John Gruber. Really any “cabinet of curiosities” blog with an abundance of links and commentary is an example of someone who took their good comments and turned them into an independent blog instead.

Thirdly, technology’s deficits also play a role. With the proliferation of blogs, and the move away from relatively few structured communities (newsgroups, forums), user identity is becoming more and more fractured. Even when entering the same personal information for each comment on ten different websites, there is no system setup to link one comment to another. On ten different websites, I am ten different Spencer Sugarmans. OpenID is a start, but a million more things need to happen to get from where things are (multiple usernames) to where things need to be (decentralized tracking of user activity). (One of those “million more things” being reconciling the need for community with the need for privacy.) Some grand solution would turn the internet into a single, giant conversation.

Lastly, technology aside, why do we even care about comments? Originally, grandiosely, comments acted as “a conversation between the reader and the author of the original post.” comments falling into this category are generally labeled as “good quality”. “Poor quality” comments, on the other hand, are usually short, off the cuff, and add little in the way of meaningful dialogue. Disregarding spam and flames, “poor” comments are pretty much the linguistic equivalent of an upvote (or downvote) on Reddit or Digg. They scream “I visited!”

But this use of the commenting system is hardly unreasonable. Spurred on by sites like Reddit and Digg, comments have become a system for feedback, showing website owners their users in ways Google Analytics can’t. And often dialogue is not disregarded in total, but shifted from a conversation between the reader and the author to a conversation betwen the reader and another reader. “A place to argue amongst themselves, quite independently of the author and his ad impressions,” as Nick says.

Of course, “good quality” comments still exist (look at this recent Hacker News thread), and are even flourishing. But so are the new kind of comments, at an exponential rate. The crap found below the videos on YouTube is an unavoidable result of changing technology, but this is only the public face of commenting. Thousands of blogs incite deep, meaningful conversations; it is just a matter of slogging through the mud to find the gold.

Pronounced “cool”, the Cuil search engine has an index spanning 121 billion web pages, and operates by comparing your keyword to the rest of a page’s contents instead of relying on “superficial popularity metrics”. This mechanism calls to mind the semantic web, but Cuil’s About page is too ambiguous to tell for sure how Cuil works.

What is sure is that Cuil’s team sees their engine as the future of search. Anna Patterson, who moved to Google after building the search engine used by Internet Archive, has low hopes for Google. From CNN Money:

Patterson enjoyed her time at Google, but became disenchanted with the company’s approach to search. “Google has looked pretty much the same for 10 years now,” she said, “and I can guarantee it will look the same a year from now.”

With these bold words and the talent behind them, as well as Cuil’s claim to fame of indexing a hypothesized three times as many web pages as Google, it is hard dismiss this Google contender along with all the others. But even a cursory glance at the engine’s functionality shows a less-than-polished product without much bite.

A lot of people have been finding massive differences in the number of search results between Cuil versus Google. An interesting example is found in “banana” and its French translation “banane”. “Banana” returns 47 million results in Cuil versus 97 million in Google, while “banane” closes the lead a little bit by returning 5 million results versus 11 million (though still roughly the same percentage-wise). A clear numbers win for Google, but dig a little deeper and you see Cuil vastly improving within only 24 hours since its launch. PontifexPrimus originally found only 15 million results for “banana”, and none at all for “banane” (unless you, curiously, turned Safe Search off).

Of course, the raw number of search results is much less important than the number of useful results. Unfortunately, Cuil falters here as well. Searching for “Spencer Sugarman” on Google returns maybe every individual website I am involved with, all on the front page. Cuil, on the other hand, returns 10 links to Undress Me Robot, and 1 link to an article about Burt Sugarman. The “banana” search mentioned above is much better, providing many relevant and unique results. But what Cuil fails to do on any search I have tried is surpass the quality of Google’s results. Instead, all they manage so far is to land somewhere along the spectrum of Much Worse to As Good.

This is likely to change. It is difficult to tell whether Cuil is incorporating changes as its engine is stress-tested, or if the learning is built-in, but either way the engine seems to be improving. If Cuil expands its hardware to ward against the downtime problems they were experiencing yesterday, and continues improving its search results, the search engine could soon stand strong against Google. But as CNN points out, Google’s immense power as a brand means winning will take a lot more than just marginally better search results or a slicker design.

Hit refresh for another image. If you’re not saying ‘Om Nom Nom Nom’ out loud at the same time as looking at these pictures then you’re doing it wrong.

© 2008 The Artists of Om Nom Nom Nom

With reverence to the great activists of the 1960s, Belgian “politician” Tania Derveaux wants to steal the cherry of any net-neutrality supporter. “In history, man has always waged war for freedom,” she says. “Now it’s time to obtain our freedom with love.”

So….if you are an able-bodied defender of internet freedom looking for love in the least likely of places, click “get laid” to arrange a private intimate meeting.

The terms of service are pretty lenient, with a small emphasis on safety and hardly a stringent policy on corroborating the applicant’s virginity or opinion on net neutrality. (Also, it is pretty optimistic: “due to time limitations each performance can last no longer than 30 minutes.” Derveaux has such hope for her virgin audience.)

Of course, the whole thing is a farce. Some of you should remember her from last year’s promise to hand out 400,000 blowjobs. The same promise that ended with a virtual blowjob by her “Asian assistant” via YouTube. I imagine this charade will end similarly.

Surprisingly, the Don’t Stay A Virgin website is entirely lacking in any information on net neutrality. After going through the trouble of flashing some sex in your face to grab your attention, one would hope that attention would be put to good use. Alas not.

So as it stands, this is a publicity gimmick (which I apologize for perpetuating) with no substance behind it. A few links or articles would go a long way in pulling Derveaux out of the realm of internet skank grasping for page views.