Web search in a post-Twitter world

By Tim Conneally | Published October 23, 2009, 7:58 PM

We all contribute to the news cycle when we post timely content online, even if it's 140 characters or fewer, and this week we learned that our little bits of information have substantive value when search giants Google and Microsoft announced that they will index our tweets and status updates.

But the sheer volume of content that we produce could be a problem, and Twitter users who find day-to-day value in the service may scoff at the idea that the "information firehose" of live content can be tapped and made searchable. It doesn't take extensive use to see that most tweets are in fact worthless, and that thousands of bots simply use Twitter as a tool for free promotion, by tagging links to a particular Web site with trending topics.

But this seemingly uncontrollable flow of information is shaping the way news gets to us, and it presents an important evolutionary hurdle for the search industry: We know there is extremely valuable information there, but how do we make it work for us?

Gerry Campbell has been dealing in the search space since the days of CompuServe, and was one of the first to work on monetizing Web search at AltaVista nearly 11 years ago. Campbell is an angel investor in nearly a dozen social search companies, including Summize, which came to power Twitter's own on-board search. He currently heads Collecta, a real-time search engine which shows how far we've come in terms of live news: It's a continuous query search engine that streams news articles and relevant tweets (which are filtered for content rather than indexed) as they happen until the user decides to hit "pause."

Campbell today shared some perspective on the trends in content creation and consumption. "There are two trends that are fascinating to me, the bright lights of all the trends: The first is the current velocity of content creation. Google has a trillion documents indexed, and about half a billion new indexable pieces of content are being published every single day. But the nature of what's being published has changed dramatically since Google began about ten years ago. Today it's short-lived, quickly published documents. They're high velocity. Old documents were meant to have more weight to them so they could be read longer. Information now is valuable for less time, and an important space is in finding a better way to surface this."

Startup search services like Worio attempt to do just this, and provide an equally valid mix of results from both traditional sources and "real-time" social sites. Worio combines keyword- and context-based search with "Web discovery" algorithms like the ones that suggest content in Pandora and Netflix. Topics which are spiking in interest on the social sites are more likely to be included in search results.

"You want current news, but you want to carefully look at what is interesting rather than just what is popular. iPhone, for example is historically more popular than Palm Pre. However, on the date such as the Pre's launch, search results for 'smartphone' would have been more skewed toward the Pre because that topic had more traction at the time. Some search results are always more popular than others, but social media activity helps us determine how interesting a topic is," Worio CEO Ali Davar said today.

The second important trend that Gerry Campbell pointed out today is the Web's changed role in our daily life.

"The Internet isn't a destination any more, we don't make time in our schedules to get on the Internet," Campbell said. "The emerging model goes beyond even the 'always on' concept. I've got my phone on, my notebook open at every meeting, with information constantly flowing to me at all times. I don't necessarily want a summary explaining what happened...I want to be involved as the story evolves. All of those little bits of information have a daily value to us which gets less valuable over time. This is the evolution of news.

"The topics have a shelf life based upon the format in which they're presented. A study recently showed the diffusion of a news story: it starts in Twitter and on social sites, then as it moves into blogs, it gets colored with interpretation and opinion, and the value goes up. Finally, mainstream media gets it and value goes up even further with trustworthy analysis."

With the increased flow of real-time data and increased daily use of mobile devices, Web search in a post-Twitter world is not going to be a destination model, but rather a model of integration.

Comments

View comments by with a score of at least

http://www.promdressonsales.com/
http://www.weddingdressart.com/
http://www.dresses4dancing.com/
http://www.dressupinyou.com/
http://www.hidresses.com/
http://www.digdress.com/
http://www.promdress2go.com/
http://www.promdress2u.com/
No- it's around for a while. But since it's used more by consumers than business users, I think opening it up will allow for some interesting consumer level products. In the enterprise, it's all about eliminating .pst files because of all of the technical and legal hurdles they present.

Score: 0

|

Meh... when I think of Twitter, this comes to mind:

http://www.youtube.com/watch?v=ALbH63Ali9U

Score: 0

|

"post-Twitter" world? Did Twitter go away? When people refer to a post-World War II world, is it continuing?

Score: 0

|

"Did Twitter go away? "

I wish.

Score: 0

|

If you want to be super-literal, "post" is Latin for "behind/after/following/later." and "postumous" (which we adopted as "posthumous") is the superlative form meaning "last/the end." Think of it as sort of like the difference between "penultimate" and "ultimate."

Thank God for Latin class.

It means after an event has occurred, not necessarily ended. An end is strictly implied in something like "postwar," not outright said. Think about words like "postmortem," that doesn't mean "after death HAS ENDED," it just means "after death." Post-Modernism and Post-Punk don't mean that Modernism and Punk have gone away or ended, In these cases it means using their existence as a point of origin. When I say "Post-Twitter," I'm saying "following the existence of Twitter."

Score: 2

|

I wish too

Score: 0

|

PDC 2009: What have we learned this week?

There was the freebie that no one will forget, the heebie-jeebies courtesy of Scott Guthrie, and a teensy bit clearer picture of how this cloud thingie should work.

Live report: Will Google Chrome OS change Linux?

The mysteries of just what Chrome OS is, and how much of an operating system it truly is, may be resolved today.

PDC 2009: Microsoft cares about Web browser performance

The effort to give users of the world's dominant Web browser the impression of quality, is a personal one for the man who leads that battle.

Nokia re-affirms its commitment to Symbian, sort of

Maemo won't necessarily be replacing Symbian in the Nokia N-Series, but that's definitely a place where it will be found.

E-book readers will be in short supply this holiday season

E-readers are hot this year, and a lot of compelling new products have been released, but are there enough electrophoretic displays to go around?

Sony looks to finally open a single storefront for downloads

Sony has had many different download portals for movies, music, e-books, and games, and now it's looking to make a single shop for all of it.

Tuning out the tablet: Time to give the endless speculation a rest

Wide Angle Zoom: Wishing and hoping and thinking and praying....won't put an iTablet on the market.

Five improvements for IT managers in 2010

If businesses are to improve their efficiency for next year, they need to stop and reassess the basic tenets of their job.

AOL's spinoff from Time Warner to shed 2,500 jobs

As AOL moves toward become an independent company again, it will cut nearly a third of its workforce.

Gartner: SMS-based money transfer will be bigger than mobile browsing, search

Gartner issues its predictions for the 10 things our phones will be doing in 2012.

Don't forget to upgrade to Firefox 3.6 beta 3 today

Mozilla has released the latest beta its Firefox 3.6 browser software, just over one week after beta 2.