Twitter to Challenge Google for Search (no, really!)

Finally, some big news about Twitter which doesn’t involve the company being bought by Google Microsoft Yahoo Apple whoever this week. Instead, at a Cnet panel last week Santosh Jayaram, Vice President, Business Operations, mentioned that Twitter will begin crawling the links in tweets, and then indexing those pages.

As Techcrunch rightly points out, this isn’t a matter of Twitter trying to beat Google at their main strength, traditional search, it is instead a brand new paradigm, with the promise of human-influenced, real-time search results.

Little is known about the details but there were comments made about weighting links based on influence (a la Google’s Page Rank system), but no word on how that would be done yet.

One thing is certain though, this finally offers Twitter the chance to prove its real worth. All those big companies sniffing around for the past few weeks weren’t doing so as a result of the buzz around Twitter (despite Oprah‘s growing disinterest after an initial flurry of tweets, the number of new users continues to climb), but for the oodles of data Twitter has unprecedented access to. As with Google’s purchase of Urchin a few years ago, seeing how real people behave online is inherently valuable.

How Will It Work?

Short answer: we’ll have to wait and see. Longer answer: I’m not sure, but I’d like to see something like this. A real-time search engine,which would compliment Google rather than replace it, where real people invisibly influence the search results.

For example, a current Twitter search for ‘obama‘ has thousands upon thousands of results with dozens more each few seconds. Great if you want to know what people are saying, not so useful if you want to discover what exactly they are reacting to, as you have to wade through page after page to see all the links.

Or ‘manny ramirez‘ – you’ll see lots of comments about his drug violation, and many of them also have links. The trouble is that with URL shortening, you can see what looks like 10 different links all going to the same article.

Wouldn’t it be great to have this page split into two columns, one with comments and another with the most popular linked-to articles/websites?

Of course, it will also become subject to the scourge of the internet – spammers – but that is where the relative influence of the writer comes in.

It’s easy to forget now, but before Google search engines were filled with spam. Yes, it still appears throughout the search results now,even on Google, but nothing like it was back in 1997/8, and that is largely because of Google’s PageRank system. For a detailed mathematical explanation see here, but basically Google used links to a webpage as a way of measuring its popularity,but with the very important caveat that not all links are equal.

If I run a golf course for example, a link from PGA.com would be far more valuable than a link from your old Geocities page – PGA.com is a popular site, and in a related field. A similar thing could be done for Twitter search, a link from ESPN columnist and professional RedSox fan Bill Simmons to a Manny Ramirez article would be worth more than one from Oprah, even though she has more followers.

And of course, Twitter search wouldn’t have to be limited to websites either. As with Google’s move towards universal search, it could easily incorporate videos, mp3s, photos, and anything else that people are discussing or linking to.

Something like this could be hugely useful, if implemented correctly. Despite Google’s success, the missing piece in their search has always been a human element. In fact they incorporate the Open Directory Project listings into their search algorithm as a way of helping to inject some personal judgment.

A search engine that shows exactly what is happening around the world right now, as discussed by real people? I can’t wait!

So what do you think? Am I missing the point? Is this another potential ‘Google-killer’ that will die on the vine? Please share your thoughts or comments.

From Bad to Worse at Cuil

Cuil, who have seen their traffic tumble into free fall since their much-hyped (but severely botched) launch, have suffered another blow today, as TechCrunch reports that VP Product, Louis Monier, has quit the company:

“Louis Monier, Cuil’s VP Product, quietly resigned from the newly launched search engine last week, we’ve heard from a reliable source. “

Monier was one of the big draws for the simply stunning amount of PR that Cuil generated – he was hired  away from Google last year in a major coup for the young start up, but is even more well known as the Father of AltaVista, everyone’s favorite search engine before Google came along. His departure is thought to be related to the path Cuil should take.

With a resume which also includes stints at Xerox PARC,  Ebay and Google, Monier is regarded as one of the big names in tech and search, so the blow will be a huge one for Cuil. After all, he left Alta Vista, then the #1 search engine, after a similar disagreement over the move from straight search to becoming a portal (how 1999!)..and look what happened to AV after that.

Not so Cuil

Why Cuil is not a Google-killer.

It’s fair to say that most search engine news is outside of the mainstream. CNN doesn’t usually cover every algorithm update. So when we have 3 clients contact us within hours of a new search engine launch, we can be pretty sure that it must be generating some buzz.

That is certainly the case with Cuil.

It’s founded by a “trio of former Googlers”. The search index is more than 120 billion pages, “three times the size of Google’s index”. It promises to be “more comprehensive and more relevant (than Google)”. And so on, and so on.

It seems that everyone is so invested in finding a challenger to the virtual monopoly that Google has on search that they aren’t looking at the bigger picture. People use Google not because they have to, but because Google works. Google has been the clear leader since it launched, and everyone else is still playing catch-up. If Microsoft and Yahoo, with all their billions of dollars sloshing around can’t compete (and they can’t) then a new upstart like Cuil has no chance, however many column inches of news they grab.

120 billion pages index might make for good attention grabbing copy, but it’s virtually meaningless. People use a search engine to find things, it’s as simple as that. Cuil fails on that very simple measurement. It may have the largest index, but the results are in many cases, useless. Even in the areas where the results are relevant, they are not *more* relevant than Google’s. So why switch?

I tried some side by side comparisons:

I can’t see, as it currently stands, any reason why people would switch from Google to Cuil. Maybe I’ll be proved wrong, and I’ll happily admit it if so, but for right now, Cuil is anything but.