The (Big) Problem For Hakia, Powerset, Mahalo, and Other Google-Killers

The "highlighter" feature that search-engine Hakia announced yesterday wasn't worth a press release, but it did get me to try the company's "semantic search" service, which is actually pretty cool.  As instructed, I asked Hakia three English-language questions:

Why did the stock market crash?
Where do I get good bagels in Brooklyn?
Who invented the Internet?

As promised, I got intelligent results for all (even the last one, which was a trick question).  For example, Hakia understood that, when I asked "why," I would be interested in results with the words "reason for"--and produced some relevant ones.  If I'm ever in the mood to ask an English language question--and I remember that Hakia exists while reaching for the keys--I might use the engine again.

But therein lies the problem--indeed, the problem for Hakia, Mahalo, Powerset, and the dozens of other companies that are pursuing next-generation search.  Contrary to the premise upon which most of these companies are based, I don't agree that current search sucks.  On the contrary, I almost always find satisfactory results immediately, conveniently, and with minimal frustration.  I also don't find myself wanting to ask the Internet English language questions all that often: It's usually easier to just type keywords.  The results (and display) could always be improved, of course, and maybe I'm always missing out on fantastic sites that have just the info I'm looking for, but ignorance is bliss.

On the questions I asked, Hakia certainly delivered nice results. But I'm used to using Google and Yahoo, and Google and Yahoo usually get the job done, and I almost never wonder whether I'm getting "the best possible results."  So unless Hakia, et al, focus on tight, defensible verticals--or sell their technology to Google/Yahoo/Microsoft--I don't think their future is promising. 

Don't believe me?  Check out Hakia's modest traffic over the past year. Or just ask the guys at IAC's Ask, who, despite being widely viewed as having the "best search on the web", despite massive advertising, and despite the brilliant Barry Diller, haven't budged off of 2% market share.

Comments

[pwb]

I cannot stand natural language search myself but I've tried to keep an open mind that perhaps others like it.

[You Mon Tsang]

Many ships have been wrecked by the siren call of natural language processing. It sounds so good, feels so right, but never seems to be better than the simpler alternatives.

[Christian]

Henry, couldn't agree more with your assessment. Over the last 2 weeks I was pitched 2 new search engines as well - i didn't even try them out because to me search in the google way is simply not broken. I'm used to the way it works, it produces great results and it's already in my browser. So unless Google's result quality deteriorates badly - no need for yet another new search engine.

[ProfitTheme]

Just what we need, another search engine LOL.

[Still Inside]

Natural language processing has always failed (recall that Ask got its start with NLP and subsequently the Jeeves guy went the way of rodney allen rippy).

The problem is pretty simple: most people can figure out that typing "can you tell me what the weather in like in Las Vegas right now?" and "vegas weather" are the same thing to a search engine, but the latter is a lot less typing.

Wiki* does not seem to have anything we haven't seen many times already and neither do any of the others. It's certainly possible somebody can invent a new paradigm for Search, but it hasn't been done yet. Google's genius was to actually focus on Search when everybody else gave it up for dead. Trying that trick again is like trying to hijack an airplane with a box cutter.

SI

[Still Inside]

Ok, I normally don't double-post, but I had to try out Hakia just for fun. Fun indeed. You can always crush these little engines by looking for something slightly obscure. So I tried the following:

"what stocks went up today?"

Hilarious.

(The question is, do they read this blog and will they "fix" that query and will they still think their idea is going to scale to billions of queries)?

SI

[Samir Shah]

I, for one, find Hakia refreshing.

Henry's tone and tone of some others in the comments sounds too much like fear and/or mockery of new technology. The kind that used to say "Command line works for me fabulously, I do not need GUI". One needs Natural Language Query as much as you needs GUI (away from command line) and some of the questions that Henry asked himself prove the point.

[NitinK]

Great post! Although it's surprising to find almost no dissent among the comments to a blog post that takes such a clear position (other than Samir Shah's comment).

I have to disagree with you, at several levels:
(Forgive me, I'm using my own older posts to provide more detail for my points)

1. I think it's easy to confuse the approaches used by Hakia and Powerset, since both are clubbed into the area of "Semantic processing". But there's a distinct difference: although Hakia, as you describe, uses semantic parsing to analyze the query, Powerset is applying linguistic algorithms to make sense of the underlying content in their index. This could create a huge bump in the relevance of search results, compared to Google, which uses the venerable PageRank and proximity of text phrases to rank results.
More details on Powerset's approach here: Powerset is Not a Google-killer!

2. You say that "current search does not suck". Oh, yes, it does! Not in ways that are obvious, until you see the possibilities, just as the designers of the ever-popular DC-3 airplane (which revolutionized air travel in its time) could not have imagined how jet aircraft could make propeller-driven airplanes obsolete.
Here's a list of Search Innovations outside Google, on the Read/WriteWeb.

3. Finally, Google has also started down the path of "special-casing" search results (an almost Mahalo-like approach, although it's more complex for Google since tweaking the search algorithm can have unintended consequences on other queries) - an interesting development, albeit troubling:
The Hidden Dangers of Special Casing Google Search Results

- Nitin