Review: Out of the gate, Cuil is ambitious but flawed

After not much fanfare but plenty of anticipation, a new search engine called Cuil has launched. In spite of all the noise about the founders’ pedigree and the company’s index size, the only thing that makes a search engine tick is the relevance of its results across several dimensions. I spent an hour testing the service last night and found it has made an ambitious and interesting start out of the gate.

When evaluating a search engine, several interesting factors come into play. Does it do a good job with obscure terms? Will it satisfy the average user for popular terms such as “Las Vegas”? Is it susceptible to spammers? (No prizes for guessing: The answer is almost always a resounding yes.) Overall, I’ll try to focus this review on features that are interesting as well things that are blatantly broken in this early version.

(Disclosure: I work at Kosmix and manage the Kosmix RightHealth product. Kosmix is focused on a product that is very different from a general purpose web search engine, and isn’t intended as a direct Google — or Cuil — competitor. Also, these opinions are my own and do not represent my employer.)

First, I tried a popular query: “Las Vegas”. The point of this test is to make sure the search engine doesn’t make spectacular errors on terms for which plenty of web pages are available. So I pulled up Cuil’s Las Vegas page. Most of the results returned are reasonable, and if you use Google as the gold standard in terms of result relevancy, the first result, LasVegas.com, is on the money.

The one thing that Cuil misses sorely is any recognition of travel or local search intent. It fails to understand that a user searching for “Las Vegas” might be interested in driving or flying there. Google, on the other hand, will show you a handy map with a zip code box for driving directions.

Next, I tried a very obscure term: “Apocrine Hidrocystoma of the Eyelid”. The point of this test is to ensure that the search engine can serve results for the potentially infinite set of queries that can be thrown at it. After all, search terms similar to the one above are tried by users every day and are the source of Google’s prowess in relevance. Unfortunately, Cuil’s 120 billion page index doesn’t help here. In comparison, Microsoft’s much-maligned Live Search does okay.

Finally, we tried a term that is notoriously susceptible to spam: “Cialis”. Cialis is a treatment for erectile dysfunction and there are plenty of folks online trying to sell it to you and me on the cheap. Cuil fails to protect me from these guys. It has been speculated that Google prefers editorial content - such as that produced on Wikipedia - over commercial content, and it shows clear as day in this query. If you want to purchase Cialis using Google, you’ll probably have to rely on the ads over the search results.

Apart from the relevance issues, Cuil has an interesting take on user interface. Results are presented in a three-column grid format. The engine also tries to associate an image with the result. Sometimes, the image is the site’s logo — presumably, the presence of images is supposed to anchor the page and make it easy to read and scan. More interestingly, Cuil tries to