Netscape Directory Making a Splash

Baldwin Louie — Tue, 14 Dec 1999 14:00:00 -0800

Netscape's volunteer-based Open Directory Project, once ignored by bigger players as a do-it-yourself Web directory, is drawing attention and criticism as it has become the Web's fastest-growing site list.

The directory, created largely by volunteer Web surfers and free to sites that want to use it, has been adopted by Netscape owner America Online (dossier), Altavista, ATT (T) WorldNet, Dogpile, Go2Net, InfoSpace, Lycos (LCOS), MetaCrawler and a host of about 90 smaller sites, including the Autism Society of Alabama and Bangkok.com. It's become popular enough to spawn a copycat, and rivals are beginning to attack both the editorial integrity and the reliability of the project.

In the past year and a half, over 1.2 million sites in more than 190,000 categories have been compiled. More than 100,000 new sites are added per month - "about the growth rate of the actual Net itself," says Chris Tolles, senior marketing manager for the Open Directory.

Its adoption by Lycos last April "really suddenly legitimized the Open Directory Project," says Danny Sullivan of Search Engine Watch.com. Now, people have the same reverence for it that they had for Yahoo (YHOO) when it was the only game in town, he adds.

Almost 20,000 people have signed up to surf the Web, write abstracts of sites and place them where they belong in the directory. Netscape has about 15 paid workers who also contribute to the site, but not full-time. Rivals and others claim that using unpaid - and therefore unaccountable - editors can result in shoddy and biased work.

"I'm not sure bigger is better," Yahoo producer Andy Gems said during a panel on directories at the Search Engine Strategies '99 conference in San Francisco in November. "More sites are more issues, more dead links." In a phone interview later, Deanna Sanford, Microsoft Network lead product manager, noted that she had discovered Open Directory editors promoting their own sites.

Not just rivals gripe. Some users have complained as well. Tricia Francis, a quotation analyst for ABF Freight System Inc. of Fort Smith, Ark., told a panel of directory companies at the conference that ABF was the victim of a competitor's foul play in the Open Directory. A worker at a competing company became an editor and pigeonholed her firm in a city category rather than placing it in a more broad national category, she said. "A complaint was submitted by another one of our competitors and the editor is gone, but we're still not where we should be," she groused.

Open Directory officials wave off such objections, saying there is plenty of oversight to catch any dubious editing. "We have a feedback form. If anyone complains about bias or problems in the directory we get to it right away," said Tolles. Editors must apply and prove themselves capable in less critical categories when accepted, he says. Their work is closely reviewed and every edit in the directory is logged for review, with one or two per week being removed.

The Open Directory was created in response to criticism that Yahoo, the granddaddy of web directories, which began building its directory in February 1994, was processing only about 10 percent of its submissions.

Rich Skrenta, now the director of engineering for Open Directory, and four others formed GnuHoo in June 1998. The company changed its name to NewHoo after complaints of confusion from GNU, an open-source software movement. NewHoo gave up the name but kept the philosophy, basing its development and distribution on the open source model.

Then Netscape acquired NewHoo in November 1998. "We didn't know what we were going to do with it, but within one month we were approached by three top portals," said Tolles.

The Open Directory model has even spawned a copycat. After finding that its 20 to 30 staffers couldn't index the web fast enough, Disney's Go.com moved to the volunteer model in September and may even consider distributing its directory for free like the Open Directory. Go.com also goes a step further and infuses some opinion in its directory by rating sites with one to three stars.

Yahoo is paying attention, although Yahoo executives say the two directories don't really compete head-to-head. "It's a little bit of a flawed comparison because they seem to be embarking on a different task. One of their biggest benefits is being able to scale to the growing volume online. They cover more ground," said Srinija Srinivasan, Yahoo vice president and editor-in-chief. Yahoo isn't going after quantity, but after the most relevant sites, she said.

LookSmart (LOOK) and Snap.com are also building directories. But LookSmart has the most to lose. It's already lost Lycos to the Open Directory Project. AltaVista has added Open Directory to its search site, downplaying its earlier-established connection with LookSmart.

But LookSmart maintains its trained staff and editors with graduate degrees in library science can provide high quality, which is more important than broad coverage.

"My gut feeling is that free is attractive, but people will see what more we bring to the table when it comes down to it," said Kate Wingerson, editor-in-chief of LookSmart, which has about 220 editors and 1.3 million entries in its directory. "We can offer that stamp of quality, that it's been through our standards" process.

AltaVista heaps higher praise on Open Directory. "The Open Directory advantage is that it is more scalable and secondly, you can take advantage of the passionate people out there who really know a tremendous amount about their subject area," said Tracy Roberts, director of marketing for AltaVista search.

The other major attraction to search sites is the fact that Open Directory is free. In order to use it sites must simply attribute Open Directory, link back to the site and offer information on how people can become editors.

Tolles and others involved in the Open Directory talk loftily about the noble aspect to the project, but Netscape isn't doing it entirely out of altruism. The company benefits from the exposure its open distribution brings and the notoriety of being the Web's fastest-growing directory.

"Look at how much it would cost to get ads on AltaVista, HotBot, Lycos, AT&T. It would be more than anyone's ever spent on a web campaign," says Tolles. "We just think it's cool that they're all competing on top of our data."

Meanwhile, the Open Directory's use of volunteers may raise some questions. In what could be a precedent-setting case, AOL's use of volunteers for its Community Guides section has prompted a class action lawsuit and an investigation by the Department of Labor into whether the situation violates the Fair Labor Standards Act. Although Netscape doesn't make a profit off its directory and the main Open Directory Project site is commercial-free, Netscape does sell ads on the search result pages at Netcenter.

Netscape executives get testy when asked if the Open Directory's use of contributors might put their work under similar scrutiny. "Netscape's Open Directory Project has never been the subject of a Department of Labor investigation," spokesman Nathan Tyler said in a carefully crafted response.

However, a labor attorney points out that for-profit organizations that don't pay people for their work are still getting free labor. As a for-profit company, Netscape wouldn't be doing anything just for the sake of being saintly and is ultimately deriving some financial benefit from the Open Directory, said Bill Sokol of the Van Bourg law firm in Oakland, Calif.

Open Directory participants see no end in sight for growth. Their goal is to create the most comprehensive Web directory.

"The most significant thing is it's really set off a revolution in terms of search engines," says Sullivan of Search Engine Watch. "In 1996, of the top six search engines, only Yahoo was human-powered. Today ... four out of six are powered by humans."

The Industry Standard - Netscape Directory Making a Splash - Comments

Netscape Directory Making a Splash