Netscape's volunteer-based Open Directory Project, once ignored by bigger players as a do-it-yourself Web directory, is drawing attention and criticism as it has become the Web's fastest-growing site list.
The directory, created largely by volunteer Web surfers and free to sites that want to use it, has been adopted by Netscape owner America Online (dossier), Altavista, ATT (T) WorldNet, Dogpile, Go2Net, InfoSpace, Lycos (LCOS), MetaCrawler and a host of about 90 smaller sites, including the Autism Society of Alabama and Bangkok.com. It's become popular enough to spawn a copycat, and rivals are beginning to attack both the editorial integrity and the reliability of the project.
In the past year and a half, over 1.2 million sites in more than 190,000 categories have been compiled. More than 100,000 new sites are added per month - "about the growth rate of the actual Net itself," says Chris Tolles, senior marketing manager for the Open Directory.
Its adoption by Lycos last April "really suddenly legitimized the Open Directory Project," says Danny Sullivan of Search Engine Watch.com. Now, people have the same reverence for it that they had for Yahoo (YHOO) when it was the only game in town, he adds.
Almost 20,000 people have signed up to surf the Web, write abstracts of sites and place them where they belong in the directory. Netscape has about 15 paid workers who also contribute to the site, but not full-time. Rivals and others claim that using unpaid - and therefore unaccountable - editors can result in shoddy and biased work.
"I'm not sure bigger is better," Yahoo producer Andy Gems said during a panel on directories at the Search Engine Strategies '99 conference in San Francisco in November. "More sites are more issues, more dead links." In a phone interview later, Deanna Sanford, Microsoft Network lead product manager, noted that she had discovered Open Directory editors promoting their own sites.
Not just rivals gripe. Some users have complained as well. Tricia Francis, a quotation analyst for ABF Freight System Inc. of Fort Smith, Ark., told a panel of directory companies at the conference that ABF was the victim of a competitor's foul play in the Open Directory. A worker at a competing company became an editor and pigeonholed her firm in a city category rather than placing it in a more broad national category, she said. "A complaint was submitted by another one of our competitors and the editor is gone, but we're still not where we should be," she groused.
Open Directory officials wave off such objections, saying there is plenty of oversight to catch any dubious editing. "We have a feedback form. If anyone complains about bias or problems in the directory we get to it right away," said Tolles. Editors must apply and prove themselves capable in less critical categories when accepted, he says. Their work is closely reviewed and every edit in the directory is logged for review, with one or two per week being removed.
The Open Directory was created in response to criticism that Yahoo, the granddaddy of web directories, which began building its directory in February 1994, was processing only about 10 percent of its submissions.
Rich Skrenta, now the director of engineering for Open Directory, and four others formed GnuHoo in June 1998. The company changed its name to NewHoo after complaints of confusion from GNU, an open-source software movement. NewHoo gave up the name but kept the philosophy, basing its development and distribution on the open source model.
Then Netscape acquired NewHoo in November 1998. "We didn't know what we were going to do with it, but within one month we were approached by three top portals," said Tolles.
The Open Directory model has even spawned a copycat. After finding that its 20 to 30 staffers couldn't index the web fast enough, Disney's Go.com moved to the volunteer model in September and may even







Hosted by Tom Sullivan, stay abreast of the latest IDG content covering IT news, product reviews, best practices, and white papers.