|
|
Front Page | |
| |
UK | |
| |
World | |
| |
Business | |
| |
Sci/Tech | |
| |
Sport | |
| |
Despatches | |
| |
World News in Audio 
| |
| |
On Air | |
| |
| |
| |
Talking Point | |
| |
Feedback | |
| |
Low Graphics | |
| |
Help | |
| |
Site Map | |
| |
Friday, April 3, 1998 Published at 16:02 GMT 17:02 UK
Sci/Tech
World Wide Web is 320 million and growing
![image: [ Where do you want to go today? Search engines may not have the answer. ]](/olmedia/70000/images/_73397_search_engine300.gif)
Where do you want to go today? Search engines may not have the answer.
Computer Science researchers have confirmed what most Internet users have guessed already: the World Wide Web is bigger than anybody imagines and search engines are only looking at a fraction of pages.
Friday's edition of Science Magazine features a paper by Steve Lawrence and Lee Giles of the NEC Research Institute in Princeton, New Jersey. Their comparative study of the performance of search engines has led them to the conclusion that the Web is now at least 320 million pages wide.
This is a significantly higher estimate (and the researchers believe it could still be a serious underestimate) than previously made:
Their report concludes that search engines cover from 3 per cent (Lycos) to 34 per cent (Hotbot) of the indexable Web.
Combining the results of six engines covered about 3.5 times as much of the Web as one, so using multiple engine sites, such as Metacrawler, is recommended.
The rankings for the six studied went Hotbot 34 per cent of indexable pages, AltaVista 28 per cent, Northern Light 20 per cent, Excite 14 per cent, Infoseek 10 per cent, Lycos 3 per cent. Yahoo was not included because of its more general directory structure.
The report says search engines may be limited by network bandwidth, disk storage, computational power or a combination of all three, despite their claims that they can continue to index close to the entire Web as it grows.
Refine your search
"This study is a good thing because people are going to become more aware of the problem, but it's not a new thing," said Danny Sullivan of Search Engine Watch, speaking to News online from a search engine conference in America.
"Of the search engines, Hotbot says its system is scalable and it can index 300 million pages if it wanted to."
"But the average user is not complaining about not getting enough information, they're concerned about not finding exactly what they want."
"That's why you're going to see more specialised search engines with people wanting to crawl only a few sites relevant to a particular topic, home DIY in Britain for example. Some of the big sites are already putting in filters to do this and then there are specialist ones such as www.devsearch.com which just finds web developments for you."
|
|
|
| Internet Links Science magazine NEC Research Institute Metacrawler Search Engine Watch The BBC is not responsible for the content of external internet sites. |
||
| ||
| ||
|
||