Dot.life - where technology meets life, every Monday
By Mark Ward
Technology Correspondent, BBC News website
There was a time when every search site aspired to be Google.
Lots of small firms are taking on Google in specialised searches
Success seemed to demand a vast, well ordered index that people could query quickly.
It made finding information so easy that searching has become central to the way we use the web.
"Three-quarters of people find their way around the web through search engines," says Warren Cowan managing director of search engine consultants Greenlight. "It's the second biggest use after e-mail."
"Users are getting more advanced," he says. "Three or more queries are more common now."
This knowledge means that users are also learning the limits of searching via a site such as Google, Yahoo! or MSN, says Mr Cowan. Especially as these sites expand into new areas of information.
"Search engines work on simple text matching - they tend to scan what is on the tin."
To get inside the tin and take a look at the ingredients, a lot of users are turning to other search sites and even stand-alone programs that do a better job than the giant indexing machines of Google and its rivals in specialised areas.
For instance, if you were trying to find a person in the UK you might turn to the phone book or one of its online equivalents. But chances are you would not find who you were looking for.
Only one-third of people are in the residential phone book, it does not list full names, or mobile numbers, and you need to have an idea of where someone lives to track them down.
Spot-on searching takes more than a basic indexing job
"It's very punishing of bad searches," says Keith Marsden, managing director of 192.com, who thinks it takes much more than keywords to do a good job of looking for people or businesses.
"The fundamental problem with Google is that it ranks websites by how big they are," says Mr Marsden, "which is fine if you are in the internet business."
"But if I'm a plumber why should I be ranked by the size of my website, if I have one? I should be ranked by my proximity to where you are, whether I'm a Corgi registered plumber or not."
Mr Marsden says his site does a better job by collating datasets from the web, the telephone book, businesses directories and the electoral roll to build its database.
He believes 192.com is getting more traffic because the "disastrous" reform of directory enquiries in the UK has put people off calling that service.
The fact that search sites are starting to get into indexing everything you have on your desktop computer is also revealing some of their limitations.
While some, such as Blinkx, can look inside documents and help you find what you are looking for, some of the others rely on you knowing what you want before you can search for it.
Map data is starting to be used in lots of searches too
"You can use a search engine to find these documents but you still have the problem of understanding what's in the documents," says Mark Thompson, chief executive of Corpora.
Corpora's Jump software uses a variety of techniques to help users find what they want and be sure that, when they find it, they can get to the part of the document that's relevant.
Jump automatically indexes documents and produces a long list of all the terms used in each one. It can also sort out who is being talked about even if only "he's" and "she's" are being used.
"People don't use their computer to read or to help them understand information because a PC is generally pretty rubbish at it," says Mr Thompson.
"This is about getting the machinery to replace cognitive process."
Lost and found
Maps and location-centred services are another specialist area that is starting to feature more in searches - increasingly on mobile phones and computers.
The UK's Multimap unites 20 different suppliers of topographic data to build its map database.
Sean Phelan, founder and managing director of Multimap, says real interest in using map data is starting to build - despite the fact that almost 10 years ago he demonstrated a way to put map information on a mobile phone.
"Fundamental changes have happened around the world in the last 10 years because of things like the internet, search engines, mobile phones and SMS," he says.
That familiarity with using large datasets is starting to drive proper use of these types of information, he thinks. Many services are starting to add the location data into other services - such as navigation in cars or to find wi-fi hotspots.
"We're on the climbing part of the 'S' curve but there's a vast amount more to do and distance to go."