Web tool may banish broken links

By Jo Twist
BBC News Online science and technology staff

The tool could be developed to banish all broken links

Students have developed a tool which could mean broken weblinks are history.

Peridot, developed by UK intern students at IBM, scans company weblinks and replaces outdated information with other relevant documents and links.

It works by automatically mapping and storing key features of webpages, so it can detect significant content changes.

The students said Peridot could protect companies by spotting links to sites that have been removed, or which point to wholly unsuitable content.

"Peridot could lead to a world where there are no more broken links," James Bell, computer science student at the University of Warwick, told BBC News Online.

"I personally hope it would be developed to a level where it would do that."

Peridot alerts web administrators not only which links are broken, but also where the original page has gone.

It can also tell which links have changed and the degree of change in content.

Donkey work

Currently, said the students, website reviews require manual maintenance.

Administration staff have to go through all the links on an intranet - internal website - or external website to check the information being linked to is still relevant.

Externally, you have cases of companies that link to disreputable content which could seriously damage their reputation James Bell, Peridot

The web-based Peridot tool registers webpages - either on an intranet or on websites - and follows link targets to compare them to webpages it has previously seen.

"The way we identify the content is through a process called fingerprinting which allows us to take representation of document like a fingerprint," explained Andrew Flegg, IBM software developer and technical mentor for the Peridot team.

On intranet sites, which are relied on by many companies to not only keep their employees but also provide links to important company documents, out-of-date links can result in loss of productivity for workers, said Mr Bell.

"Internally, you have users who are trying to do their jobs and the intranet is there to facilitate that. If they can't get the information they cannot do their job properly.

"Externally, you have cases of companies that link to disreputable content which could seriously damage their reputation."

This happens when a website which contained previously vetted information changes ownership or is updated content with inappropriate material.

It is a particularly sensitive problem if the firm is pitching itself as reliable and trustworthy company, added Mr Bell.

Large scale

The team, which is working on IBM's Extreme Blue internship programme at its UK labs in Hursley, has already filed two patents for the project.

Similar tools do already exist, but they tend to simply detect which links have been broken.

Peridot's innovation is that is detects more substantial changes and has adjustable levels of autonomy, according to the researchers.

It generates a report on a website and it can automatically e-mail admin staff with appropriate information.

Staff can either review the changes and accept or ignore, or they can rely on its apparent accuracy and choose to be alerted after the tool automatically updates a link.

The team has prototyped the tool so it runs reliably over 100,000 pages, said Mr Bell.

So far, it has never replaced a link with something else that is equally unsuitable, but work still needs to be done to optimise its performance, said the student team.

The Peridot team were presenting their research to top executives and engineers this week in Amsterdam, along with other intern research project team across five countries.

Peridot is a green gemstone which, legend has it, was used in ancient cultures to help people find what they had lost.

It was also believed to possess mystical powers including helping dreams become a reality, attracting wealth, calming anger, and dispelling negative emotions.