Computer scribe hits the web

Newsblaster aggregates thousands of news stories

By Jane Wakefield BBC News Online technology staff

Online journalists could find their jobs under threat as a virtual reporter has been created that trawls the web for all the best stories.

The Newsblaster is a piece of software designed to edit, summarise and rewrite the huge amount of news currently on offer in cyberspace.

It is the brainchild of computer scientists at Columbia University in the US.

The project has been funded by grants from the National Science Foundation and the Defence Agency.

Extracts sentences

The software was originally intended to make it easier for intelligence gatherers to find information on the web.

But it could also be used to aggregate the thousands of news stories on the internet.

At this point we don't have the quality of a human editor and inevitably it will make mistakes

Professor Vasileios Hatzivassiloglou, Columbia University

The software program trawls news websites including the BBC, Reuters and CNN, gathering articles that it sorts by topics such as world news, entertainment, science and technology.

It extracts nouns, verbs and phrases to measure similarities between articles and condenses it into a five-sentence summary.

The sentences can either be taken verbatim from the original article or rewritten in a slightly different form.

A programme allows it to put the sentences in order, relying in part on the order of information in the original articles.

Hatzivassiloglou: It makes mistakes

This is an extract from its summary of the death of the Queen Mother.

The Queen Mother's coffin, draped in her personal flag and crowned by a wreath of pink camellias, had been driven the 25 miles from Windsor Castle in a hearse flanked by police motorcycle outriders. Princes Charles paid a moving public tribute to his late grandmother Majesty Queen Elizabeth II on Monday, praising her "panache, style and unswerving dignity" as mourners across Britain left teddy bears, candles and wreaths in public places in her honour.

Not infallible

Quite often the summaries have longer sentences than standard news articles and some have an odd turn of phrase.

"It might form longer sentences because it is trying to pack a lot of information in," explained Vasileios Hatzivassiloglou, one of the professors working on the project.

He admits that the virtual journalist is not entirely foolproof.

It cannot exercise the editorial judgement needed to sort complex material and to achieve balance

Dr Damian Carrington, newscientist.com

It cannot, for example, summarise different sides of the story.

"It is an automated system and at this point we don't have the quality of a human editor and inevitably it will make mistakes," he said.

However 88% of those that have used the system have found it better than going to different websites for their news, said Professor Hatzivassiloglou.

Human jobs safe

Dr Damian Carrington, editor of web-based science journal newscientist.com believes that the software is no substitute for a human journalist.

"I think the program will be a threat to news aggregator websites, such as newsnow.co.uk, but not to real editors," he said.

"It can quickly chop down stories into bite-size chunks, but it cannot exercise the editorial judgement needed to sort complex material and to achieve balance.

"As for investigative or analytical pieces, it has no hope," he said.