BBC NEWS Americas Africa Europe Middle East South Asia Asia Pacific Arabic Spanish Russian Chinese Welsh

 You are in:  Sci/Tech
Front Page 
UK Politics 
Talking Point 
In Depth 

Commonwealth Games 2002

BBC Sport

BBC Weather

Tuesday, 9 April, 2002, 07:46 GMT 08:46 UK
Computer scribe hits the web
Newsblaster website
Newsblaster aggregates thousands of news stories
test hello test
By Jane Wakefield
BBC News Online technology staff
Online journalists could find their jobs under threat as a virtual reporter has been created that trawls the web for all the best stories.

The Newsblaster is a piece of software designed to edit, summarise and rewrite the huge amount of news currently on offer in cyberspace.

It is the brainchild of computer scientists at Columbia University in the US.

The project has been funded by grants from the National Science Foundation and the Defence Agency.

Extracts sentences

The software was originally intended to make it easier for intelligence gatherers to find information on the web.

But it could also be used to aggregate the thousands of news stories on the internet.

At this point we don't have the quality of a human editor and inevitably it will make mistakes

Professor Vasileios Hatzivassiloglou, Columbia University
The software program trawls news websites including the BBC, Reuters and CNN, gathering articles that it sorts by topics such as world news, entertainment, science and technology.

It extracts nouns, verbs and phrases to measure similarities between articles and condenses it into a five-sentence summary.

The sentences can either be taken verbatim from the original article or rewritten in a slightly different form.

A programme allows it to put the sentences in order, relying in part on the order of information in the original articles.

Professor Hatzivassiloglou
Hatzivassiloglou: It makes mistakes
This is an extract from its summary of the death of the Queen Mother.

The Queen Mother's coffin, draped in her personal flag and crowned by a wreath of pink camellias, had been driven the 25 miles from Windsor Castle in a hearse flanked by police motorcycle outriders. Princes Charles paid a moving public tribute to his late grandmother Majesty Queen Elizabeth II on Monday, praising her "panache, style and unswerving dignity" as mourners across Britain left teddy bears, candles and wreaths in public places in her honour.

Not infallible

Quite often the summaries have longer sentences than standard news articles and some have an odd turn of phrase.

"It might form longer sentences because it is trying to pack a lot of information in," explained Vasileios Hatzivassiloglou, one of the professors working on the project.

He admits that the virtual journalist is not entirely foolproof.

It cannot exercise the editorial judgement needed to sort complex material and to achieve balance

Dr Damian Carrington,
It cannot, for example, summarise different sides of the story.

"It is an automated system and at this point we don't have the quality of a human editor and inevitably it will make mistakes," he said.

However 88% of those that have used the system have found it better than going to different websites for their news, said Professor Hatzivassiloglou.

Human jobs safe

Dr Damian Carrington, editor of web-based science journal believes that the software is no substitute for a human journalist.

"I think the program will be a threat to news aggregator websites, such as, but not to real editors," he said.

"It can quickly chop down stories into bite-size chunks, but it cannot exercise the editorial judgement needed to sort complex material and to achieve balance.

"As for investigative or analytical pieces, it has no hope," he said.

See also:

28 Mar 02 | Sci/Tech
Robo-reporter goes to war
13 Feb 02 | Sci/Tech
When technology makes the news
14 Sep 01 | Sci/Tech
Search engines swamped
Internet links:

The BBC is not responsible for the content of external internet sites

Links to more Sci/Tech stories are at the foot of the page.

E-mail this story to a friend

Links to more Sci/Tech stories