There used to be a time when faced with the problem of where to send your child to school you asked around or even sent it to the nearest one. Now, each year, the government publishes performance tables with information about how primary and secondary schools are doing.

The government just lists the schools in alphabetical order. It is the news media which perform the crucial act of re-arranging the schools according to their scores in the form of a league table. It seems the government is a mite uncomfortable about accepting full responsibility for the enormous interest these league tables evoke among parents hungry to choose the best schools for their children.

There is no doubt that performance tables seem to make things a lot easier for parents. Instead of having to compare schools in terms of whether or not they were impressed by the headteacher's speech on the open day or how impressive the science block was, the whole thing is boiled down to a number.

But not everyone sees performance tables as a good idea because as well as simplifying things numbers can also be misleading.

Dr Keith Devlin, Dean of Science at St Mary's College of California, says one of the greatest potential dangers with statistics is to take numbers developed to deal with the inanimate world and apply them to the world of people with all of their imprecision and unpredictability.

"You take those numbers, you apply them to people, you make a calculation and you've produced some numbers correct to three or four decimal places but what do they tell you?

'They tell you nothing'

"They probably tell you nothing at all because people, most of the time, cannot be measured to three or four decimal places. People just don't work that way."

The numbers in school league tables obtained by dividing the number of GCSE passes by the number of children in a school are not as straightforward as they look because they are not only numbers they are statistics - in other words, rather confusing mixtures of how well the schools have actually done and purely chance factors. As such they need to be treated with care.

"Statistics will be very useful if you're a government minister because you're dealing then with large populations of students with large numbers of schools. But if you're a parent dealing with one or two children the statistics are going to be far less useful to you."

It seems that the statisticians are perfectly well aware of the limitations of performance tables. So why is the information given out at all?

Snappy

Professor Ted Wragg of the School of Education, University of Exeter, says the appeal of a league table is its simple accessibility.

"It's snappy and instant and you can see it at a glance but you can't have, alongside it, a huge account of every single school and its background factors so it's very difficult to say: 'This school came tenth out of a hundred schools in this area but it had children of very low ability, the staff must have done marvels and the children must have worked very hard but another school that came 20th had very bright pupils and should have been top.'

By taking into account some of these special factors some of the distortions might be made to go away. But unfortunately, in the end, there will always be one distorting factor that will not go away, which is the element of pure chance.

It is well known to statisticians but not often to politicians or journalists that all statistics are confused by chance and among other things this critically affects the measurement of schools.

Professor Stephen Senn of University College London says a school's superior performance in GCSE exams in a given year may well be due to superior teaching or superior pupil intake - but it may well be due merely to chance.

Particular distortions

"I know myself as a marker we try our best to be honest and fair in marking but nevertheless there's a certain subjective element in that and these scripts are marked by different people as well and that's something which could be affected by chance.

"It could well be that there's some particular problem in the town or the community which affects results in a given year and so forth.

Professor Harvey Goldstein, being a statistician, naturally puts the whole thing in mathematical terms:

"Supposing you were to take, from a set of a hundred schools, just one student and look at their examination results and then rank those results so that you finish up with a rank ordering of a hundred schools.

"If now you took another child from each school you would, of course, expect a different ordering because we know that children vary.

"So the issue, then, is how many do you need to take to be able to say something precise? And the answer is: more children than actually exist in most schools."

With the typical size of school the uncertainty and what statisticians call 'the sampling error' is so large that most schools cannot be statistically separated. That means when you try and compare one with another the uncertainty swamps any difference that you have.

"It does invalidate the league tables there's no question about that," he says.

What to do?

So if you are a parent and you are trying to decide between two schools there does not seem to be much point in looking at the performance tables because the uncertainties involved in creating the statistics are too great. What are parents supposed to do?

Prof Wragg says that if you are a social snob you would probably go for the schools that are high in the league tables because by and large they are the schools that are attended by the children of middle class, professional parents.

"However, the real question - and it's very hard to answer it from a league table - would be: If my child is capable of getting say a grade B in an exam, if they went to one of these schools they might only get a C because the school might only be good with the very brightest or it might simply get good results because it does have very able children.

"If they went to one of the schools that was lower down but had very good teaching then they might actually get an A not a B because this school is very good at improving on what people come in with.

"So a league table simply can't tell you that unless you were able - and it would be extremely complicated to do it - to superimpose on it ... some kind of grid that would tell you the raw material with which each school in the league table was actually working."

Value added

There is a proposal to construct just such a grid. It is called a value added performance table. The idea is to measure how much the school adds to its pupils' education.

Prof Wragg says the rationale behind value added is that you do not simply regard every school as having started at the same point.

"What you do is you look at the starting point and compare it with the end point.

"So the point about value added is you try and get some kind of measure at the beginning - if it's five year olds you do a baseline test to see what their language development's like, if they've got any grasp of number and so on.

"The result of doing that kind of thing is you turn some of the league tables on their heads. What's going to happen when we get value added tables ... is that there will be some huge shock waves going through the education system.

"Schools that no one has ever heard of will come very high and secondly schools that are very well known and regarded as successful may well come in the middle or below the middle."

It can only get worse

There is another problem. Statistics is a subject notorious for being scattered with logical traps. And unhappily for anxious parents and educational planners the nastiest of them all lies precisely in this area of league tables.

It has to do with the idea of an average. Parents want to send their children to above-average schools not below-average ones.

But Dr Devlin says that as soon as you quote an average people get very worried about the things that fall below it - yet the very nature of an average is that roughly half of the data will be above the average and half of the data below.

"There's the old joke about the man who had his head in the oven and his feet in the refrigerator. When asked how he felt, he said: 'On average I feel just fine.' The average clearly is meaningless in that case.

"Averages, in fact, are only useful if you know how much the actual figures disperse around that average. If all of the data is close to the average then the average tells you something very useful. If the data's spread way, way out from the average then the average really doesn't tell you very much at all."

Top and bottom

And unfortunately the most eagerly read parts of league tables are the bits at the top and the bottom of them. Everybody wants to know which the best schools are and which are the worst. But in doing so they fall foul of something nasty lurking in the statistical woodshed.

"One of the things that often happens when you start quoting statistics is people look for the school or the company or the stock that's the top of the league table today - the thing that's right at the top," says Dr Devlin.

"They forget the fact that anything that's at the top, there's only way for it to go and that's down."

The principle is called 'regression to the mean'. According to the way you look at it, it seems either plain common sense or extremely puzzling.

It says that what goes up will come down to the average and at the same time what is at the bottom of the table will eventually come up to meet the average.

So for example if your child came top of the form this year the chances are that next year it will not. This sort of thing can happen whenever institutions or people are arranged in order of merit.

Beware

"The main consequence of this for studies of public health and public policy are that one must keep in mind the idea of regression," says Prof Stigler, "not let it overwhelm your senses, it is not the only thing that's happening - but the evaluation of changes over time should only be done cautiously with a full analysis of the consequences of regression."

So the next time you glance at a league table beware. One of the reasons why league tables are so popular is that they are used to manage people and institutions by providing what seems to be concrete numerical evidence about performance.

It is considered a triumph for example if an inner city sink school which was sitting on the bottom of a league table one year improves its ranking the next. And it might be - but it might also be just another example of regression to the mean.

The lesson seems to be that statistics can give you a good overall picture of what is going on but if you want to know exactly how your school, hospital or even favourite football team is doing then you need to watch out.