It is unacceptable that it will be four years before trains return to pre-Hatfield crash punctuality levels, the national watchdog for rail users has warned.
Many passengers' last journey was 'late, overcrowded or unpleasant'
In its annual report the pressure group likened the railways in the past year to a duck; with furious activity below the surface, but little impact above the waterline.
The Rail Passengers Council (RPC) said people using the railways had been asked to wait patiently for better performance for too long.
It warned Railtrack's successor, Network Rail, that it had a long way to go and challenged it to bring the "spiralling costs of work" on the network under control.
RPC chairman Stewart Francis said: "With the railways
and their passengers, perception is only as good as their last journey.
"For too many passengers, their last journey was still late, overcrowded or unpleasant."
Mr Francis said the Hatfield crash, in October 2000, was still causing knock-on problems for the smooth running of trains.
And he said the May 2002 Potters Bar accident, which killed seven people, had cast a shadow over the early part of the report period.
Mr Francis said the causes of the derailment remained "worryingly unanswered".
He asked: "How can a whole year go by without the matter being closed?"
The RPC said passengers' journey experiences over the past year had been extremely varied.
Improvements were reported in many areas, including Scotland and England's eastern, north western, western and London areas.
But passenger groups in Wales, the Midlands and southern England all said services had got worse.
Mr Francis praised the train companies who now had nine out of ten trains arriving on time.
But he said that performance across the rail industry as a whole had "limped along".
Mr Francis said he remained optimistic that passengers could be delivered "a safe, reliable and affordable railway".
But he attacked fare increases with the warning: "The danger is that the railways could price themselves off the travel market for the less well-off in this country."
Mr Francis said it was "a sterile argument to say you can have cheap fares or a good railway but not both."
"Passengers are frequently told 'Take the pain to get the gain'. I would just like to get the gain defined," he said.