Summary Care Records are central to the NHS computer upgrade in England
Doctors leaders have called for a halt in the development of a medical records database for patients in England.
The British Medical Association says the computer-based Summary Care Records are being set up at "break-neck speed", sometimes without patients' knowledge.
Ministers have expressed surprise at fears of fast change after previous criticism that it was moving slowly.
The NHS IT upgrade will link more than 30,000 GPs to nearly 300 hospitals through an online appointments system.
It will also feature a centralised medical records system for 50 million patients, e-prescriptions and faster computer network links.
But the plans have received repeated criticism over security fears and a lack of enthusiasm among doctors for the technology.
Doctors' leaders have written to the government calling for parts of the programme to be suspended.
In a letter to Health Minister Mike O'Brien, the British Medical Association called for further independent evaluation of pilot schemes set up to test the system.
Summary Care Records are central to the NHS computer upgrade in England, which is the biggest healthcare IT programme in the world.
The basic record includes information on allergies, medication and adverse reactions. Further details may be added over time.
Patients are told by letter before their details go on the system, giving them the chance to opt out.
In December, the Department of Health announced that the roll-out of the records system would be accelerated.
To date, more than 1.25 million records have gone on the database and the process is speeding up. The BMA say this is happening too fast.
It says some people are not aware that they are getting these records. It also argues that opting out should be easier.
Hamish Meldrum, chairman of the BMA, said the "break-neck speed" with which this programme is being implemented is of "huge concern".
He added: "Patients' right to opt out is crucial, and it is extremely alarming that records are apparently being created without them being aware of it.
"If the process continues to be rushed, not only will the rights of patients be damaged, but the limited confidence of the public and the medical profession in NHS IT will be further eroded."
But Dr Simon Eccles, the medical director for the agency delivering the IT programme, Connecting for Health, said many patients were astonished that hospital doctors did not have access to basic information such as prescribed medications.
He told the BBC: "It's incredibly important that where people want it, they will be able to share health information that will save their lives because clinicians will know what is wrong with them."
However, the government says the process to opt out is already straightforward.
It also argues that the BMA has supported similar schemes in Scotland and Wales.