Ministers are to press on with plans for a controversial electronic medical records system.
There are fears the system will not be secure
The government's patients' tsar Harry Cayton will say the system, which will hold records for 50m people in England, is needed to modernise the NHS.
Only people who can prove the system will cause them substantial mental distress will be exempt.
But doctors warned creating the record without a patient's consent could harm the doctor-patient relationship.
Under the system, everyone will have a computer-based care file with basic information such as medication and allergies, drawn from GPs' records.
A poll of over 1,000 GPs by the Guardian newspaper last month found half would consider refusing to put patient records automatically on to a new national database.
Many said they doubted the security of the new system.
Pilots will begin in the spring with national roll-out expected by the end of the year.
The government says it aims to make unscheduled treatment - including care in emergencies - quicker and safer, as well as protect patient confidentiality.
Patients will only be able to have their records removed if they can show holding them will cause them substantial mental distress.
However, they will be allowed to check the details are correct and make amendments online.
How more detailed and sensitive data will be stored is still being looked at.
Proposals for a so-called "sealed envelope" containing information which will only be shared on the patient's agreement are under consideration.
The electronic records system is part of a 10-year IT programme aimed at linking more than 30,000 GPs in England to nearly 300 hospitals by 2012.
It will also involve an online booking system, e-prescriptions and fast computer network links between NHS organisations.
Mr Cayton will publish a report on Monday making the case for the system.
The report is expected to say the current record-keeping system, which uses paper records, desperately requires modernisation.
It will point out many patients are harmed every year due to the unavailability of records or lack of quality data.
Health Minister Lord Warner said it had been designed to "improve care and treatment" to patients.
"The idea that any Tom, Dick or Harry can access the information is simply nonsense."
A British Medical Association spokesman said: "The BMA supports, in principle, the concept of an integrated centralised health record system.
"But it also recognises that there are important decisions for patients to make. Patients need to be properly informed and enabled to make informed choices.
"It is BMA policy that patients should give explicit consent before any healthcare data is uploaded - ie they should specifically opt-in to the system."
And Michael Summers, of the Patients Association, added it could have a damaging impact on the doctor / patient relationship.
"It will lead eventually to sensitive information not being divulged to GPs.
"What we feel is that provided that proper information about certain treatments or allergies or a particular condition is included, I don't think patients would object. But they don't want any more information [adding]."