Large tracts of farmland in the eastern region may need to be taken out of production to help protect water quality, a new report has warned.
Farmers said it was vital to continue to produce food in the UK
Adas, a rural policy think-tank, says the UK needs to cut the amount of nitrate pollution in water to meet European Union targets.
The Adas spokesman said "eastern farming hangs in the balance" as the problem is worst in the East.
This is due to the relatively dry climate in the region.
This leaves less rain water available to dilute water contaminants such as nitrates and phosphates.
Adas says that as many as 50% of farmers may have to look again at how their land is used.
Nitrates naturally occur in the soil as a by-product of growing crops as well as being an ingredient of fertilisers used by farmers.
Dr Stephen Bolt, the Suffolk-based head of Adas Environmental Management, told BBC News that there needed to be a national debate about how the country was going to deal with nitrate pollution.
He said stricter guidelines from the European Union about the level of nitrates (50 milligrams per litre) in drinking water are coming into force in 2009 and the UK could be fined hundreds of thousands of pounds if it failed to meet them.
"There needs to be more urgency about this debate," he said.
Prof Roger Sylvester Bradley, responsible for Adas' fertiliser research, said: "Good farming practice alone cannot sufficiently address the nitrate problem.
'More local food'
"The only way to safeguard the future of our water resource is to significantly reform land-use ... and convert much of our existing arable land into unfertilised, restorative grassland or forest."
Brian Finnerty, regional spokesman for the NFU, said: "It's strange [for people supporting the environment] to give up on food production in the UK and have it flown in from abroad with all the bad environmental consequences.
"We want to see more local food produced."
A spokesperson for the Department for Environment, Food and Rural Affairs (Defra) said the government was working hard to meet nitrate level targets.
"The Nitrates Action Programme operates in all areas that drain into water containing more than 50 milligrams of nitrate per litre of water," the spokesperson said.
"Existing measures under the programme are not providing effective protection of waters from nitrate pollution and need to be tightened, and we will be consulting shortly on revisions to those measures as part of the programme's four-yearly review."