Tumour blood vessels (in red), surrounded by cancer cells (in blue)
A type of drug designed to stunt tumour growth has actually been found to fuel cancer if given at too low a dose.
UK scientists were investigating a kind of drug called an anti-angiogenesis, still under development, which hampers the growth of tumour blood vessels.
Avastin and Sutent, which act in a similar way, have been proven to work and were not covered in this research.
But cancer experts say the study in Nature Medicine could help make those drugs more effective.
The researchers focused on a drug called Cilengitide which is designed to prevent blood vessel cells sticking together and moving - an important part of angiogenesis.
Previous tests on people have found that a few patients with brain tumours benefited from high doses of the drug, but that it failed to work for most.
In this research, tests carried out on mice showed that low doses of Cilengitide actually stimulated the growth of cancers.
Further investigation showed it did this by switching on a molecule called VEGFR2, which triggers the angiogenesis process.
That is significant because although when a patient is initially given a drug, its level in the blood rises quickly ensuring a big dose goes to the tumour, after a while levels start to fall as the body begins to deal with the drug.
This is likely to be why trials of the drug have shown such poor results.
Dr Kairbaan Hodivala-Dilke of the Institute of Cancer, who led the study, said it was important that the trials looking at this drug continued.
"We've got evidence now that low doses can enhance tumour growth. So there is no benefit of giving a high dose, which then drops, and then a high dose again.
"But that's not to say it can't work at all. It can, but there is this caveat."
She said it may be more effective to give the drug via an infusion pump, which would allow the dosage to remain topped up at an effective level.
Dr Andy Reynolds, from the Breakthrough Breast Cancer Research Centre at the Institute of Cancer Research, who also worked on the study, said: "Knowledge of this mechanism will help us develop new ways to make these drugs as effective as possible.
"In the future, we may be able to combine these inhibitors with other drugs to maximise their effectiveness for patients."
Dr Lesley Walker, director of cancer information at Cancer Research UK, which helped fund the research, said: "This study is important because it may help to explain the mixed results previously seen in patients and turn around disappointing results so people may still benefit from the drug without the potential harm."
But Dr David Reardon of Duke University Medical Center in the US who is involved in the trials of Cilengitide, said the drug had shown promise for patients with glioblastoma (brain tumours).
He added: "If sufficiently positive, the results of these trials will lead to its approval to further improve outcomes for patients with this devastating disease."
The research also has implications for the existing drugs Sutent, used to treat kidney cancer, and Avastin, for colorectal cancer.
They work by the same process, but on different targets.
At the moment, they can extend a patient's life by several months. Experts hope that this research could lead to a better understanding of the drugs' mechanisms and so to ways of making them more effective.
Dr Walker said: "Sutent and Avastin have proven effective enough for use in the NHS but there is still need to understand why they can sometime fail.
"It may be that there are similar mechanisms at work."
Last month, the National Institute for Health and Clinical Excellence - the NHS's drugs appraisal body - said Sutent could be considered for people with advanced kidney cancer. However, it decided against recommending Avastin for advanced colorectal cancer.