The claim is made in a new book, "Your Life in Your Hands" by Professor June Plant which is published on Thursday.
Professor Plant says that excluding dairy products from the diet can prevent and treat breast cancer.
However, the charity Breakthrough Breast Cancer has challenged the claims.
Delyth Morgan, chief executive, said: "We are extremely concerned about the claims made in Professor Plant's book.
"With all due respect to Professor Plant, her views are not based upon scientific evidence.
"Adopting a healthy diet and lifestyle can help reduce the risk of developing cancer in general.
"But we do not yet have the evidence to promote the exclusion of specific foods to protect against breast cancer."
Ms Morgan said the key was to maintain a balanced diet.
"With so much anecdotal evidence about what foods can or cannot do for us, it's not surprising that women are left confused.
"It is worrying that such a diet is being promoted without scientific foundation."
Tumour shrinkage
Professor Plant, chief scientist at the British Geological Survey, contracted breast cancer in 1987. She had five recurrences of the disease, which, by 1993, had spread to her lymph system.
She says she finally defeated the disease by combining traditional medicine with simple lifestyle changes, including the adoption of a dairy-free diet.
She also says that other women have successfully fought breast cancer using her programme.
In her book, Professor Plant writes: "I believe that the evidence suggests that consuming dairy products (milk and meat) including low-fat products such as yoghurt, does indeed increase the risk of breast and prostate cancer in exposed populations.
"When I eliminated all dairy products from my diet, a large 'secondary' tumour comprising cancer cells in the lymph nodes in my neck, which was thought to be incurable, disappeared in weeks.
"To me, this is pretty compelling evidence. I believe that I, and all those who have suffered from breast cancer but shared in my advice, have avoided death by dairy."
'My book is scientific'
Professor Plant told BBC News Online that she was very disappointed with the stance taken by Breakthrough Breast Cancer.
"My book is very heavily based on science.
"I knew I would get this sort of attack, but nevertheless I thought that morally I had to put this knowledge in the public domain so that people could judge for themselves."
Professor Plant said there was a wealth of scientific evidence to suggest that dairy products were linked to cancer.
For instance, rates of breast and prostate cancer were much lower among women in the Far East until they started to adopt Western diets and eat dairy products for the first time.
In China, breast cancer is referred to as "rich woman's disease" because it afflicts the more affluent classes who tend to eat a westernised diet.
She said milk was known to contain growth factors that stimulate breast development in teenage girls, and chemicals that mimic the effects of the female sex hormone oestrogen - which has been strongly linked to breast cancer.