It added that children with identified mental health problems, or deemed to be at high risk of developing them, should not be placed in detention.
Professor Steve Field, chairman of the Royal College of General Practitioners, said children in immigrant families were already disadvantaged and vulnerable.
"Detaining children for any length of time - often without proper explanation - is a terrifying experience that can have lifelong consequences," he said.
"As well as the potential psychological impact, these children invariably experience poor physical health as they cannot access immunisation and preventative services."
Professor Alan Maryon-Davis, president of the UK Faculty of Public Health, said the issue went "straight to the very heart of social justice and human rights".
He said: "It is unfair and wrong to exclude these very vulnerable children and young people from equitable access to normal health and social care.
"The UK is a civilised nation - let's demonstrate that by ending this discrimination right now."
The policy statement was "warmly welcomed" by the Children's Commissioner for England, Sir Al Aynsley-Green, who in April said the practice of holding children in detention should be stopped.
Last month, the home affairs select committee said it was "not acceptable" that some were being detained for up to two months.
Lisa Nandy, policy adviser at the Children's Society, said: "We have seen first hand the damage that immigration detention causes - many of the children we work with experience depression, bed-wetting, weight-loss and even self-harm."
David Wood, head of criminality and detention for the UK Border Agency, said: "We agree with the Royal Colleges that families at Yarl's Wood should get the same level of care available on the NHS, and they do.
"The Royal Colleges have relied on research which is up to three-years-old. There have been significant changes since.
"Yarl's Wood Immigration Removal Centre has been praised on numerous occasions for its children's facilities."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.