The British Medical Association is set to call for the NHS to utilise the skills of qualified refugees at its annual conference in Harrogate this week.
There are about 650 qualified doctors registered by the BMA/Refugee Council list, but only a small proportion of them are actually working.
In the West Midlands, there are 70 doctors, only 12 of whom have jobs in the NHS.
In total, experts estimate there could be about 1,500 medically qualified asylum seekers and refugees.
The majority come from Iraq, Afghanistan or Somalia.
'Burger bars'
Doctors have to pass medical and language tests before they can work in the NHS, and it can take around two years before they can look for jobs in the health service.
Many have to take work in burger bars or on building sites to raise enough money to pay for the language and medical tests they must pass, which can cost almost £700.
And like all doctors coming from outside the European Union, refugee doctors have to have proof they have been offered a job in the UK before they can be registered as fit to practise.
Dr Edwin Borman, chairman of the BMA's International Committee told BBC News Online things were improving for refugee doctors arriving in the UK.
But he added: "It is still very difficult. It's difficult arriving in a foreign land, being subject to a system which is not easy to understand.
"They are then required to learn a new language, which they might never have spoken before, find somewhere to live, and adopt to the way the NHS works."
He added: "Training a junior doctor in the UK can cost around £200,000.
"Here we have refugee doctors for free, who are looking for the opportunity to contribute their skills."
'Barriers'
Dr Borman said he hoped the NHS, BMA and other organisations could work together to help refugee doctors and nurses and dentists work in the UK.
Edie Friedman, director of the Jewish Council for Racial Equality, said: "There needs to be more resources and more thought given to the whole process of resettling refugees.
"There are so many barriers.
"People need to have access to English tuition, somewhere adequate to live. Refugee doctors also need clinical attachments, and tuition in medical English as well."