Doctors have called for better funding and more reliance on family doctors for out-of-hours GP cover.
Family doctors no longer have to provide out-of-hours services
Night and weekend shifts are staffed by teams of GPs, nurses and paramedics in the majority of areas.
But doctors at the British Medical Association's GP conference said some local health bosses were trying to get the service on the cheap.
They also said a GP should be on hand at all times, or patients could be put at risk.
Out-of-hours cover has proved controversial since nine in 10 GPs opted out of providing the service last year under the terms of a new contract.
Many patients are now seen by nurses or paramedics.
But Dr Peter Holden, a member of the BMA's GPs committee, said every patient who received an out-of-hours home visit should be seen by a GP.
"I am in favour of mixed teams, but if someone requires a home visit I think only a GP has the expertise to make sure they receive the best treatment," he said.
"If a patient is well enough to travel to get their treatment it is fine for them to be seen by someone who is not a GP."
Family doctors were also critical of the funding arrangements during their annual conference, where a motion was passed criticising the current arrangements and calling for improvements.
Dr Andrew Green, a GP from East Yorkshire, said primary care trusts, which fund the out-of-hours service, had been driving the price meaning the service was being put under strain.
And Cornish GP Phil Dommett said funding in his area had been cut by £2.2m.
The London conference passed a motion calling for better funding and use of GPs for out-of-hours cover.
"If the government and public want a service based on home visits it must be funded properly."
Earlier this year it was claimed GP co-operatives - groups of doctors and nurses which cover half of the out-of-hours services in England - were being under-funded by 10%.
One co-operative in North Yorkshire has already folded after having a request for more money rejected.