A third of people who are undergoing fertility treatment do not tell their employers because they fear it will harm their career.
More support at work for infertile couples
A survey carried out by Infertility Network UK of nearly 300 patients showed many worried their boss would not be sympathetic.
One in seven couples in the UK experience fertility problems.
The charity has produced a fertility information leaflet to encourage employers to be more supportive.
The survey, which was done for National Infertility Day, found that although the majority of people tell their employers they are having fertility treatment, 29% felt they could not broach the subject.
Two thirds of those patients had kept their treatment secret because they were worried about the consequence of their employer knowing they were trying to get pregnant.
The other reasons for not telling employers were worries about work colleagues finding out, concerns that employers would not be sympathetic and feeling that their infertility was a private matter.
Although the amount of time patients had taken off for treatment varied considerably, from just one day to a few weeks, the majority had taken less than ten working days.
More than half had taken time off as sick leave, 47% had used annual holiday leave, 14% had taken unpaid leave and 20% had been given paid leave by their employer for their treatment.
Some employers recognise that infertility is a medical condition, and allow their staff to take time off to have treatment.
According to Infertility Network UK, examples of good employers are Asda, who offer five days paid leave with the option of extra unpaid leave and London and Quadrant Housing, who give staff unlimited time off for fertility treatment.
But they stressed that the responses to the survey suggest that not all employers appreciate that infertility is an illness deserving of treatment.
The leaflet aims to explain what happens when an employee is going through fertility treatment, and how a degree of flexibility from their employer can ease the emotional strain on them, minimise disruption and make the process easier.
Clare Brown, chief executive of the patient support group Infertility Network UK, said worries about work were a recurring theme when people called for advice.
"The emotional and physical impact of infertility and treatment is often underestimated," she said.
"Couples are having to deal with a very distressing illness that they can never get away from, and anything employers can do to help should be encouraged."
Dr Allan Pacey, Honorary Secretary of the British Fertility Society, said: "I'm not surprised by the survey results, dealing with any illness and your employer is very difficult.
"But infertility treatment is something that can last for several years, and it can be very stressful.
"A bit of flexibility from employers, to allow staff to go for things like blood tests, would make all the difference."