Fathers on paternity leave receive a statutory £123 a week
Almost half of working fathers do not take their right to two weeks' statutory paternity leave because they cannot afford to, research suggests.
The Equality and Human Rights Commission said paternity pay should be increased from the statutory £123 a week to 90% of fathers' actual pay.
It found 40% of men felt they spent too little time with their children.
Some 20% of men among the 4,500 parents polled feared asking for flexible working would harm their careers.
New mothers are currently entitled to nine months of paid leave, six weeks of which is paid at 90% of their salary and the rest at the statutory rate of £117.18 a week.
As well calling for an increase in paternity pay, the commission called for parents to be able to share four months of parental leave between them and for it to be paid for eight weeks at 90% of salary.
In Norway and Germany, fathers on paternity leave are given 60-80% of their income and the period of time they have off is ring-fenced.
Anne Madden, policy lead on improving working for the commission, said: "We were rather taken aback with the difference in take-up of paternity leave here, in comparison to other countries.
"Other countries have thought it through and developed structures which better support fathers to take time off."
She added: "The idea that we are a nation where fathers are the breadwinner and mothers stay at home and look after the children has shifted now.
"The current system doesn't reflect the way families operate at all and is running behind the behaviour of modern families."
The study also suggests that six in 10 fathers work more than 40 hours a week and half think they spent too much time at work.
One in three of the fathers questioned said the availability of flexible working was important when looking for a new job.
Andrea Murray, acting group director of strategy at the commission, said the research highlighted an opportunity for businesses to use flexible working as an incentive for attracting and retaining the most talented employees.
"Some companies which have adopted forward-thinking policies towards families are reporting increased productivity, reduction in staff turnover, reduced training costs and an ability to respond better to customer requirements," she said.
Meanwhile, Adrienne Burgess, from the Fatherhood Institute think tank, said those men not taking paternity leave "are taking it in annual leave, because it's fully paid".
However, she added: "That, of course, has a really bad knock on effect, because then they haven't got any leave to speak of in the rest of the year to take.
"And so this means the mothers are enormously unsupported, the fathers are not having nearly enough time to get to know their babies, and the families are under terrible strain."
The study was based on a 2008-2009 YouGov quantitative survey of 4,500 parents in England, Wales and Scotland of children aged under 16.
It also used qualitative data from several online forums involving a wide range of groups of fathers.