Deputy PM John Prescott's department is being accused of bullying, with one in 10 workers claiming to have been victimised in the past year.
Mr Prescott was criticised for not appearing at the committee
There is also concern at discrimination against black and disabled staff, according to a staff survey carried out by the department last year.
MPs on the committee which covers the work of the Office of the Deputy Prime Minister(OPDM), are demanding action.
The department's head said the concerns of staff were being addressed.
The survey results were revealed in the annual report and accounts of the ODPM, which covers housing, planning, local government and the regions.
The survey found 10% of staff felt they had been bullied in the past year, 8% had experienced discrimination and 6% reported harassment.
Some 22% of staff had witnessed unfair treatment and a larger proportion of black or black British staff (14%) had experienced discrimination than other staff.
The survey also found disabled people were more likely to have suffered discrimination and revealed that a third of staff did not know how to report unfair treatment.
Failure to appear
The department's top civil servant acknowledged that the bullying and intimidation was of real concern, which was welcomed by the committee.
But the committee said: "The department should take steps immediately to reinforce the message that bullying and intimidation is unacceptable."
"It should ensure that all staff are confident, such reports will be taken seriously."
The report also criticised the failure of Mr Prescott himself to appear before the committee, due to dealing with work connected with the UK's past presidency of the EU. "We believe that the most senior minister in a department should make himself available to a parliamentary select committee when his presence in sought," it said.
The committee's report, out on Thursday, also accused Mr Prescott's department of spin, highlighting "an unjustifiably favourable presentation of its achievements" in the ODPM's report.
'More to do'
Members also referred to a "leadership deficit" at board level with criticism from staff of the leadership, visibility and openness of the board.
Staff morale, in the light of the department's plans to cut hundreds of jobs through the Government's efficiency savings, also raised concern.
The committee also voiced concern over the £6.45 billion savings the department is demanding of local councils.
Commenting on the survey, ODPM Permanent Secretary Peter Housden, who is in overall charge of the office, said: "We have taken very seriously the views expressed in our stakeholder and staff surveys and put in place robust measures to address these concerns.
"We can demonstrate considerable strides forward over the past four years, but we all recognise we have more to do."
He had implemented a "wide-ranging" programme" to improve the communication and visibility of senior managers, he added.