Charles Clarke has rejected calls to stop sending back to Zimbabwe people whose UK asylum claims have failed.
Protesters in London were voicing their opposition to Mr Mugabe
The home secretary said successful applicants could stay - but argued that a blanket suspension of all removals would lead to asylum abuses.
Dozens of Zimbabweans in detention centres across the UK have gone on hunger strike, while the Lib Dems urged a review of 116 people's cases.
Tory spokesman David Davis called UK Zimbabwe policy a "miserable failure".
He said Robert Mugabe's regime was guilty of "crimes against humanity on a massive scale".
"We wouldn't be facing the issue we are today if [the UK government] had shown a clearer lead in the past, put greater pressure on governments such as that of [South Africa's] Thabo Mbeki and forced the issue on to the agenda of the UN Security Council," said the shadow home secretary.
ZIMBABWEAN HUNGER STRIKERS BY REMOVAL CENTRE
Source: Home Office
Note: Campaigners say some 100 are refusing food
But Mr Clarke argued: "Not all Zimbabweans who claim asylum here genuinely face persecution."
He acknowledged that members of the African nation's opposition faced prosecution by Mr Mugabe's regime and needed "international protection".
And he pointed out that in the 15 months to March either asylum or leave to remain was granted to 270 Zimbabweans.
He said that of those who had been returned there had been "no substantiated reports of mistreatment".
Earlier Tony Blair said the danger of halting all deportations was that a signal would be sent around the world "that Britain is open for business" even for failed asylum seekers.
"If we engage in a generalised moratorium, our fear is that we would literally be back in the situation we were two or three years ago where people were hammering us for not getting the asylum system under control," he said.
A ban on deportations to Zimbabwe, which had been in force for two years, was lifted last November.
During the first three months of this year 95 Zimbabweans were sent home.
Recent moves in Zimbabwe to demolish illegal buildings - which the UN says has left 275,000 people homeless - have drawn objections from the Foreign Office.
Hundreds of thousands have been left homeless in recent weeks
UN envoy Anna Tibaijuka has arrived in Harare to assess the demolitions of illegal shantytowns.
On Monday the Home Office said 57 people due to be deported from the UK were now refusing food, up from 41 at the weekend.
However, the group organising the protest, the United Network of Detained Zimbabweans in the UK, said some 100 people were involved.
The group said this figure included 46 women in the Yarl's Wood centre near Bedford, 35 people near the Harmondsworth centre near Heathrow and 18 at nearby Colnbrook.
Maeve Sherlock, head of the Refugee Council, said she was "extremely disappointed" ministers had failed to recognise the dangers face by people forced to return to Zimbabwe.
"Our responsibilities are clear: even if someone is not accepted by our government as being a refugee, we do not send them back into danger, and Zimbabwe is clearly not safe," she said.
The Home Office meanwhile said a hunger strike of some 100 inmates at the Campsfield centre in Oxfordshire came about after a Turkish asylum seeker was found dead, possibly after committing suicide.