Genuine refugees could be turned away from the UK under Tory immigration limits unless there is an unforeseen world crisis, Michael Howard has said.
Mr Howard says the UK must recognise reality on asylum
Under the plan, Parliament would set a quota - expected to be about 20,000 - for refugees every year.
Mr Howard told ITV's Jonathan Dimbleby programme the policy recognised the reality that there was a limit to the number of refugees the UK could take.
The Tories say the UK would still take more genuine refugees than it does now.
They plan to use off-shore processing centres to stop people whose asylum claims fail coming to the UK.
Labour instead says the quotas are inflexible and some refugee groups claim they could put lives at risk.
Asked what he would do if faced with victims of persecution applying for asylum
after the quota had been filled, Mr Howard said: "We would say 'I am really
very sorry but actually there is a limit to the number of refugees we can
There were millions of people who could justifiably claim they were persecuted and the UK could not possible take them all, he argued.
"We cannot possibly take them all into this country," he said.
"It is a completely unrealistic expectation. We should be honest and up-front
about this and say 'yes, we are going to play our share in this country, we are
going to take a significant number of refugees'.
"But you cannot, in all honesty, expect a small and crowded island to accept
a wholly disproportionate burden."
'No turning right'
Mr Howard, whose family fled from Romania to Wales as refugees in 1939, said there were many other possible destinations for refugees.
"All we are saying is 'don't pay the people-smugglers, don't come from
these distant parts of the world to Britain because you think it is going to be
much easier in Britain. We are taking our fair share, but I am sorry to say we
can't take you all'."
Mr Howard's comments come despite shadow chancellor Oliver Letwin saying the Tory plans would not include a cap on refugees.
The Tory leader denied Labour claims he was "lurching to the Right" and said he was "leading from the centre".
His quota plan was branded unworkable by the Refugee Council.
A spokeswoman said: "If we renege on our responsibilities, how can we expect poor countries like
Chad to take in refugees fleeing persecution in Darfur?"
Refugee penalty worries
Meanwhile, the United Nations High Commission for Refugees has said lives could be jeopardised by new government deportation plans.
An order being discussed in the House of Lords on Monday would allow refugees to be deported if found guilty of "particularly serious crimes".
UNHCR's representative to the UK, Anne Dawson-Shepherd, said she had "serious concerns" about the plan.
The interpretation of the type of crime involved was too wide so it could see people deported at the risk of their lives for relatively minor offences.
But a Home Office spokesman said the great majority of offences covered by the order included violence, sexual crimes, abuse of children or drug dealing.
"The safety of citizens in this country is paramount," he said. "David Blunkett [the home secretary] is not prepared to have criminal asylum seekers abuse our hospitality."