Mr Woolas has made a controversial start as immigration minister
Phil Woolas has rejected criticism from the Archbishop of York about his stance on immigration and asylum issues, saying "being tough is not immoral".
Dr John Sentamu attacked "unmerciful" immigration policies in a speech on Thursday and comments by Mr Woolas about asylum lawyers.
Although he took the criticism "very seriously" Mr Woolas said it was moral to have a "fair and efficient" system.
Mr Woolas has sparked much controversy since becoming immigration minister.
Dr Sentamu condemned his "tough talking" rhetoric and said attitudes to Zimbabweans seeking asylum in the UK lacked mercy.
And he singled out recent comments by Mr Woolas that many lawyers for asylum applicants undermined the system by dragging out appeals and did "more harm than good", saying they were simply wrong.
Dr Sentumu also suggested the language used by Mr Woolas on sensitive issues since being appointed to the job in October had muddied the waters in the immigration debate.
Mr Woolas has said he was appointed to raise the profile of the government's immigration policy and get its message across to readers of tabloid newspapers.
In a recent interview in the News of the World, he vowed the government would "kick out" more illegal immigrants next year.
He has also said he wanted to reassure people that Britain's population will not reach 70 million as some experts, including the office for national statistics, have predicted although he has said he does not favour a "cap" on immigration.
"May I be forgiven for suggesting that the honourable member in question does not advance his stated desire to have 'a mature debate about immigration' by this carry on?" Dr Sentamu argued.
Mr Woolas told BBC Radio 4's Today programme that he did not believe his comments on immigration and asylum polices were either "unmerciful or authoritarian".
"I don't accept the central charge that being tough is being immoral," he said. "I would argue the opposite."
"I think the morally right thing to do is to have an efficient and fair immigration and asylum system."
Mr Woolas said he would not back down from his argument that some delays in the asylum process was caused by lawyers "frivolously" dragging out the appeals process.
Dr Sentamu had said Mr Woolas' stance was "worrying" given the number of initial decisions refusing asylum subsequently overturned.
But the minister said unnecessary delays in the process "perpetuated" the suffering of applicants and said he believed it was moral to ensure decisions were taken faster.
However, he pointed out that he was not accusing the majority of lawyers of such behaviour and accepted that some delays in the asylum process were the result of failings in the system itself.
"You cannot manage a system unless it is efficient. That is fairer for the immigration and asylum seekers who are using the system."