The government has "every sympathy" for the case for abolishing blasphemy laws, Justice Minister Maria Eagle has said.
Several amendments have been tabled for the Criminal Justice Bill
Speaking at the end of a debate on proposals to scrap the laws, she told MPs they wanted to first consult the Church of England.
Labour MPs were told to vote against Lib Dem MP Evan Harris's proposals but ministers feared a backbench revolt.
Dr Harris, who called the law "ancient, discriminatory and illiberal", withdrew his Criminal Justice Bill amendment.
Ms Eagle said the consultation with the Church of England would be ""short and sharp" .
Subject to those discussions, the government intended to bring forward amendments to the Criminal Justice Bill to abolish the offences of blasphemy and blasphemous libel, she said.
During the debate, Dr Harris described the laws as "ancient, discriminatory, unnecessary, illiberal and non-human rights compliant". He said the last conviction under the law was 1979.
"So it's not needed any more, old-fashioned, ancient and out of time," he said.
"The Almighty does not really need the protection of these ridiculous laws and that's why large numbers of people of a religious perspective share the view that these offences need to be abolished," he said.
Labour MPs had been whipped to oppose the amendment - but there had been concerns that there would be a significant backbench revolt.
Earlier Labour MP Keith Vaz told BBC Radio 4's World at One: "A lot of MPs, including myself, would have voted for it - because this is a pretty archaic law."
He said he welcomed the government's moves to bring forward its own plans, adding: "There would have been a rebellion and I think the government would have faced a defeat."
Backbenchers involved in talks with ministers say the government is anxious not to be accused of "bouncing" the Church of England into accepting abolition.
Dr Harris's amendment to the bill comes after leading figures, including former Archbishop of Canterbury Lord Carey, wrote to the Daily Telegraph on Tuesday, arguing the legislation was discriminatory as it only covers attacks on Christianity and Church of England beliefs.
The letter said it served "no useful purpose" and offered Christian activists a means to intimidate broadcasters, publishers and performers.
The High Court last month rejected an attempt by a Christian evangelical group to prosecute the director general of the BBC for blasphemy over the screening of the musical Jerry Springer - The Opera.
Its director, Nicholas Hytner, told BBC Radio 4's Today programme: "What we have now is essentially a secular country based on a common bond of decency which includes free speech.
"And free speech includes the expectation that there should be mutual respect between those with different beliefs.
"And I don't believe that the law should address what people believe. The blasphemy laws protect belief; they don't protect people."
Don Horrocks, of the Evangelical Alliance, agreed that there was "no real argument" for retaining the law, saying: "Everybody knows it's not really going to be used again."
But he warned that changing the legislation could "send out a signal" that "gratuitous abuse and offence" is acceptable.