Lord Smith said more gay people were being attacked
The government has suffered another defeat in the Lords over a so-called "free speech" defence to a new law on homophobic hatred.
Peers voted by 179 to 135 to keep the defence - despite justice minister Lord Bach pointing out MPs had voted to overturn it on four occasions.
It is one of several government defeats in the Lords on its wide-ranging Coroners and Justice Bill.
It will go back to MPs on Thursday, the last day of this Parliamentary session.
The defence was inserted in the Criminal Justice and Immigration Act - which created the offence of inciting homophobic hatred - last year by the Conservative peer Lord Waddington.
The government ran out of parliamentary time to try to overturn it then but this year inserted a clause into the Coroners Bill that would have removed it.
However with the Parliamentary session due to end on Thursday, ahead of the Queen's Speech next week, they may run out of time again.
For the government, Lord Bach said MPs had voted to overturn the defence on four occasions, by majorities of more than 150.
"There must come a point where this House, with all its great virtues, gives way to the House that has been elected by the people of this country," he told peers.
The defence provides protection for "discussion or criticism of sexual conduct or practice" to the law on incitement to hatred on the grounds of sexual orientation.
Lord Waddington said his amendment did not weaken the offence of stirring up homophobic hatred but paralleled "that in the religious hatred offence".
He asked what the point was of repealing it "without waiting to see whether, in practice, it interferes in any way with proper enforcement of the law".
And former senior judge Baroness Butler-Sloss said she supported it because while she did not "support what the minority says, I support their right to say it because there is, and should be, freedom of speech".
Former chief constable, crossbencher Lord Dear, said he had been approached by many police officers who backed the defence because it would "allow them to use discretion and common sense" and deal with situations "with a light touch".
But independent peer - the former Labour cabinet minister Lord Smith - said he supported the government.
"I happen to be gay. I happen also to be Christian. I like to believe that I am robust enough to be able to be criticised, to have offensive things sometimes said to me because of my sexual orientation," he said.
But he said while he would defend people's freedom to criticise, violence against gay men and lesbians was on the rise.
"My fear is that if the signal that is sent out is that it is all right to be intolerant, if that's the signal this House sends, then I fear that we will end up seeing more violence and more attacks," he added.
The vote sets the stage for a last-minute clash between the Commons and the Lords, with the Parliamentary year due to end on Thursday.
Earlier in the week MPs also overturned Lords amendments on the bill that opposed government plans for "secret inquiries" to replace inquests in deaths involving intercept evidence and a bid to remove infidelity as a partial defence for murder.