Fears have been raised about police powers
The defeat hinges on the new powers wanted by the government to detain foreigners suspected of being international terrorists.
It came as Home Office Minister Lord Rooker hinted the government was considering changes to the plans to get the legislation passed in force by Christmas.
The Conservatives say the Anti-Terrorism, Crime and Security Bill makes a false distinction between international terrorism and terror acts committed in the UK.
Their amendment was passed by 149 votes to 139, a slim 10 vote majority.
Appeasement claim
Former Tory chairman Lord Tebbit said the government's position was part of its policy of appeasing Irish republicans.
Conservative home affairs spokesman Lord Dixon Smith argued it was an "oddity" that the law made the distinction between domestic and international terrorism.
Drawing such a distinction was "invidious", he said.
The Liberal Democrats refused to back the Tory amendment, whose supporters were mainly Tory peers with some crossbenchers.
The setback for the government comes on the issue which prompted a joint attack on the plans from Conservative leader Iain Duncan Smith and Northern Ireland First Minister David Trimble.
In a newspaper article, they criticised the government for trying to create "good" and "bad" types of terrorist.
Tackling all terror
But the government says tough laws are already in place to counter domestic terrorism and those are continued by other parts of the new bill.
The change affects the parts of the bill allowing detention of foreign terror suspects without trial - one of the most controversial plans.
Some observers stress the government's anxiety to prevent any move to bring back internment in Northern Ireland.
Lord Rooker told peers: "We want to exclude from the detention powers in the legislation, domestic terrorism where it is conducted by domestic citizens because we have got other ways of dealing with it."
He said the ways proposed to deal with international terror would not be suitable for tackling "home grown terrorism".
After the vote, a Home Office spokesman said the bill targeted all types of terrorism, except for the parts bringing changes on immigration and asylum rules.
'Position the same'
"We maintain our position that the need to take these powers is based on the changed circumstances since 11 September relating to international terrorism.
"We regret that the Lords didn't agree. We will be considering the best way forward."
Lord McNally later expressed disappointment that the Conservatives had "hijacked" the debate by "narrowly focusing" on Northern Ireland.
But Tory Lords leader Lord Strathclyde called the defeat "striking and gratifying".
The defeat follows strong criticism for the bill during the committee stage of detailed scrutiny in the Lords.
Lord Rooker's comments will fuel speculation that ministers are ready to offer piecemeal concessions to stave off the risk of peers making major changes to the legislation.