Mr Rumsfeld said the campaign would be long, difficult, and dangerous, and more lives might be lost.
US Deputy Defence Secretary Paul Wolfowitz will brief Nato defence ministers in Brussels on Wednesday, and rally support for possible US military action.
The coalition against the Taleban has been further strengthened, after Saudi Arabia announced it was severing diplomatic ties with the regime in Afghanistan.
The Saudi Government is however still apparently resisting US pressure to allow American fighter planes to use its bases for an attack on Afghanistan. Pakistan is now the only country in the world maintaining diplomatic links with the Taleban.
Click here for a map of possible targets President Pervez Musharraf has said that the country should maintain contacts with the Taleban, but on Sunday it did withdraw its staff from Kabul.
In other developments:
The Saudi Government has given Afghan diplomats 48 hours to leave the country.
He also discussed co-operation with the visiting Japanese Prime Minister Junichiro Koizumi.
He repeated that the US wanted action against the Taleban, not the Afghan people.
"They have made the decision to harbour terrorists," he said.
"The mission is to root out terrorists, to find them and bring them
to justice.
"And the best way to do that, and one way to do that, is to ask
for the cooperation of citizens within Afghanistan who may be tired
of having the Taleban in place."
'Wrong and cruel policies'
Taleban leader Mullah Mohammad Omar said on Tuesday that the 11 September attacks on the United States were to avenge the "cruelty" of American foreign policy.
"The American people must know that the sad events that took place recently were the result of their government's wrong policies," read the message from Mullah Omar, delivered through Pakistan-based private news agency, the Afghan Islamic Press.
The Taleban maintain that they do not know the whereabouts of Bin Laden, and he has disclaimed any responsibility for the attacks, which killed more than 6,800 people.
It was Mullah Omar's second message to the American people in two days. On Monday, he warned the US Government to withdraw its forces from the Middle East and stop its support for Israel if it wanted to eliminate the threat of terrorism.
Coalition building
US President George W Bush has been briefing leaders of Congress on the US military deployments within striking distance of Afghanistan, telling them not to expect a conventional war.
In a message to the American people, he reiterated that Osama Bin Laden, the Saudi-born dissident whom the US blames for the bombings, was incapable of such a sophisticated attack.
Related to this story:
US military threats dismissed
(24 Sep 01 | South Asia)
Saudi statement in full
(25 Sep 01 | Middle East)
US split on Bin Laden evidence
(26 Sep 01 | Americas)
A significant step for Russia
(25 Sep 01 | Europe)
Who is Osama Bin Laden?
(18 Sep 01 | South Asia)
Will Bush's asset freeze work?
(24 Sep 01 | Business)
Analysis: Building a coalition
(16 Sep 01 | Americas)
UN agency reviews airline security
(26 Sep 01 | Business)