He said America's most vital national security interests were at stake in the region and that the US "cannot succeed in Afghanistan without Pakistan's support and involvement".
Mr Holbrooke cautioned against describing Pakistan as a failed state and reconfirmed US support for President Asif Ali Zardari, saying Washington's goal "must be unambiguously to support and help stabilise a democratic Pakistan headed by its elected president".
But he said Pakistan had to "demonstrate its commitment to rooting out al-Qaeda and the violent extremists within its borders".
The US believes that Taleban fighters are able to seek sanctuary in Pakistan's border regions in order to launch attacks in Afghanistan.
Mr Zardari insisted the Taleban would not cause his government to collapse.
"They're not threats to my government. They are threats to my security," Mr Zardari told CNN.
"My government is not going to fall when one mountain is taken by one group or the other."
Speaking at the Brookings Institute think tank in Washington DC, Mr Karzai said his country could never be stable or peaceful unless alleged Taleban sanctuaries and training grounds in Pakistan were removed.
He said Afghanistan would use the talks to do "all that it can in immense friendship and brotherhood with Pakistan and alliance and friendship with America" to address the issue.
BBC diplomatic correspondent Jonathan Marcus says Mr Obama is fighting on multiple fronts.
The talks come at a time of increasing Taleban activity in the region
He needs to convince Pakistan's leaders of the gravity of the situation but also to convince Congress to approve the huge aid package for country, a vital element of the US strategy.
Wednesday's talks between Mr Obama, Mr Zardari and Mr Karzai come at a time of new unrest in both countries as a result of Taleban fighting.
Thousands of residents of Pakistan's Swat Valley have fled their homes as a peace deal between the army and the Taleban appeared close to collapse.
Officials say up to 500,000 people are expected to flee and authorities are preparing six refugee camps for them.
There have been clashes between the army and militants in Swat and heavy battles in neighbouring districts.
A major army operation against the Taleban in Swat now seems likely, the BBC's Syed Shoaib Hasan reports from Islamabad.
Afghan MP Khaled Pashtun told the BBC there were high expectations for the US talks.
He said the US should "compel [Pakistan] to work closely and work in a comprehensive way to bring peace and security".
One key issue for Mr Karzai will be continuing civilian casualties from coalition attacks.
The US has said it is investigating reports that more than 20 Afghan civilians were killed by US air strikes in Farah province on Tuesday.
Fighting there also reportedly left 25 Taleban militants dead.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.