In December, Barack Obama ordered the deployment of an extra 30,000 US troops to Afghanistan.
Only a few thousand of the extra forces have arrived so far, with most expected to be in place by the summer.
Shortly after his speech to the troops, the US president, who is travelling on his Air Force One jet, left the country.
US troops make up the majority of the Nato-led coalition force in Afghanistan.
Speaking to the large indoors gathering at Bagram, Mr Obama said: "There is no visit that I consider more important than this visit I am making now.
AT THE SCENE
Lyse Doucet, BBC News, Kabul
Mr Obama came here as senator in 2008, when he was very critical of the Afghan leadership. When he came into the White House, the big question was, when would he feel the moment was right to make his second visit?
President Hamid Karzai has been waiting since his controversial re-election last year for an invitation to visit the White House. Only yesterday, I heard his aides complaining about the president not being asked, and asking if this cast some shadow over the relationship.
So there is no mistaking the important symbolism of a visit that is going to send a strong message of support for Mr Karzai.
Mr Obama made it clear he was here to back Mr Karzai and the Afghan people, but also to thank the tens of thousands of US troops in Afghanistan for their sacrifices made.
"My main job here today is to say thank you on behalf of the entire American people."
He added: "The Afghans have suffered for decades, decades of war, but we are here to help Afghans forge a hard-won peace... and we want to build a lasting partnership founded upon mutual interest and mutual respect."
Mr Obama said America had not chosen the war, and had not sought to meddle in Afghan affairs or expand its influence.
It had, he said, been "attacked viciously on 9/11", and al-Qaeda leaders and their Taliban allies were still in the region and had to be defeated.
The US aimed to deny al-Qaeda a safe haven and reverse the Taliban's momentum, he said.
He promised troops that they were being "backed up by a clear mission and the right strategy" and that they would have "the support to get the job done".
"I'm confident all of you here are going to get the job done in Afghanistan," he said to applause and cheers.
Operations in Pakistan
Mr Obama also talked of operations across the border in Pakistan, where he said the Pakistani government had pushed "violent extremists out of their sanctuaries".
"We have struck blows against al-Qaeda's leadership as well as the Taliban's," he said.
"They are hunkered down. They are worried about their own safety. It's harder for them to move and to train and to plot and to attack.
"All of that makes America safer and we are going to keep them on the run."
In the 14 months since he took office, Mr Obama has almost doubled American forces in Afghanistan.
US troops are now leading a massive military operation to push Taliban militants from their stronghold in the south of the country.
The BBC's Steve Kingstone in Washington says Mr Obama's visit comes at a time when he has momentum domestically because he has just signed a new healthcare reform bill into law and agreed a new nuclear arms reduction treaty with Moscow.
The White House clearly wanted to build on that momentum and say that this is a president who can deal with more than one big issue at once, our correspondent says.
Also, there is an impression that US efforts in Afghanistan may be turning a corner, he adds, prompting a cautious sense of optimism.
Invitation to Karzai
At a joint press conference in the heavily guarded presidential palace, Mr Karzai thanked Mr Obama for US support and said he hoped the partnership would continue.
President Karzai thanked President Obama for US support
"The American people are encouraged by the progress that's been made," Mr Obama said, adding he hoped to see more improvement on governance, anti-corruption efforts and judicial process.
The pair also held private talks, as well as meetings involving Afghan cabinet ministers and senior Obama administration officials.
The White House confirmed that the Afghan president would visit Washington on 12 May.
US officials initially said President Karzai had been informed of Mr Obama's impending visit just an hour before his arrival.
However, White House spokesman Robert Gibbs later said the Afghan government was told about the trip on Thursday.
Mr Obama travelled to Afghanistan during his campaign for the 2008 presidential election, while still a senator, but this is his first visit to the country since becoming the US commander-in-chief in January 2009.
There were strained relations between the White House and the Karzai government last year over the disputed Afghan elections.
Mr Obama paid a similar unannounced visit to Iraq, where US forces are also involved in a long-running conflict, last year.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.