The blast took place at 0800 local time (0500 GMT) during morning rush hour at a crowded intersection in the southern Pat district.
The bus was full of schoolchildren and office workers. The bomber was also killed.
It was the deadliest attack in Jerusalem since February 1996, when 26 people were killed in a bus explosion.
Tuesday's attack was also the worst since the Passover massacre in Netanya on 27 March, which triggered a major Israeli incursion into the West Bank.
"There was a huge explosion, smoke and pieces of the bus and body parts were flying everywhere. It was horrible," a witness told Israeli radio.
The bus - which had stopped at a traffic light after pulling away from a stop - was lifted into the air by the blast.
Police say the bomb was packed with high explosives and metal fragments designed to caused maximum casualties.
Islamic militant group Hamas said they had caused the attack, naming the bomber as 22-year-old Mohammed al-Ghoul from the al-Faraa refugee camp near the West Bank town of Nablus.
One of his brothers, Iyad al-Ghul was arrested, according to witnesses quoted by the French news agency AFP.
In a separate incident, Israeli commandos are reported to have killed a senior member of the radical Jihad group in the West Bank city of Hebron.
Condemnation
Israeli Prime Minister Ariel Sharon has convened a security meeting of the Israeli cabinet.
He made an unusual visit to the site two hours after the attack, and vowed to fight "Palestinian terror".
Jerusalem in firing line:
"The terrible sights we have seen here are stronger than any words," Mr Sharon said. "It is interesting to know what kind of Palestinian state they mean. What Palestinian state?"
The BBC's Jim Fish says the words were clearly aimed at influencing the United States and President George W Bush's long-awaited policy speech on a possible Palestinian state.
It is thought the speech - which the White House says will be given "very soon" - will include a recommendation for a provisional state with temporary borders and limited sovereignty.
"The Palestinian Authority condemns this attack," Palestinian Information Minister Abed Rabbo told AFP.
"The only beneficiary of this operation is Sharon and the occupation. Israel will use this operation to escalate its aggression against the Palestinian Authority."
Scepticism
Police have mounted a huge security operation in recent days after being warned of an attack on Jerusalem, setting up roadblocks and trying to locate the bomber with a helicopter.
But Jerusalem mayor Ehud Olmert told Israeli radio that the information received referred to a different bomber and that police were still on alert.
The attack came as Israel puts up a security fence along its border with the West Bank.
But the erection of the fence has been met with scepticism among both Israelis and Palestinians.