Under the pact, each side is allowed a maximum of 1,550 warheads, about 30% lower than the 2002 Moscow Treaty.
It also limits the number of deployed delivery vehicles - ballistic missiles and heavy bombers - to no more than 700. However, each bomber counts as one warhead irrespective of the fact that it might carry multiple bombs or missiles.
Speaking after the signing ceremony, President Obama said the treaty demonstrated that both countries had halted the deterioration of their relations, which had prevented agreement on mutually important issues in the past.
"When the United States and Russia are not able to work together on big issues, it's not good for either of our nations, nor is it good for the world. Together we've stopped that drift and proven the benefits of co-operation," he added.
Mr Obama said the pact was "an important milestone for nuclear security and non-proliferation" and set the stage for further arms cuts.
"While the New Start treaty is an important first step forward, it is just one step on a longer journey. This treaty will set the stage for further cuts, and going forward, we hope to pursue discussions with Russia on reducing both our strategic and tactical weapons, including non-deployed weapons."
Dmitry Medvedev: Nuclear treaty a 'win-win'
He said the talks - beginning this summer - would cover missile defence, threat assessments, and the completion of a joint assessment of emerging ballistic missiles.
For his part, President Medvedev said the negotiating process had not been simple, but the treaty represented a "win-win situation" that would enhance strategic stability and bilateral relations.
"The result we have obtained is good," he said. "We have got a document that fully maintains the balance of interests between Russia and the US. The main thing is that there are no victors or losers here."
But Mr Medvedev said disagreements remained between Moscow and Washington over US plans for a missile defence shield, which have been modified by Mr Obama.
By Jonathan Marcus, BBC News, Prague
Numbers here are not hugely important though in the sense that these arsenals are still far in excess of what might be needed to deter each other or, for that matter, any other potential nuclear competitor.
This agreement really is a starting benchmark; a formal treaty that sets the scene for much more significant reductions in the future. Indeed, much of the new agreement's importance is in its collateral benefits.
It marks an important improvement in US-Russia relations and it gives President Obama in particular an important boost ahead of next month's review conference for the Nuclear Non-Proliferation Treaty (NPT). Bolstering this agreement, which is the central pillar of efforts to prevent the spread of nuclear weapons, is a high priority for him.
On Tuesday, Russia's foreign minister warned that it could abandon the New Start treaty "if a quantitative and qualitative build-up of the US strategic anti-missile potential begins to significantly affect the efficiency of Russia's strategic forces".
It was Moscow's concerns over Washington's plans to base interceptor missiles in Poland and a radar station in the Czech Republic that helped delay the new treaty. President Obama shelved the idea in September, although new plans include ground-based interceptor missiles in Romania.
The White House has said it hopes and expects the US Senate to ratify the New Start treaty this year. Senate ratification requires 67 votes, which means it must include Republicans.
The Russian lower house of parliament must also approve the treaty, but as long as the Kremlin supports it, ratification there is expected to be a formality.
During private talks before the signing ceremony, Mr Obama and Mr Medvedev also discussed Iran's nuclear programme.
The US wants the UN Security Council to approve tougher sanctions against Tehran, over its refusal to halt uranium enrichment.
"Unfortunately Tehran is not reacting to an array of constructive compromise proposals. We cannot close our eyes to this," Mr Medvedev said afterwards.
BBC diplomatic correspondent Jonathan Marcus, who is in Prague, says the real significance of this deal is that it marks a warming of US-Russian ties and heralds, perhaps, tougher Russian action on Iran's nuclear programme.
It also gives Mr Obama a disarmament success that he hopes will strengthen his hand at next month's review of the Nuclear Non-Proliferation Treaty (NPT), our correspondent says.
An overhaul of the 40-year-old pact is seen as the central pillar of the US president's efforts to prevent the spread of nuclear weapons.
However, much more significant cuts in long-range nuclear weapons could take years of negotiation with the Russians, who do not share Mr Obama's ambitious disarmament vision, our correspondent says.
Nuclear weapons are in fact looming larger in Russia's security equation at a time when their role in US strategic thinking is becoming more circumscribed, he adds.
On Tuesday, President Obama unveiled the new Nuclear Posture Review, which narrows the circumstances in which the US would use nuclear weapons.
"The United States will not use or threaten to use nuclear weapons against non-nuclear weapons states that are party to the Non-Proliferation Treaty and in compliance with their nuclear non-proliferation obligations," it said.
Countries which the US regards not complying with the NPT, including Iran and North Korea, will not be spared a nuclear response.
North Korea pulled out of the NPT in 2003, while the US claims Iran is secretly developing nuclear weapons, which Tehran denies.
Mr Obama also pledged not to develop any new nuclear weapons, a move pushed through in the face of resistance by the Pentagon.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.