She added: "We've overturned a Labour majority of 5,500 to a Conservative majority of over 7,000. If we hold this seat at the general election, we'll have a majority of over 100."
The Lib Dems came third with 4,803, narrowly ahead of UKIP on 4,068, who enjoyed their best ever by-election result. The Greens - who are the main opposition party on the local council - were pushed into fifth on 3,350, although it was also their best-ever by election result.
Gordon Brown said it was "clearly a disappointing result" for Labour but people should consider what was happening in the constituency.
"The voters were clearly torn between their anger and dismay at what's been happening with MPs' expenses, something we have been trying to clean up and at the same time support for the former MP, the Labour MP Ian Gibson, who was very popular.
"I don't think any party can take a great deal of cheer from this, the Conservative vote went down, the Liberal vote when down - only the fringe parties saw their votes going up."
Liberal Democrat leader Nick Clegg said that result was a "disaster" for Labour and he hailed the "excellent campaign" of his party's candidate, April Pond, who gained a slightly reduced share of the vote compared with 2005.
Ms Smith, who at 27 will be the youngest MP in the House of Commons, paid tribute to Dr Gibson in her victory speech saying she hoped to serve the voters "with the same honesty and conviction".
She also wished her Labour rival Mr Ostrowski a speedy recovery - he was hospitalised with suspected swine flu days earlier. His wife represented him at the count.
Ms Smith said the result sent a signal to Downing Street: "The people of Norwich North have rejected the old politics of personal attacks, of bickering, of smears and scare stories.
"They have voted for change. And in doing so they have sent a message to Gordon Brown very loud and very clear."
The Conservatives had put a lot of effort into the by-election. Party leader David Cameron paid six visits and many members of the shadow cabinet had been to Norwich North to campaign.
Dr Gibson had won the seat with a majority of 5,459 at the 2005 general election. Friday's by-election result represents a swing of 16.5% from Labour to the Conservatives.
Voter turnout was 46% - less than the 61% who voted at the general election but about average for recent by-elections.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.