He added that the UK was "working to ensure that the will of the people of Zimbabwe is upheld".
Mr Brown said: "No one thinks, having seen the result at the polling stations, that President Mugabe has won this election.
"A stolen election would not be an election at all. The credibility of the democratic process depends on there being a legitimate government.
"So let a single message go out from here that we are and will be vigilant for democratic rights and that we stand solidly behind democracy and human rights for Zimbabwe and we stand ready to support the Zimbabwean people build a better future."
Zimbabwe's opposition Movement for Democratic Change believes leader Morgan Tsvangirai won last month's election outright.
But this is disputed by the governing Zanu-PF party, with the full results not yet released.
Earlier this week, Mr Mugabe, in power since Zimbabwe achieved independence in 1980, dismissed Mr Brown as a "tiny dot in this world".
During his US visit, Mr Brown plans to meet presidential hopefuls Barack Obama, Hillary Clinton and John McCain to talk about bilateral relations between Britain and the US.
He will also hold talks with US President George W Bush and Federal Reserve chief Ben Bernanke on global economic problems.
Gordon Brown interviewed on Good Morning America
And he is expected to urge Wall Street bankers, at a meeting later, to reveal their financial losses quickly.
The start of Mr Brown's trip was somewhat overshadowed in the US by the visit of Pope Benedict XVI, who was greeted by the president and cheering crowds on his first official trip to the US.
In an interview with ABC's Good Morning America, the prime minister was asked what he liked about the job.
He said he had had to cope with a series of problems, from terrorist attacks to foot and mouth and the global credit crunch, but added: "Every day you wake up and you know there's going to be a new challenge. And it will be different from the day before.
"But it is one of the best jobs in the world, and one of the most challenging."
He also wrote in the Wall Street Journal about plans for the British Council and American organisations to help anyone in the world who wanted to learn to speak English.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.