Mr Brown, who has had record low approval ratings in some polls over recent months, said as leader he expected to be in "the firing line" as the economy slowed and people "started" by blaming the government.
But, he added, people were then seeing the reasons for the slowdown were global ones such as oil trebling and the US credit crunch.
The prime minister is keen to get the Millennium Development Goals back on track.
The goals, agreed by 189 heads of state in September 2000, represent a global commitment by all of the nations who signed to reduce poverty and improve lives.
There are 18 targets which are supposed to be met by 2015.
These include ensuring children are educated to primary schooling level and halving the numbers of people who contract malaria and HIV.
Mr Brown, along with Microsoft boss Bill Gates, used the summit to announce a new $3bn drive to eradicate deaths from malaria in the developing world by the 2015 deadline.
The funds, which will come from a variety of public and private sector bodies, will be used to try to develop a vaccine and pay for the distribution of bed nets and sprays.
Sarah Brown met Jordan's Queen Rania and Rupert Murdoch's wife Wendi
The drive is expected to save the lives of one million people who die from the disease every year.
The UK's support for research into malaria treatments and vaccines is to reach at least £5m a year by 2010. It has also pledged to provide up to £40m towards improving the availability of anti-malarial drugs.
The summit will also see further commitments made on the issue of education and healthcare. They include an additional $2.9bn - which includes a £50m contribution from Britain - to help 10 million children go to school.
Sarah Brown, also at the summit, co-hosted a dinner on Wednesday for influential women - including many first ladies, US vice-presidential hopeful Sarah Palin and Sarah Ferguson, the Duchess of York - to raise awareness of the need to improve childbirth support in the third world.
Other celebrities expected to be at the summit include Elle Macpherson, Bono and Bob Geldof.
Mr Brown will also be calling on major donors to make up a $700m funding gap to tackle food shortages, particularly in the Horn of Africa.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.