The Inter-American Development Bank (IDB) has said debt relief is possible in South and Central America, but only if the bank can be made more agile.
Mr Morales has been urging Mr Moreno for debt relief
The IDB's president, Luis Alberto Moreno, was speaking in Brazil during the institution's annual meeting.
Mr Moreno said the IDB's funds were not trickling down to the poorest people.
Mr Moreno was responding partly to calls from Evo Morales, Bolivia's president, to eliminate $3.5bn (£2bn) in regional debt.
Mr Morales had called for a portion of the $3.5bn debt owed by Bolivia, Haiti, Honduras, Guyana and Nicaragua be written off.
The challenge for Mr Moreno will be how to use loans most effectively in a region where economic progress has been much slower than in many parts of Asia, analysts say.
Bolivia owes the majority - 48% - of the total debt to the IDB, according to Brazil's governmental newswire.
"I want to create a bank that is closer to its clients, the people of Latin America, and I will foster decentralisation," said Mr Moreno.
"Secondly, we will make the bank more agile, reducing the time between the approval and the disbursement of loans."
Mr Moreno also voiced his support for infrastructure projects in Latin America and the Caribbean.
The US delegate to the meeting in Belo Horizonte, Brazil, said more loans needed to be given to private companies.
More than 90% of job creation in the region comes from small and medium sized companies, said a US Treasury Department official.
Mr Moreno is facing pressure from environmental and indigenous rights groups, who argue that the bank's loans do not help out the poor but do cause environmental disaster.
Of particular concern has been the bank's financing of the Camisea project, a gas pipeline in Peru's Amazon.
It has leaked five times since it started pumping gas in 2004 and the bank has provided $135bn for the project.
Opposition groups at the meeting are hoping to stop the bank from providing further funding of up to $400bn, to provide money for a liquefied natural gas (LNG) project in the Amazon.