The Americans, from Idaho-based charity New Life Children's Refuge, said the children had lost their parents in the earthquake, and they were taking them to an orphanage in the Dominican Republic.
But the BBC's Rupert Wingfield-Hayes, in Port-au-Prince, says the Americans could be in deep trouble as some of the children may not be even orphans.
An international charity, SOS Children's Villages, which is now looking after the children, says at least one of them, a little girl, said her parents were alive.
George Willeit, the spokesman for the aid group in Port-au-Prince, told journalists the girl said she had believed she was being taken to a boarding school or summer camp.
Mr Willeit said many of the children were also found to be in poor health, hungry and dehydrated.
One of the smallest, just two or three months old, was so dehydrated she had to be taken to hospital, he added.
Haitian authorities said none of the children, some as young as two months, had documentation or proof that they were actually parentless.
The leader of the Idaho-based group, Laura Silsby, said the arrests were the result of a mistake.
"Our understanding was that, we were told by a number of people including Dominican authorities that we would be able to bring the children across," she said.
"The mistake we made is that we didn't understand there was additional paperwork required."
But our correspondent says the regulations are very clear - each case of child adoption must be approved by the government.
AT THE SCENE
Karen Allen, BBC News, Port-au-Prince
In the grounds of a former children's hospital, in one of the worst-affected areas of Port-au-Prince, there is a queue of about 200 women snaking around the building. It's all very orderly - the women are coming in and collecting a bag of rice which is to be shared between two families.
Some distance away, behind security tape, the men are being held back. The reason is that, in the past, food distribution points have been trigger points for potential violence and scuffles.
The UN estimates that up to a third of the population will be reliant on some kind of relief in the months to come.
Even before the earthquake, he adds, child-smuggling was a massive problem in Haiti, with thousands of children disappearing each year.
In the capital Port-au-Prince, the UN's World Food Programme (WFP) has begun feeding families - distributing food only to women - at a series of 16 sites across the capital.
Each woman will receive a 25kg (55lb) bag of rice, enough to feed her family for two weeks.
Only women will be allowed to collect earthquake relief supplies, because - the WFP says - this has proved that is the best way to get food to the people who need it.
The WFP will work with the local authorities to ensure that men in need of assistance are not excluded, it said.
"Up until now the nature of this emergency has forced us to work in a 'quick and dirty' way simply to get food out," said Executive Director Josette Sheeran.
"This new system will allow us to provide food assistance to more people, more quickly through a robust network of fixed distribution sites."
The WFP says it has reached 600,000 people with over 16 million meals since the earthquake, amid huge logistical problems caused by damage to local infrastructure.
Meanwhile, doctors working to treat survivors voiced concern over the US halting the evacuation of the critically injured to the country.
Dr Barth Green, Dr Cathy Burneit and Dr David Pitcher describe difficulties in Haiti
A senior US medic told the BBC that scores of people injured in the earthquake could die if the US did not resume emergency evacuations soon.
"The consequences - in the kids with crushed chests and on ventilators and respirators, and some of the adults - are they will die," Barth Green, a senior American doctor at a field hospital in Port-au-Prince airport, told the BBC.
The US military stopped the flights to Florida on Wednesday.
A White House spokesman told the BBC the move was due to "logistical issues", not because of a row over medical costs as had been reported earlier.
Hundreds of patients with spinal injuries, burns and other wounds have been evacuated to the US since the 12 January quake that killed up to 200,000 people.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.