The dad of the baby boy Madonna wants to adopt has said he didn't want to the singer to have his son "for good".
Yohane Banda said he thought Madonna was going to make sure 13-month-old David got a good education before returning him home to Malawi in Africa.
He added that because he can't read or write someone from the government had to explain what was happening.
He said: "Had they told us that Madonna wanted to adopt my son and make him her own son, we would not have agreed."
Mr Banda added: "I would prefer that David goes back to the orphanage where I can see him any time I want, rather than send him away for good."
Madonna was allowed to take David to her home in London after Mr Banda signed the papers giving her permission.
David will live with Madonna and her family for 18 months and then a final decision will be made on the adoption.
A number of charity groups in Malawi don't think that Madonna should adopt David. They say that the law doesn't let anyone who isn't from Malawi adopt a child from that country.