Members of the Awakening movement guarded the crowd as a crane removed sections of the wall.
The Sunni leader stood on the base of the crane next to his Shia counterpart, and inveighed against sectarianism.
"Iraq is one family," he said - to cheers from the crowd. "We are united now, Sunni, Shia, Christian, Turkmen and Kurd. One Iraq."