He said he planned to make sure wage rises were kept low in the next three-year deal, starting in 2011.
He went on to stress there were no plans to cut the number of teachers and teaching assistants, but said reductions could be made to the number of bureaucrats and senior staff without the quality of teaching suffering.
Mr Balls spoke of comprehensives merging to form federations, so a head teacher and a team of deputies would work across the different schools.
He estimated this option could save the department about £500m a year.
He told the BBC's Politics Show that "pooling leadership together" in primary and secondary schools could free up resources to release back to the front line.
"If a third of schools did that, that would be getting us about a third of a billion pounds savings," he said.
Another £250m could be saved by losing about 3,000 senior school jobs, mainly through "natural wastage", he told the Sunday Times.
It was also likely that more than 300 jobs in Whitehall which involve advising schools about the curriculum could go, he added in the interview.
Mentioning the head teacher resonates with pupils - if they were absent, this wouldn't be the case.
The BBC's Ross Hawkins said that for months the government had refused to say spending cuts were needed, and now the schools secretary had revealed in detail where Labour planned to make some of them.
He added that it "opened the floodgates" and other ministers would be pushed during interviews to set out the exact nature of any cutbacks.
Liberal Democrat leader Nick Clegg strongly opposed the schools secretary's proposals.
"It would be madness, absolute madness, as a society, to blight the life chances of the young as the economy comes out of recession," he said on the BBC's Marr programme.
Nick Clegg: "I think it's a very silly idea"
"The people who're least to blame for what's happened are the very young.
"And if we want to make sure the shadow of this recession doesn't hang over young people for generations to come: long term unemployment, social divisions then we need to deal with that."
The Shadow Chancellor, George Osborne, says Mr Balls is lacking credibility. "Just a couple of months ago he was promising real increases in the schools budget in the months ahead, and the years ahead, so there's a total credibility gap", he says.
Mick Brookes, general secretary of the The National Association of Head Teachers, said: "The impression he [Mr Balls] gives is that head teachers are among the 'bureaucrats' who can be replaced.
"If he thinks that, what does he think they're doing all day?"
He said the deluge of bureaucracy and regulations poured on to schools in recent years by the government had to stop if head teachers were to get away from "form-filling" to spend more time with children.
"We're looking for him to get his own house in order before criticising school leadership," he added.
The National Union of Teachers says there are areas where savings can be made, but rejected cuts in staff.
"We've criticised the academies programme, where actually there is what we believe is overspending on some very senior head teachers.
"So there are certainly places where spending can be reorganised and getting rid of the academies would be a good one," commented NUT General Secretary, Christine Blower.
But another teaching union, the NASUWT, said the government could cut staff in federation schools.
"For example three schools together, you're retaining three governing bodies, three administrative teams, three head teachers, and sometimes a head teacher over that."
"We've said that needs to be looked at, and looked at in terms of value for money and cost, but we wouldn't want to see anybody being made compulsorily redundant, " Chris Keates, General Secretary of the NASUWT, said on the BBC News Channel.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.