Some inner-city primary schools do not spend money intended for gifted children correctly, as they feel it is unfair to other pupils, a report says.
Targets for children are "too variable", the report says
Instead it is used for such things as textbooks and computer software for those of all abilities, Ofsted said.
The government's Excellence in Cities scheme aims to help children in England break out of a "cycle of poverty".
But four out of 28 schools surveyed by the education watchdog did not think higher standards were attainable.
'Culture of blame'
A "pervasive culture of blame" held parents responsible for their children's poor test performances, Ofsted said.
Excellence in Cities (EiC) was launched in 1999 to improve standards, originally in inner-city secondary schools.
These lagged far behind the national average in exam results, and had higher truancy and expulsion rates.
EiC concentrates on underachievement among gifted pupils and provides "mentors" for the badly behaved.
In September 2000 it was extended to 1,104 primary schools.
Ofsted said the scheme was "beginning to have a positive impact" at this level.
However, the setting of targets by heads was "still too variable", while monitoring was below standard in "just over a third" of cases.
Ofsted said some schools were holding children back, with money allocated for "gifted and talented" pupils being "diluted".
One head - not named in the report - told inspectors: "The gifted and talented initiative is divisive and, at this stage in my career, I don't feel threatened by league tables and targets. I am concentrating on the whole child."
The report said: "A small number of schools believed that the gifted and talented strand of the programme was not conducive to promoting equal opportunities."
These were not breaking the law and Ofsted said they would face no penalties as a result of failing to spend the money for its intended purpose.
The report asks the government to provide better leadership of the EiC programme and calls on local authorities to focus their efforts on schools with weak head teachers.
However, some schools are praised for having already introduced initiatives of their own.
One - Soho Parish School, in London - had a "technologist in residence" week, where children ran scientific experiments designed to enthuse them.
It also put on an extension maths class for bright pupils.
In three-quarters of the schools visited by Ofsted, there were "good" partnerships between parents and staff.
A Department for Education and Skills spokeswoman said EiC was "driving up standards" and that the rate of improvement in test results was "outstripping the national average".
She added: "Ofsted itself notes that EiC is realising its potential in the overwhelming majority of schools, finding just four schools who were unable to fully embrace the EiC vision, and we will be working with them to ensure that their pupils can benefit from the programme."
Deborah Eyre, director of the National Academy for Gifted and Talented Youth, said: "[Schools'] failure to spend the money on gifted and talented children shows little faith in the range of ability of their own pupils and creates much more stark inequalities of opportunity for gifted children across our schools.
"It will impact particularly badly on gifted children in our most deprived schools."
Ofsted's report - Excellence in Cities: the Primary Extension - is based on data compiled between spring 2003 and summer this year.