Hundreds of stray cats and kittens face being crushed during demolition at the Olympic stadium site, it is claimed.
Three litters of newborn kittens were said to have been found on the site
The Celia Hammond Animal Trust said London 2012 bosses had reneged on a deal to allow the charity access to the east London site to rescue cats there.
Former model Celia Hammond said about 150 cats and kittens on the site had been saved but more were at risk.
The Olympic Delivery Authority said it was discussing the timing of future visits with the trust.
But a spokesman said prolonged access to the site was no longer possible.
"We have been working closely with the Celia Hammond Trust for a number of months to capture and remove over 150 feral kittens and lactating cats," he said.
"Each building is thoroughly checked for wildlife ahead of demolition and the ODA has agreed to contact the Celia Hammond Trust if [cats] are discovered..."
He said there was significant contamination on the Olympic site and the ODA's number one priority was ensuring the safety of its workforce and visitors.
A petition on the Celia Hammond Trust website has attracted more than 5,000 supporters' signatures.
Actress Joanna Lumley is also supporting the campaign.