The Football Association has rejected Bury's appeal against their expulsion from the FA Cup.
Casper offered to resign over the mix-up
Bury were thrown out for fielding an ineligible player - on-loan Hartlepool midfielder Stephen Turnbull - in their second-round replay win over Chester.
The FA reinstated Chester, who are set to face Ipswich Town in the third round on 6 January.
"If that's justice, the FA can keep their justice as far as I'm concerned," said Bury manager Chris Casper.
Casper had offered to resign after the Football Association ejected Bury but the club's board turned him down.
The Bury boss added: "When teams lose, fielding an ineligible player, nothing happens basically. Is that right? "We also got the same punishment as people that don't come clean."
The punishment will cost Bury £24,000 in prize money for beating Chester and 45% of the gate receipts had they played Ipswich.
"On a personal level my feelings for the players and the supporters, I'm hurting like hell," said Casper.
"I've never felt like this and I feel so sorry for them, but we're going to have to move on quickly and we can't feel sorry for ourselves."
Turnbull is on loan from Hartlepool but Bury did not get the official go-ahead for him to play in the FA Cup.
Hartlepool manager Danny Wilson had given Caspar verbal permission for Turnbull to play but the paperwork was never sent to Bury.
Chester manager Mark Wright insisted the FA had made the correct decision.
"Rules are rules - and they are there not to be broken," said Wright. "Bury must be gutted, and it's unfortunate for them.
"Of course, I feel for them. But the FA make the rules; they have to stick to them, and the right decision was made."