Victim Arthur Budzinski says Vatican members knew about the scandal
Pope Benedict XVI failed to act over complaints during the 1990s about a priest in the US who is thought to have abused some 200 deaf boys, victims say.
As head of the Vatican office dealing with sex abuses, the then Cardinal Ratzinger allegedly did not respond to letters from an archbishop on the case.
A Church trial of the priest was halted after he wrote to Cardinal Joseph Ratzinger pleading ill health.
The Vatican newspaper said the claims were an "ignoble" smear attempt.
The Holy See has been plagued in recent months by abuse cover-up claims in Europe, echoing a similar scandal that hit the Church in the US eight years ago.
David Willey, BBC Vatican correspondent
Hardly a day goes by without new allegations of sexual abuse of children by Catholic priests somewhere in the world being reported in the media.
The Pope's spokesman defended Benedict, saying the Vatican department which the future pontiff was in charge of had not been informed of these latest allegations until 1996 - 20 years after the priest's victims first informed the police.
But the Vatican's rather lame excuse for lack of any action is that canon law, as Church law is called, "does not envision automatic penalties".
The Catholic Church teaches that paedophilia is a grave sin, but the evidence is that accused priests were usually moved to another parish rather than punished.
While the Pope is now promoting a policy of zero tolerance to clerical abuse, the suspicion remains that for many years he failed to react to the damning evidence which arrived on his desk.
For more than 20 years before he was made pontiff, Cardinal Ratzinger led the Congregation for the Doctrine of Faith - the Vatican office with responsibility, among other issues, for response to some child abuse cases.
An archbishop wrote letters in 1996 to the Vatican watchdog led by Cardinal Ratzinger calling for disciplinary proceedings against Fr Lawrence Murphy, according to Church and Vatican documents.
Fr Murphy was a popular priest who is believed to have molested some 200 boys at St John's School for the Deaf in St Francis, Wisconsin, between 1950 and 1974.
A canonical trial authorised by Cardinal Ratzinger's deputy was halted after Fr Murphy wrote to the future pope asking that proceedings be stopped, despite objections from a second archbishop.
The accused priest said in the letter that he was ill and wanted to live out the remainder of his time in the "dignity of my priesthood".
Victims say Fr Murphy - who died in 1998 - assaulted boys while hearing their confessions, in his office, his car, at his mother's house and in their dormitory beds.
He was quietly moved to the Diocese of Superior in northern Wisconsin in 1974, where he spent his last 24 years working freely with children in parishes and schools, according to one lawsuit.
Lawsuits have been filed on behalf of five men alleging the Archdiocese of Milwaukee in Wisconsin did not take sufficient action against the priest.
'Fall on the sword'
At a news conference on Thursday in Milwaukee, one of the victims, Arthur Budzinski, said Fr Murphy had begun to assault him when he was 12.
Neither the clerical authorities, nor the police had intervened when he reported it, the 61-year-old said.
Mr Budzinski was asked through a sign language interpreter what he wanted to see happen now.
"Ratzinger can have all of the colonels and lieutenants they want fall on the sword for him, but eventually he has to 'fess up," the interpreter said.
Meanwhile, members of a group of clerical abuse victims who denounced Benedict's handling of the case in a news conference outside the Vatican were briefly detained by Italian police for not having a permit.
The Vatican newspaper L'Osservatore Romano said there was no cover-up, denouncing the allegations as "clearly an ignoble attempt to strike at Pope Benedict and his closest aides at any cost".
The Pope's official spokesman, Federico Lombardi, called it a "tragic case", but said there was no provision in Church law for automatic punishment.
Fr Lawrence Murphy died in 1998 with no official blemish on his record
He noted that police did investigate the allegations at the time but did not press charges.
The papal spokesman said the Murphy case had only reached the Vatican in 1996 - two decades after the Milwaukee diocese first learned of the allegations and two years before the priest died.
The diocese was asked to take action by "restricting Father Murphy's public ministry and requiring that Father Murphy accept full responsibility for the gravity of his acts", he added.
Last week the Pope issued an unprecedented letter to Ireland addressing the 16 years of clerical cover-up scandals.
He has yet to comment on his handling of a child sex abuse case involving a German priest, which developed while Benedict was overseeing the Munich archdiocese.
The Rev Peter Hullermann had been accused of abusing boys when the now Pope approved his 1980 transfer to Munich to receive psychological treatment for paedophilia.
The disgraced priest was convicted in 1986 of abusing a youth, but stayed within the Church for another two decades.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.