The Tories have stepped up pressure on the government as new revelations about registered sex offenders being allowed to work in schools emerged.
Ms Kelly has been criticised for clearing teachers to work
Keith Stuart Hudson was on a teaching blacklist but cleared to work in girls' schools despite being convicted of importing indecent images of boys.
The Tories say parents are losing confidence in the system following a series of similar revelations.
The Department for Education has launched an urgent review.
The former Chief Inspector of Schools, Chris Woodhead, told BBC News 24 that Education Secretary Ruth Kelly should resign over the revelations.
Police caution: Given to people admitting minor offences - placed on sex offenders' register but not automatically barred from schools
Conviction: Defendant sentenced in court - goes on sex offenders' register and put automatically on education blacklist, List 99
"I mean a Secretary of State used to take responsibility, they didn't only talk about taking responsibility, they actually did, they went, they voted with their feet. They said no, the buck stops with me, I'm going to go. So she should."
Chancellor Gordon Brown told Sky News parents were "rightly concerned", adding that it was "pretty clear we need to legislate changes and that is what we plan to do."
Those convicted of offences against children are placed on List 99 - a blacklist of those who are barred for life from working in schools.
But the Times reported that Keith Stuart Hudson was cleared to work as a teacher despite being on List 99 following a conviction for importing indecent and obscene publications after compiling albums of explicit images of boys.
Former Education Secretary Estelle Morris placed him on List 99 in 2001 with the condition that he could teach only in all-girl schools.
Mr Hudson, a science teacher who qualified in 1973, unsuccessfully attempted to challenge that restriction in a hearing before the Care Standards Tribunal.
The tribunal, set up in 1999 to look at cases involving work with children, also deals with appeals against the decisions of the Education Secretary.
In proceedings summarised on the internet, the tribunal heard medical evidence that the science teacher's interest in young boys was "homosexual, paedophilic and inappropriate", but that Ms Morris's decision was justified because he had "no interest in girls".
The Times claimed that after the tribunal upheld the decision Mr Hudson set himself up as a private tutor, giving him potential access to boys.
In response the Department for Education said: "In 2000 we introduced new regulations to make sure that barring somebody convicted of sexual offences against a child was absolute.
"Clearly there is a difference between decisions made by the department and those made by an independent appeals tribunal but we recognise we need to strengthen the decision making process still further and that is what we will do."
The latest case follows revelations about two other teachers, one with a conviction for a child sex offence and another with a caution.
William Gibson, 59, was allowed to work as a teacher at schools in South Tyneside and Co Durham from 2003 to 2005 despite being convicted for an indecent assault on a child in 1980.
Two of those schools dismissed him when they found out about his conviction.
On Saturday it emerged that a sex offender had been working at a Bournemouth school and the BBC understands that that person was William Gibson.
Bournemouth Borough Council issued a statement confirming that an individual, who had been working at a local school as a supply teacher full time since the beginning of term, had been named in the media.
It said the teacher was employed by a supply agency and cleared to work after undergoing security checks.
The statement said: "Measures have been taken over the weekend to ensure that the teacher concerned remains away from the school indefinitely, with immediate effect, while we ascertain the full facts of the situation".
Tory education spokesman David Willetts told BBC Radio Four's Today programme it was particularly worrying that this case occurred after the release of the Bichard Report into how Soham murderer Ian Huntley was able to work in schools.
"You know, local authorities will not allow someone to start work as a taxi driver until they check them out with the Criminal Records Bureau, so how on earth could someone get to work for three months as a supply teacher," he said.
Mr Gibson's case emerged as Education Secretary Ruth Kelly was already facing criticism after her department allowed Paul Reeve to work as a PE teacher in Norfolk.
He was arrested in 2003 by Norfolk Police and received a police caution for accessing banned images of children on the internet.
Minister Kim Howells, who now works in the Foreign Office, admitted making the decision to clear him to teach, but said he had been told Mr Reeve "did not represent an ongoing threat to children".
The teacher would automatically have been banned from teaching if he had been convicted of a sex offence.
But because Mr Reeve had accepted a caution he was not placed on List 99 and was only stopped from working at the school when police alerted the school's headteacher.
On Thursday, Ms Kelly told MPs she wanted to close this loophole by ensuring cautions and convictions were "treated identically", with the "closest possible alignment" between the sex offenders register and List 99.