Minister Tony McNulty has apologised for not knowing sooner about a Home Office failure to update police records of Britons convicted abroad.
Mr McNulty faces pressure from the Tories
But he said a letter detailing police officers' concerns about inadequate information should not be published until an inquiry had been completed.
Both the Tories and Lib Dems have ridiculed the decision not to publish the letter, which was sent in October.
The issue concerns 27,500 files which were not logged on a police computer.
'Should not have happened'
Mr McNulty told BBC Radio 4's Any Questions: "I apologise for the failures that there have been in terms of ministers not knowing about the backlog.
"I think that's right, and [it] should not have happened in public policy terms."
SOME OF THE CASES
3 attempted rapes
17 other sex offenders
9 attempted murders
Asked whether he could say for sure that no files containing details of offences committed by Britons outside Europe still resided in boxes in the Home Office, Mr McNulty said: ''I can't, no, to be perfectly frank."
The 27,500 figure was calculated by the Association of Chief Police Officers (Acpo) when it took over responsibility for processing the data from the Home Office in March last year, but emerged publicly only earlier this week.
Home Secretary John Reid has said he was unaware of the backlog.
The episode has prompted fears that people who committed crimes abroad may have been cleared to work with children on their return to the UK, as their files had not been updated.
Of the 540 most serious cases, which include rapes, murders and paedophilia, 260 have now been entered onto the police national computer.
Mr Reid has said another 280 cannot be entered yet, because there were not enough details.
The October letter apparently warned the Home Office that police were having problems tracing hundreds of British criminals convicted overseas - although Acpo says it did not mention the backlog.
The minister said he had been sent the Acpo letter in October but had seen it only two days ago following Acpo's public acknowledgement of its existence.
He said: "It was passed, as is totally normal if it's not someone's area, on to [Joan Ryan, minister responsible for the Criminal Record Bureau], who answered and answered it in an entirely appropriate fashion."
It has been suggested the letter said the home secretary should be appraised of the situation, but Ms Ryan decided not to tell him.
'Publish the letter'
Shadow home secretary David Davis has said the position of Mr McNulty and Ms Ryan would be "untenable" if they had known of the problems with the offenders' details.
The Lib Dems' Norman Lamb said: "What possible reason is there to delay publication [of the Acpo letter] until the end of the inquiry? If they have nothing to hide then publish the letter."
Mr Davis has written to the Foreign Office to clarify whether or not it holds the information needed to identify Britons who have committed crimes abroad.
It comes after ministers said the Home Office could not enter details of some offenders on to the police database because information received from other countries had been incomplete.
There are concerns about the flow of information from countries outside Europe, by Mr McNulty also said there were problems with some European countries.
"As I understand it, for example, there have been since last May no notifications of any UK citizen causing any crime at all in Spain or Greece. And just intuitively that doesn't sound right."