A judge should look into the David Blunkett controversy as key questions remain unanswered, the Lib Dems say.
Mr Oaten says public confidence must be restored
Sir Alan Budd's inquiry linked the ex-home secretary to the speeding up of a visa claim by his ex-lover's nanny.
But he could not say whether Mr Blunkett had treated the nanny as a special case or had used her as an example of immigration problems.
Lib Dem spokesman Mark Oaten said the number of officials who had forgotten what happened was worrying.
He told BBC News: "I'm extremely concerned that 20 individuals appear to have forgotten what happened or more suspiciously are not prepared to say what happened.
"That must be bad for government, it must be bad for public confidence in the system."
Mr Oaten said people would be mystified why Sir Alan's inquiry failed to uncover why processing of the visa application was speeded up.
He said: "There is a strong case for a judge-led judicial review of this because, of course, next time it happens it may not be about a nanny and their visa. It may be about something even more important than that.
"So we do need to understand what took place and who was involved."
The Tories criticised the Lib Dem stance, saying Mr Oaten had initially said he accepted Mr Blunkett's word unless further evidence emerged.
But Lib Dem officials say Mr Oaten was speaking when an inquiry was under way and was avoiding rushing to judgement.