Reports of Mr Blunkett's relationship first appeared in August
An official inquiry has found a "chain of events" linking David Blunkett to the speeding up of a visa application by his ex-lover Kimberly Quinn's nanny.
Former Treasury adviser Sir Alan Budd said the application for leave to remain in the UK was processed in 52 days - 120 days faster than average.
In a separate report, Mr Blunkett was criticised by MPs for giving Mrs Quinn a rail pass meant for MPs' spouses.
Tory Michael Howard said they showed Tony Blair led a "grubby government".
'Whitewash' claim rejected
Mr Howard said the reports revealed a government that was "low on integrity, light on honesty and lacking in humility".
He called for a "judge-led inquiry" into the visa affair as there was a "gaping hole" in the evidence and he accused Mr Blunkett of lying when he originally denied fast-tracking the application.
The Home Office asked Sir Alan to hold an inquiry into how the visa application for Mrs Quinn's nanny, Leoncia Casalme, was handled after newspaper allegations about Mr Blunkett's conduct.
His report says: "I believe I have been able to establish a chain of events linking Mr Blunkett to the change in the decision on Mrs Casalme's application."
But he said it was "not a straightforward matter", because few involved in it could recall the details.
The report says: "I believe there are two broad possibilities: Mr Blunkett was seeking special help for Mrs Quinn's nanny (or) he was raising the case as an example of the poor performance of the Immigration and Nationality Directorate (IND).
"I do not have direct evidence that allows me to choose between the two possibilities."
Sir Alan said he thought there had been no deliberate attempt to conceal or destroy evidence - even though a key fax from Mr Blunkett's office to IND had never been found.
KEY E-MAILS - MAY 2003
"Just wondering if you have any update on the settlement (domestic worker) case I faxed through to you?"
Mr Blunkett's private secretary e-mails the Immigration and Nationality Directorate director general
"Sorted - she has been granted ILR - papers will be sent to her shortly. The case was in ICU so they pulled it out of the queue and made a decision - (no special favours, only what they would normally do - but a bit quicker)"
The IND director general's private secretary's reply
Following the report's publication, he told reporters: "I have been unable to link Mr Blunkett to the sending of faxes to IND. There must have been such a link but I have been unable to discover what its nature was."
Mr Blunkett quit as home secretary last week after being told in advance of Budd's findings.
In a statement, the ex-home secretary said: "I want to make it clear that I fully accept the findings of Sir Alan's report, where his findings differ from my recollections this is simply due to failure on my part to recall details."
He said he had told the truth throughout and had raised the nanny case as an example of unacceptable backlogs in the system.
Matthew Taylor, for the Liberal Democrats, said: "It beggars belief that not a single person including the home secretary can remember what they did. All his team at the Home Office have had a collective failure of memory."
The chairman of the Committee on Standards in Public Life, Sir Alistair Graham, told BBC Newsnight there were still unanswered questions and the inquiry's "impossibly narrow terms of reference" had caused problems.
Home secretary Charles Clarke said there would be new rules drawn up on how ministers should handle cases brought to their attention.
Earlier, in a separate inquiry into Mr Blunkett's gift of an MP's spouse rail ticket to his ex-lover, the Commons standards and privileges committee said he should have been aware of the rules he had broken.
The committee agreed with the verdict of Sir Philip Mawer, the parliamentary standards commissioner, that the rules had been broken.
But officials were unable to find any record of a travel warrant being used for any journey between Doncaster and London King's Cross on or around 19 August, 2002.
A source on the standards committee said the MPs did not believe any further action was necessary as they accepted Mr Blunkett's apology and explanation.
Correction 19 February 2010: Following representations from Mr Blunkett, the headline on this article was changed from Blunkett 'did speed nanny's visa' to more precisely reflect the Budd report's findings.