Home Secretary David Blunkett has repaid Parliament for the train ticket he gave to ex-lover Kimberly Quinn and apologised for his "genuine mistake".
Mr Blunkett denies the visa allegation
The rail warrants are intended for MPs' spouses and Mr Blunkett's spokesman said he had now realised his error.
And he would be repaying the cost of the £180 ticket to Doncaster.
There is a separate inquiry into whether Mr Blunkett abused his position by intervening in a visa application for Mrs Quinn's nanny.
Mrs Quinn, who is seven months pregnant, was admitted to hospital on Monday night amid the stress caused by the controversy.
Her husband, Stephen, said the problems were not serious but he did not know when she would be discharged.
Spectator magazine publisher Mrs Quinn was the home secretary's lover at the time she was given the disputed rail ticket.
Mr Blunkett had previously said he thought he was entitled to claim for a partner's travel.
But on Tuesday, his spokesman said: "David has always said that if he
had breached parliamentary rules over issuing a return ticket for spouses to
Kimberly Fortier (Mrs Quinn's maiden name) he would be willing to reimburse the House.
"Having examined the detailed rules today he realises he has made a genuine
mistake and will be repaying the cost of the ticket to the parliamentary
authorities and apologise for his mistake."
The move comes after a member of the public complained to Parliament's standards watchdog Sir Philip Mawer about improper use of the ticket.
Sir Philip said he would look at the evidence before deciding whether to investigate.
The Conservatives still say the use of the ticket and other allegations should be part of the inquiry headed by ex-civil servant Sir Alan Budd into the visa claims.
Downing Street denies suggestions the scope of that inquiry is too narrow, saying the Home Office would look at less serious issues.
Tony Blair's official spokesman also rejected criticisms from the chairman of the Committee on Standards in Public Life, Sir Alistair Graham, over the prime minister's prediction that Mr Blunkett would be exonerated.
Mr Blunkett pictured with Mrs Quinn at a function last year
The spokesman said that by giving Mr Blunkett his public backing, Mr Blair had been expressing trust and confidence in his home secretary - and not pre-judging the review.
Mr Blair intends to review the code that governs the conduct of ministers after the next general election but the government says that decision was taken some time ago and was not prompted by the Blunkett claims.
Mr Blunkett requested Sir Alan's independent review into claims he fast-tracked a visa application for permanent residency by Leoncia Casalme, who left Mrs Quinn's employment two months ago.
He is said to have checked the application was filled in correctly but not intervened to get it approved.
Other claims made in the Sunday Telegraph newspaper were that Mr Blunkett, 57, shared confidential security information with Mrs Quinn.
This included telling her parents to avoid Newark Airport near New York hours before a security scare.
It is also alleged that Mr Blunkett took her to Spain accompanied by bodyguards at taxpayers' expense.
Another claim is that he ordered his government chauffeur to drive Mrs Quinn between London and his Derbyshire home.
Mr Blunkett's spokesman says information about the New York security scare was already in the public domain.
And his bodyguards were already in Spain as part of his normal protection.
Mr Blunkett says Mrs Quinn was only given lifts on trips his chauffeur was already scheduled to make on government business.
He also dismissed as "nonsense" claims that he had asked for a policeman to be posted outside Mrs Quinn's central London home during a May Day riot.