Chief Treasury Secretary David Laws has apologised after it emerged he had been claiming MPs' expenses to rent rooms in homes owned by his partner.
Mr Laws said he would immediately pay back the money which the Daily Telegraph said totalled £40,000.
He said his motivation was to keep the relationship with the man private and not to reveal his own sexuality.
David Cameron said he agreed with Mr Laws's decision to refer himself to the Parliamentary Standards Commissioner.
Since 2006 parliamentary rules have banned MPs from "leasing accommodation from a partner".
The Liberal Democrat cabinet minister claimed up to £950 a month for eight years to rent a room in two properties owned by his partner, James Lundie, according to the Daily Telegraph.
Mr Laws said he claimed back the costs of sharing a home in Kennington, south London, with Mr Lundie from 2001 to June 2007.
The Yeovil MP said his partner bought a new home, in London, in June 2007 and he continued to claim back his share of the costs until August 2009.
The minister said he extended the mortgage on his Somerset property to help Mr Lundie purchase the new property.
Mr Laws said: "At no point did I consider myself to be in breach of the rules which in 2009 defined partner as 'one of a couple who although not married to each other or civil partners are living together and treat each other as spouses'.
"Although we were living together we did not treat each other as spouses - for example we do not share bank accounts and indeed have separate social lives.
"However, I now accept that this was open to interpretation and will immediately pay back the costs of the rent and other housing costs I claimed from the time the rules changed until August 2009."
He added: "My motivation throughout has not been to maximise profit but to simply protect our privacy and my wish not to reveal my sexuality.
"I regret this situation deeply, accept that I should not have claimed my expenses in this way and apologise fully."
BBC political correspondent Norman Smith told BBC Radio 4's Today programme Mr Laws's future would be decided through "hard-headed politics" whereby it will have to be determined whether his position is now "untenable" because he is "no longer able to be the front man for the government's efforts to reduce the deficit".
A spokesman for Prime Minister David Cameron said: "The prime minister has been made aware of this situation and he agrees with David Laws's decision to self-refer to the Parliamentary Standards Commissioner."
And Labour MP Alan Whitehead, a member of the standards and privileges committee in the old parliament, said Mr Laws's position would become "very difficult" if it was decided that serious breaches of the rules had been made.
Meanwhile, Sir Alistair Graham, the former chairman of the Committee on Standards in Public Life, voiced his surprise the details had only just emerged.
He told the BBC: "I'm a genuinely shocked that somebody who is now chief secretary to the Treasury is faced with disclosure of this nature where he clearly hasn't told the full truth to the people dealing with expenses in the House of Commons.
"Given all the expenses farrago that has gone on over the past two or three years, the fact that it has come to light now when he is a key part of a coalition government is staggering really."
Mr Laws's parliamentary colleague, the Liberal Democrat MP for Taunton Deane Jeremy Browne, said the matter was "a human story, not a financial story" about a "deeply private man".
"This is not about David being motivated by money," he said.
Mr Laws was one of five Liberal Democrats named in Mr Cameron's coalition cabinet following the election, working with Chancellor George Osborne at the Treasury.
A press release on Mr Laws's website, dated 18 June last year, said he had not "gained" through the taxpayer from buying a property because he rented accommodation in London.
The release said: "Recent publicity means that there is particular interest in which homes MPs have designated as their second homes, how much they have spent on these, and what items have been purchased.
"I have rented a property in London, rather than buying one, so I have made no gain from buying a property with help from the taxpayer."
In an interview with the Times newspaper, carried out on Thursday before the revelations emerged, Mr Laws had said he was single.
But, speaking to the newspaper after details of his relationship emerged, he said: "When I grew up, being gay was not accepted by most people, including many of my friends.
"So I have kept this secret from everyone I know for every day of my life. That has not been easy, and in some ways it is a relief not to have to go on misleading those close to me about who I am."