Peter Hain has come under fire from MPs as alleged "incompetence" over his donations is linked to his running of the Department for Work and Pensions.
Mr Hain is on the rack over donations to his deputy leadership campaign
Conservative spokesman Chris Grayling said it "all appears to be getting a bit much" for Mr Hain.
But he was warned to stick to the subject of the debate, illegal immigration, by Speaker Michael Martin.
Mr Hain said Mr Grayling was "far more interested in making political points" than dealing with illegal workers.
The Conservatives switched the subject of the Commons debate from pensioner poverty to illegal immigration earlier on Wednesday.
'Tip of iceberg'
It follows claims 6,500 illegal immigrants employed as security guards were given National Insurance numbers.
Mr Grayling said this was the "tip of the iceberg" and he accused the government of handing National Insurance numbers to "tens and possibly hundreds of thousands" of illegal immigrants.
"Some people might call that incompetence," said the Tory work and pensions spokesman.
"Just when the secretary of state thought that things could not get any worse, the chaos spreads to another part of his job."
At the same time, Mr Grayling added Mr Hain "could not sort out the confusion with his campaign finances because he was too busy getting on with his two jobs in government - what a state of utter chaos".
He said Mr Hain had been responsible for a "whole series of calamities" including a "fiasco" over migrant worker figures and said "keeping things private has become a bit of a habit" for Mr Hain.
"Can you now tell us why you believe you are still the right man to do the job?," Mr Grayling asked.
Mr Hain accused Mr Grayling of framing the debate under a "false premise".
He said the government had introduced new "checks and controls" on the issuing of national insurance numbers to foreign workers.
He repeatedly declined to answer questions from Tory backbenchers referring to the row over donations to his deputy leadership campaign and allegations of "incompetence," saying he "would not dignify" them with a response.
Labour MP Paul Flynn accused Mr Grayling of pursuing a "very nasty vindictive campaign against the minister".
The Conservatives also say that work permits were issued to 270,000 non-EU migrants between 2004 and 2007.
They say during the same period 900,000 were given NI numbers - indicating that more than 600,000 had NI numbers despite apparently not having the right to work in the UK.
But Mr Hain said Mr Grayling did not know what he was talking about and not everyone given an NI number had to have a work permit - for example foreign students.
The minister said possession of an NI number was no longer proof of right to work and it was "positively irresponsible" of Mr Grayling to say in interviews that it was.
He said all employment-related NI applicants now had to prove their had the right to work and the system would be tightened further with the introduction of ID cards.
He said the shadow minister would be "embarrassed" by his own arguments and needed a "proper education" on the subject.
For the Liberal Democrats, Danny Alexander said the government needed to do "much more to ensure there is confidence" in the system.
"That requires competence, including from the Secretary of State and his ministerial team," he said.
Prime Minister Gordon Brown, in an interview with ITN on Tuesday, said Mr Hain had made a "mistake" over the failure to declare donations to his deputy leadership campaign.
"It was an incompetence that he has readily admitted to," added Mr Brown.
"This now goes before the standards committee in the House of Commons, and before the Electoral Commission.
"And I believe they will understand that this was a failure, that there was no corruption involved, that there was no illegal donation made, and I hope that they will be able to accept his apology."