He described the civil service as "honest, stuffed full of decent people who work hard".
But he added: "Frankly the job could be done with half as many, it could be more productive, more efficient, it could deliver a lot more value for money for the taxpayer.
"I was amazed, quite frankly, at how many people deserved the sack and yet that was the one threat that they never ever worked under, because it doesn't exist."
Lord Jones' period as a minister proved colourful and at times controversial and Jonathan Baume, general secretary of the senior civil servants' union the FDA, said he was "disappointed but not surprised" at the latest comments given the ex-minister's "traditionally maverick approach".
Mr Baume said the government's love of launching initiatives - and the current economic crisis - meant that there was an argument for more civil servants rather than less.
And he suggested that it might be better to halve the number of ministers - back to the levels that served the Churchill government well during the Second World War.
During his appearance before the public administration committee Lord Jones said the job of junior minister was "one of the most dehumanising and depersonalising experiences a human being can have. The whole system is designed to take the personality, the drive and the initiative out of a junior minister".
Lord Jones stepped down as a government minister in Gordon Brown's reshuffle in October, saying it was not a judgement on Mr Brown's performance.
His appointment in July 2007 was controversial at the time because he was not a member of the Labour Party.
He countered that criticism by saying that the job of promoting trade and investment "should transcend the factionalism of party politics".
Lord Jones has been involved in some controversy during his time in the job - notably when he expressed concerns about plans to tax "non-domiciled" foreigners in Britain.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.