A close ally of Tory leader Iain Duncan Smith has urged his party to dump the language of compassion and to follow Labour's example in opposition by becoming more "relentless and destructive".
Mr Legg, who was chief executive of the Conservative Party for three months after being appointed by Iain Duncan Smith, said the party should be "shooting real bullets" in opposition, and said that at the moment "too often the party has not got a point of view".
Barry Legg: Party needs to "dare greatly"
He told a fringe meeting organised by the right-wing Freedom Association: "Any idea that opposition is for nice people has not got any basis in fact. If you want to be nice in politics you have to go out the door."
Mr Legg, who was also the Tory leader's chief of staff, also urged the Tories not to use "the language of the left", saying compassion is not for governments - and he refused to comment on Mr Duncan Smith's leadership, saying he would not be drawn into a discussion on personalities.
Mr Legg's comments come the day after Mr Duncan Smith told a reception for Tory MEPs: "If you want to shoot the enemy you need bullets and that's what this conference is doing - providing the bullets to cut them down."
The former MP, who left his job as Tory chief executive in May after just three months in the job amid disquiet over Mr Duncan Smith's failure to consult the Conservative Party board over the appointment, urged the Tories to "look at the language we use".
Mr Legg's appointment to Mr Duncan Smith's team in February had been seen as a snub to modernisers seeking a more socially liberal approach in the party's policies.
And his comments will be seen as a swipe at those attempting to modernise the Conservatives.
There have been attempts in recent months to portray the Tories as offering "compassionate Conservatism", while last year chairman Theresa May said the party had to shed its image as the "nasty party".
But Mr Legg said: "If we use the language of the left we start then to be fighting the battle on their ground.
"I believe compassion is a very fine virtue and characteristic for an individual, and many in the Conservative Party show their compassion by working for voluntary organisations and helping people in need.
"But governments should not express compassion. The amount of pain is unlimited - governments cannot create happiness and if you pursue the agenda of compassion in government you are pursuing unlimited government.
"We in the Conservative party believe in limited
government, limiting the role of the state."
He went on: "Tolerance is also a very fine virtue. I am pleased that we live in a tolerant country, but tolerance as a virtue is not the same as neutrality.
"If we are tolerant we can still make judgements. We can still disagree with the other man's point of view.
"All too often the Conservative Party has not got a point of view and that's one of the problems we have at the moment and why people find it difficult to relate to us."
Mr Legg told the meeting that the Tories would not be able to "outcharm" Tony Blair, but added: "We may out punch him.
"Labour were relentless and destructive in opposition. And they got personal."
He called on the Tories to be more aggressive in attacking the proposed constitution for the European Union and to develop "a coherent strategy" on Europe.
He said he was "amazed" that Tory spokesman had not succeeded in getting what he suggested was Labour's real agenda over the euro - a wealth tax on housing, capital gains tax on housing and higher stamp duty - into the open.
Conservative spokesman "should be running with the ball", he said.
He concluded the meeting by saying: "The price of daring greatly - and I think the party needs to dare greatly - is the certainty of powerful opposition.
"In the past sometimes we have had Conservative leaders who have rescued our country and I believe we will have that day again when a Conservative leader rescues the country."
Asked if he supported Mr Duncan Smith he said: "I have not come here to talk about personalities."
He added later: "When we have great difficulties (in the country) we normally have a leader to seize the opportunity."