Mr Sheerman said some of the allegations were 'shocking'
Teachers in England accused of misconduct should always be presumed innocent until proven guilty, MPs say.
A report by the Children, Schools and Families Committee said too many teachers were suspended needlessly and some have had their careers ruined.
The MPs said they were "shocked" the government let teachers be "unjustly" barred from contact with colleagues.
Ministers said the government would consider the report carefully, but added most cases were resolved quickly.
It is "unsatisfactory", the committee said, that no data was collected on how many teachers went on to be charged with any offence after being suspended.
Teaching unions say just 5% of allegations result in a charge of misconduct or a conviction.
The committee said head teachers were referring too many incidents to the local authority to investigate, leading to prolonged inquiries which could have been dealt with more quickly in school.
It recommended the number of allegations referred to local authorities and leading to a police investigation should be collected, as well as the number of allegations leading to suspension.
Outcomes should be recorded, including whether there were any charges or a dismissal.
In addition, schools should themselves investigate whether suspensions were justified, MPs said.
The MPs say schools must ensure key members of staff are trained in how to investigate allegations against staff, and ensure there is access to support, including a helpline.
On occasions investigations are "outsourced" to a third party which may have a particular viewpoint, or may be used simply to assemble a case against an individual, and this is unacceptable, their report said.
It criticised the government for allowing schools to prevent suspended teachers from contacting colleagues to build a defence, calling this "inhumane and unjust".
MPs have asked the government to examine again whether anonymity for a teacher accused of an allegation would be justified.
The committee chairman, Barry Sheerman MP, said: "There is a fine balance to be struck between safeguarding the rights of children and the rights of those who work with children.
"Allegations proven to be true must be punished.
"But the vast majority of complaints made against school staff have little or no foundation.
"My committee heard shocking evidence about the treatment of accused staff and the devastating impact unfounded allegations of misconduct can have on those involved, which can ruin careers and can come at a significant physical, mental and financial cost.
"I urge the government to take immediate action to ensure people's lives are not ruined by a failure to deal appropriately, sensitively, and quickly with complaints when they are made."
The NSPCC told the committee that in 2007-08, Childline received 68,758 calls about abuse and bullying.
It said for 1,491 of the children counselled, a teacher was identified as the perpetrator of abuse.
Teaching unions indicated to MPs that allegations of physical violence had been rising in recent years.
Children's Minister Delyth Morgan said: "False or misleading accusations against teachers or other staff are completely unacceptable.
"They run the risk of discrediting hard-working and committed teachers and support staff who work tirelessly to improve the lives of young people.
"False allegations also risk undermining genuine allegations.
"Children making false allegations often have complex needs, and in some cases may be trying to draw attention to abuse or other issues elsewhere in their life rather than being deliberately malicious."
She added that 64% of allegations were resolved within one month, but that this figure should increase.
Are you a teacher? Does the system for investigating allegations against teachers need to be changed? Send us your comments using the form below:
A selection of your comments may be published, displaying your name and location unless you state otherwise in the box below.
The BBC may edit your comments and not all emails will be published. Your comments may be published on any BBC media worldwide.