The government has pledged to cut class sizes
The teachers' union has urged ministers to cut class sizes across the board, warning of "problems" when pupils move from smaller to larger classes.
The SNP has pledged to cut class sizes for primaries one to three to 18.
But the Educational Institute of Scotland (EIS) has said smaller classes sizes need to be continued through schools.
The union is calling for class sizes to be brought down over time to a maximum of 20 in primaries and secondaries.
It has submitted a petition, which was was signed by almost 80,000 people, calling for "significant reductions" in class sizes over the next four years, to the Scottish Parliament.
Members of Holyrood's education, lifelong learning and culture committee heard evidence on Friday to suggest there was "significant" evidence that cutting class sizes benefited students.
EIS vice president David Drever welcomed the Scottish Government's target for reducing class sizes in the first three years of school.
But he said members wanted to see the work being undertaken in primary one to primary three extended to cover all primary classes.
He said: "There is evidence of setback in attainment where youngsters have moved from a small class size situation into a much larger one.
"Obviously a time will come when these primary three youngsters, who are sitting in a class of less than 18, will move potentially to a class of 30.
"That's going to cause teaching and learning problems and social disruption with social groupings that have been set up in these classes."
He added: "We want to see a smoothing out in the direction of smaller class sizes beyond primary one to primary three."
EIS vice president elect Helen Connor added that when pupils moved from a small primary class into a much bigger one "you actually lose any benefit you have gained fairly quickly".
She added: "If you have smaller classes at the early stages then you really need to have a staged reduction and make sure that continues through."
A paper submitted to MSPs from experts at SCRE Centre - an educational research centre at Glasgow University - stated that smaller classes "do seem to impact on pupils' attainment".
It added that such a policy was "costly" to implement and warned other factors, such as the quality of teaching, could also have an impact.
EIS general secretary Ronnie Smith conceded there was a "variety of opinion" on the issue.
However, he said: "I don't think there is any disagreement, however, that reduced class sizes are desirable.
"I have certainly never heard anyone argue for increased class sizes."
Glasgow University researchers found that work conducted in the US could not distinguish between class sizes of 15 and 20.
In addition, research in England found that there was little apparent improvement in performance between class sizes of 18 to 25.
The researchers found that other factors were probably more important such as the quality of teaching and the home background of pupils.