Outline plans have been approved for Dumfries and Galloway College's £32m move from Heathhall to become part of a "super campus" on the Crichton site.
The old college will be demolished and probably sold for housing
The flagship development will see it share facilities with the University of Glasgow, University of Paisley and Bell College at the same location.
Nithsdale planning councillors expressed serious concerns about the traffic impact of the plans.
They agreed outline permission with a series of conditions.
The Scottish Funding Council recently approved more than £21m funding for the project.
Once completed, the new campus will be the first of its kind in Scotland with higher and further education on one site.
College principal Tony Jakimciw said the move was the only option if the facility hoped to survive.
"Our current Heathhall location is no longer fit for purpose," he said.
"It would cost £20m to make it fit for purpose and it is money that we as a college do not have.
"The Scottish Funding Council would not fund a refurbishment of the existing site.
"This is a once-in-a-lifetime opportunity for Dumfries and Galloway College, the Crichton and Dumfries and Galloway," he added.
He said it would allow the Crichton to reach the "critical mass" of student numbers it needed to develop facilities and courses.
Mr Jakimciw also outlined options for the current site at Heathhall if the plans go ahead.
The old building would be demolished, probably to make way for houses.
He also promised that the Catherinefield Nursery would remain on the site and to provide land for playing fields for a new school proposed in the area.
Councillor for the Crichton area Billy Little expressed a number of concerns about the plans.
He said many people needed to be convinced of the benefits of the development because of its traffic impact.
"They are very sceptical about it at the moment," he warned.
"If we almost double the amount of cars we have on it, where on earth are these traffic queues going to end?"
The committee agreed to approve the plans with a number of requests submitted to the authority's planning and environment committee.
They will seek roads investment, an updated traffic management plan and a review of further building developments around the Crichton.
The project is targeted for completion in time for the beginning of the academic year in August 2008.