Dartington College of Arts could move to Cornwall and merge with the University College Falmouth.
Totnes could suffer if students move says a councillor
Dartington is recognised internationally as a premier institute for contemporary performing arts and
has 680 students.
But the college said it was attracted by the prospect of tapping into European cash support which is more widely available in Cornwall.
The move has raised concerns in Totnes, home of the college since 1961.
Dartington principal, Professor Andrew Brewerton said: "The decision to move is inevitable given that it is beyond the means of a small, private, charitable institution to guarantee our long-term sustainability."
Consultants are now looking into the feasibility of the move.
The move, which has been discussed with the Higher Education Funding Council for England, would take place by 2009 at the earliest.
Prof Brewerton said: "The colleges are an almost perfect academic fit.
"There is a strong academic vision driving this, which is to produce one of the most exciting centres in educational and artistic terms outside London.
"We want to achieve that through funding that may be available through European funding in Cornwall."
He said that artistic activities on the 850-acre Dartington estate would continue.
"I cannot foresee a point in the future when there will not be a year-round artistic programme at Dartington," Prof Brewerton said.
Professor Alan Livingston, principal of University College Falmouth which has 2,200 students, said: "We are delighted to be working with Dartington on this exciting merger and hope to satisfactorily conclude our discussions shortly.
Local councillor John MacAdie said: "I can see the town suffering.
"Students buy and rent properties, use restaurants and bring people to the area.
"I can't see how we can replace the cultural aspect of young students leaving the area.
The college is part of the Dartington Estate which was bought in 1925 by Leonard and Dorothy Elmhirst as a centre for an experiment in rural regeneration.