The crunch session lasted for over three hours and saw Fisheries Minister Rhona Brankin ask the industry to come up with new proposals to address its problems.
But as fishing chiefs left the meeting in Edinburgh, on Tuesday night, they said they were disappointed the deadlock over short-term compensation payments had not been broken.
East coast skippers have stayed in port because they claim an European plan to conserve cod stocks - which has seen areas of the North Sea closed down - was forcing them to fish areas populated by immature haddock stocks.
They want compensation for that action, which they say is motivated by conservation concerns.
But the executive has refused, instead offering a multi-million pound package to cut the size of the Scottish fleet in the long-term.
The skippers warned - ahead of Tuesday's meeting - that without compensation they would be forced back to sea this weekend.
However, Ms Brankin presented the meeting with new scientific evidence to back her claim that a tie-up was not going to improve fish stocks.
As she emerged from the meeting she said the talks had been constructive.
Ms Brankin has asked the fishing leaders to meet their members and come up with new proposals which will then be discussed with her as a matter of urgency.
Scottish Tory Leader David McLetchie said: "I am deeply disappointed that these talks have failed to produce a commitment from the executive to fund a tie-up scheme as part of the overall package of support for the industry.
"This does not meet the short-term needs of our fishermen or reflect the will of parliament as expressed in last Thursday's vote."
Mr McLetchie added: "Henry McLeish is trying to brazen it out, and in doing so he does our fishermen and the parliament a profound disservice. He should hang his head in shame."
SNP fishing spokesman Richard Lochhead said: "Yet again, our fishermen have met ministers only for their pleas to fall on deaf ears.
"Rather than deliver the tie-up scheme supported by Scotland's Parliament, they have chosen to force fishermen to stay in port and face bankruptcy or return to sea and slaughter juvenile haddock stocks."
He added: "Our fishing communities, environmentalists, independent scientists, and Scotland's Parliament backed the campaign for a tie-up scheme, but Rhona Brankin thinks she knows best."
Ahead of the ministerial meeting John Buchan, of the Fishermen's Action Committee, said: "If Rhona Brankin is so assured that technical measures are the answer, if she likes she can have the services of my boat for two weeks, it will cost her nothing.
"If at the end of these two weeks she can still say that technical measures will work then I will stand up and say that I was wrong."
Mr Buchan said around 25%, 200 of the Scottish fleet, should be paid to tie-up until the end of April, costing in the region of £5m.