In a vote in the Stormont Assembly on Friday, his return to the post was blocked by opponents of the Good Friday Agreement.
It came four months after Mr Trimble resigned in protest at the refusal of the IRA to get rid of its weapons.
Failure to fill the top two Stormont posts means the power-sharing executive can no longer function and faces collapse at midnight on Saturday.
The British Government will now have to decide whether to call fresh elections in Northern Ireland or to consign devolution to another period of suspension.
Northern Ireland secretary John Reid is due to chair talks with the pro-Agreement parties at Castle buildings at 1600 GMT on Friday.
The leader of the Alliance Party, David Ford, confirmed he had talks with the British Prime Minster, Tony Blair, before Friday's debate.
Mr Ford would not say for certain whether his party would re-designate as unionists for voting purposes, stressing that his aim was to remove a system of designation.
Just over 70% of assembly members backed Mr Trimble, but under assembly rules he needed cross-community support.
He needed more than half the unionist votes to be re-elected, but of the 59 unionists who voted, only 49.2% supported Mr Trimble.
The UUP leader said two hardliners - Peter Weir and Pauline Armitage - who failed to support him in the crucial vote - had "behaved dishonorably".
However, Mr Trimble added: "One should not regard today's decision of being in any way final. It is not.
"This process is not ended by it, this process was from the outset remarkably robust and it will proceed."
The vote came more than a week after Mr Trimble re-instated his party's ministers at Stormont following the IRA move to put some weapons beyond use.
Meanwhile, the leader of the anti-Agreement Democratic Unionist Party, Ian Paisley, has called for fresh assembly elections.
'Disappointment'
Speaking after the result, he said the decision was a "bloody nose for all who took part in the charade of decommissioning".
Sinn Fein's Martin McGuinness said the result was a "great disappointment" to everyone, and called for assembly elections.
He added: "The opponents of the peace process have had their day, but they must not be permitted to have their way."
Mr Trimble's prospects of returning as first minister received a blow on Thursday when hardline UUP assembly member Pauline Armitage said she would not support his nomination.
Mrs Armitage said she had grave doubts about the IRA's gesture on decommissioning and the general direction of the peace process.
Earlier, the Women's Coalition succeeded in changing the rules of the assembly to allow two members to re-designate themselves in an attempt to give Mr Trimble at least one extra unionist vote.
The Progressive Unionist Party supported Mr Trimble, but the party's two votes were not enough to secure the future of the assembly.
The vote of another dissident Ulster Unionist, Peter Weir, could have decided the outcome of the devolved institutions.
Before the vote, senior Ulster Unionist Lord Kilclooney (John Taylor) said the party should "run with David Trimble" to ensure decommissioning continued.
"If we don't run with him today, decommissioning will stop, there will be a collapse in the political institutions in Northern Ireland, the economy will be damaged and I fear violence will begin once again," he said.