In a written statement he said: "We are talking about political fraud and blatant stealing of votes... This is an offer which must be rejected."
The opposition have rejected as electoral "fraud" the decision by the Federal Electoral Commission to hold a second round in the country's presidential elections.
But Mr Kostunica condemned the move, insisting that he won Sunday's vote outright.
In a written statement he said: "We are talking about political fraud and blatant stealing of votes... This is an offer which must be rejected."
US President Bill Clinton said Mr Milosevic appeared to have lost his "last vestige of legitimacy".
He offered to lift sanctions imposed on Belgrade if what he called the will of the people was respected.
Non-violent protests
The electoral commission said Mr Kostunica polled 48% of votes to Mr Milosevic's 40%, but neither candidate got the necessary 50%, so a second round run-off would be held within two weeks.
But the main opposition alliance said it would call its supporters onto the streets on Wednesday unless the authorities accepted Mr Milosevic's defeat.
"We will defend the election results by all non-violent methods. We want to believe that common sense will prevail on the other side too," Mr Kostunica said.
In Montenegro, where the pro-Western administration urged voters to boycott the election, Deputy Prime Minister Dragisa Burzan told the Reuters news agency: "The opposition would be making a huge, huge mistake by accepting [the run-off]."
'Outright victory'
The spokesman for the Democratic Opposition of Serbia (DOS) coalition announced earlier on Tuesday that with 97.5% of the votes counted, Mr Kostunica had won outright with 54.7% of the vote. According to the DOS, Mr Milosevic trails with 35% of the vote.
There was immediate reaction in Washington to the announcement of a second round.
"The failure to award a first round victory to the opposition is highly suspect given the other credible reports that came out about the polling," White House spokesman Joe Lockhart said.
"I think now we have to wait until these numbers are final and take a cue from the opposition to see what their intentions and plans are," he added.
Pressure
Earlier on Tuesday, Western governments continued to pressure Mr Milosevic to concede victory to the opposition.
"Whatever the authorities in Belgrade may say, it seems now undeniable that Slobodan Milosevic has lost and Vojislav Kostunica has won," said Hubert Vedrine, foreign minister of France, which currently holds the EU presidency.
Chris Patten, the EU Commissioner for Foreign Relations, went further.
"I hope the question of Mr Milosevic's survival doesn't arise," Mr Patten said. "It certainly shouldn't because people in Serbia clearly want to get shot of him, the sooner the better."
UK Foreign Secretary Robin Cook earlier warned Mr Milosevic not to use violence to hang on to power, pointing out that Western powers maintained a significant military presence in Europe.