At least 80,000 of those are now living in camps run by the UN refugee agency (UNHCR), based around the city of Mardan.
The BBC's Imogen Foulkes, in Geneva, says the sheer speed with which people have fled their homes in north-west Pakistan has taken even experienced UN aid workers by surprise.
Owen Bennett-Jones, Islamabad
There are currently three separate large-scale military offensives in the north-west of Pakistan. People leaving the affected areas are terrified by the scale of the army action.
Stories are circulating about the terrible decisions war imposes. One family in Swat for example is said to have left behind a son who could not be moved because he is a polio victim. They gave him a supply of bread and water and left him on his own as they headed for safety.
For the moment public opinion is with the army. Those who have actually lived under the Taleban held areas have been particularly disgusted by the floggings and the beheadings. But the generals know the public mood could change if there are too many civilian casualties.
Aid workers say some parts of the Swat valley have been emptied of their populations.
The UNHCR says the crisis could destabilise the entire region unless there is a massive response from the international community. It is expected to launch an emergency appeal for Pakistan in the next few days.
Pakistan's Prime Minister, Yousuf Raza Gilani, has described the situation as the country's worst refugee crisis since the bloody partition of India and Pakistan in 1947 at the end of colonial rule.
Speaking at the opening of a briefing session for key Pakistani political figures, Mr Gilani said Pakistan was "at the crossroads of its history", but conceded that the army operation in Swat could not be a "permanent solution".
The military commander in North West Frontier Province, Gen Tariq Khan, said the current military operations would probably be finished in about a month, but the process would continue elsewhere, turning into police operations by the end of the year.
The army has blockaded the town of Mingora, the administrative hub of the Swat valley.
The first wave of people left Mingora a week ago, and food and other vital supplies are now scarce inside the city, says the BBC's Barbara Plett in Islamabad.
But the army remains reluctant to engage in street-to-street combat while large numbers of civilians remain in Mingora, she adds, and authorities are keen to allow as many civilians as possible to leave the city.
After days under curfew and bitter fighting between government troops and rebels, buses were laid on to ferry to safety some of the 150,000 civilians thought to remain inside Mingora.
People have been getting used to new living conditions
But residents spoke of the need to walk 4km to the edge of the city to pick up transport, running the gauntlet between Taleban positions and army attacks.
One man, Iftikhar, told the BBC that Pakistani helicopter gunships fired on residential areas, killing civilians.
"When I was leaving I saw many bodies lying on the roadside," he said.
"Thousands of men, women and children are on foot trying to get out of Swat."
The road from the southern edge of the conflict zone towards Mandar was clogged with buses, trucks, motorbikes and rickshaws throughout Friday.
As the army pressed forward, it urged civilians to help them catch insurgents trying to flee the area.
It said some militants were disguising themselves by shaving off their beards and cutting their hair, and it gave a hotline number for people to call if they thought they had seen a Taleban fighter.
Up to 15,000 troops have now been deployed in the Swat valley and neighbouring areas to take on some 5,000 militants.
The army said on Friday that it killed 55 militants across Swat over the previous 24 hours, with three Pakistani security personnel also killed.
In a statement, the army also said a leading Taleban commander, Dawa Noor, identified as an instigator of the Taleban's recent move into Buner, south of Swat, was among those killed.
But a Taleban spokesman, Muslim Khan, says the militants have killed at least 37 soldiers in fighting since Wednesday, with just three of their fighters killed.
The BBC's M Ilyas Khan in Islamabad says independent confirmation of these claims is difficult as the phone system across Swat is down and tens of thousands of mobile phones have gone dead because of a lack of electricity.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.