Local officials in the North Western Frontier province said several thousand men were gathering in Dir district in the semi-autonomous tribal areas bordering Afghanistan.
These are Pashtun-dominated regions where the fellow Pashtun of the Taleban are widely supported for tribal and ethnic reasons as much as their shared Islamic faith.
"The situation is very tense, but we are putting a security cordon in place. We're taking measures to stop them from marching into Afghanistan," Dir district official Ghulam Farooq told AFP news agency.
Call to arms
The militants rallied to the call of a radical Islamic group, Tanzim Nifaz Shariat-i-Mohammadi, TSNM, which has been trying to enforce Islamic Sharia law in the area.
Many of the volunteers brought their own guns, swords and axes, with around 2,500 said to be carrying automatic weapons.
The TSNM leader, Maulana Sufi Mohammed, also said they would send a delegation of clerics to meet the Taleban, who, he said, had welcomed their offer of support.
Previously the Taleban have suggested they do not need reinforcements from outside, possibly because many such volunteers are untrained.
The volunteers say they plan to go in through Afghanistan's eastern Kunar province.
But the BBC's Susannah Price in Islamabad says Pakistan, which backs the United States-led coalition, is unlikely to allow the volunteers to leave, and the volunteers will have to find other ways to cross the long and porous border.
Pakistan does, however, want Afghanistan's biggest ethnic group, the Pashtuns, to have a central role in any future government in the country - a move that would be crucial for stability in its own Pashtun-dominated areas.