Many voters accused the BJP of not doing enough in Uttar Pradesh
The BJP, which leads the national coalition government, has lost control of three states - Uttar Pradesh, where the chief minister swiftly resigned, Punjab and Uttaranchal.
Results from a fourth state, Manipur, have yet to be announced.
Correspondents say the defeat, especially in the most populous state of Uttar Pradesh, has dented the credibility of the Prime Minister, Atal Behari Vajpayee.
But senior party leaders tried to keep a brave face.
"The result will have no bearing on the stability of the government at the centre," said Information Technology Minister, Pramod Mahajan.
"Rather it make us stronger as our allies would be firmly behind us".
Hindu heartland
The Chief Minister of Uttar Pradesh, Rajnath Singh, has resigned following the defeat of his party.
The BJP had been in control of the state for the last five years - now it looks set to sit in opposition, having come second to a regional party, the Samajwadi Party.
Mr Singh said he wanted to resign to accept moral responsibility for his party's defeat and described the election results as totally unexpected.
Many in his party will share his sense of shock.
Uttar Pradesh is a key political asset - it sends the largest number of lawmakers to the parliament - and is often described as the Hindu heartland.
Some analysts said the BJP might have suffered from a strong anti-incumbency factor - UP is one of India's poorest states and many local people complained that the BJP failed to deliver the development they need.
Some right wing Hindu groups, traditionally staunch supporters of the BJP, have also been dissatisfied.
They accuse the party's leaders of being too moderate in office.
Rout in the hills
The BJP has been routed in the newly-created hill state of Uttaranchal.
The party lost most of its seats to the main opposition Congress party.
The party won 36, securing a simple majority in the 70-member house.
Uttaranchal was carved out of Uttar Pradesh two years ago.
The BJP, which governed the state since, finished second with 18 seats.
Prosperous Punjab
The Congress party has wrested control of wealthy Punjab state from the ruling alliance of the Shiromani Akali Dal and the BJP.
Final results are out for 116 of the 117 seats in the state assembly.
The Congress party has secured 62, enough to form a government on its own.
The party has already begun the exercise to chose a leader in the state.
A meeting of the newly-elected legislators is scheduled for Tuesday.