The vote by 142-100 against Chang Sang followed two days of bitter wrangling in the National Assembly over her suitability.
Ms Chang, the president of Seoul's elite Ewha Women's University, was nominated in a 11 July cabinet reshuffle by President Kim Dae-jung.
But the opposition Grand National Party (GNP) has been attacking her ever since, accusing her of lying over her property dealings and her academic background.
Correspondents say the rejection will be seen as a setback for the status of women in male-dominated South Korea.
Blow to women
The appointment of Ms Chang was greeted with some shock in South Korea, where few women reach the top of their professions.
National Assembly
259 members
128 in opposition GNP
125 in pro-government parties MDP and ULD
6 independents
Women's groups had hailed the news as an important step forward in promoting gender equality.
But parliamentary approval was required after the GNP questioned her suitability.
Two weeks ago, a top campaigner for opposition presidential candidate Lee Hoi-Chang resigned after being accused of making sexist remarks against Ms Chang.
Morality questioned
A two day televised confirmation hearing this week turned increasingly ugly.
The BBC's Caroline Gluck in Seoul says that while one or two lawmakers had cast aspersions on Ms Chang as a woman, the main sticking point seemed to be allegations which undermined her moral integrity.
Ms Chang has denied any wrongdoing.
The accusations:
While property speculation is not illegal in South Korea it is subject to high government taxation and raises ethical problems when high-profile leaders are involved.
Ms Chang said she and several university colleagues bought a piece of land outside Seoul to set up a nursing home for the elderly.
Ms Chang said her aides made a translation mistake, believing the two schools to be connected.
They also questioned why he qualified for free South Korean health care.
Ms Chang said her son, who was born in the US in 1973, chose US citizenship over South Korean citizenship in 1978.
In South Korea, the premier is far less powerful than the president. But the prime minister does take over the government if the president becomes incapacitated.
The vote against Ms Chang will be seen as a further blow to President Kim Dae-jung, who has already been tainted by a rash of scandals involving his sons and close aides.