Rights groups have long campaigned for a repeal of the law
A court in the Indian capital, Delhi, has ruled that homosexual intercourse between consenting adults is not a criminal act.
The ruling overturns a 148-year-old colonial law which describes a same-sex relationship as an "unnatural offence".
Homosexual acts were punishable by a 10-year prison sentence.
Many people in India regard same-sex relationships as illegitimate. Rights groups have long argued that the law contravened human rights.
Delhi's High Court ruled that the law outlawing homosexual acts was discriminatory and a "violation of fundamental rights".
The court said that a statute in Section 377 of the Indian Penal Code, which defines homosexual acts as "carnal intercourse against the order of nature" and made them illegal, was an "antithesis of the right to equality".
The ruling is historic in a country where homosexuals face discrimination and persecution on a daily basis but it is likely to be challenged, says the BBC's Soutik Biswas in Delhi.
It also promises to change the discourse on sexuality in a largely conservative country, where even talking about sex is largely taboo, our correspondent says.
Gay rights activists all over the country welcomed the ruling and said it was "India's Stonewall".
New York's Stonewall riot in 1969 is credited with launching the gay rights movement.
"It [the ruling] is India's Stonewall. We are elated. I think what now happens is that a lot of our fundamental rights and civic rights which were denied to us can now be reclaimed by us," activist and lawyer Aditya Bandopadhyay told the BBC.
"It is a fabulously written judgement, and it restores our faith in the judiciary," he said.
Leading gay rights activist and the editor of India's first gay magazine Ashok Row Kavi welcomed the judgement but said the stigma against homosexuals will persist.
"The social stigma will remain. It is [still] a long struggle. But the ruling will help in HIV prevention. Gay men can now visit doctors and talk about their problems. It will help in preventing harassment at police stations," Mr Kavi told the BBC.
But the decision was greeted with unease by other groups.
Father Dominic Emanuel of India's Catholic Bishop Council said the church did not "approve" of homosexual behaviour.
"Our stand has always been very clear. The church has no serious objection to decriminalising homosexuality between consenting adults, the church has never considered homosexuals as criminals," said Father Emanuel.
"But the church does not approve of this behaviour. It doesn't consider it natural, ethical, or moral," he said.
The head cleric of Jama Masjid, India's largest mosque, criticised the ruling.
"This is absolutely wrong. We will not accept any such law," Ahmed Bukhari told the AFP news agency.
In 2004, the Indian government opposed a legal petition that sought to legalise homosexuality - a petition the high court in Delhi dismissed.
But rights groups and the Indian government's HIV/Aids control body have demanded that homosexuality be legalised.
The National Aids Control Organisation (Naco) has said that infected people were being driven underground and efforts to curb the virus were being hampered.
According to one estimate, more than 8% of homosexual men in India were infected with HIV, compared to fewer than 1% in the general population.