Oil firms and African states have been accused of "contracting out" of their human rights obligations in Africa's biggest investment project.
Thousands of farmers depend on land in the pipeline zone
The Chad-Cameroon oil pipeline contract may impair the countries' ability to protect farmers, fishermen and others affected, Amnesty International says.
The governments face cash penalties if they interfere with the ExxonMobil-led project, according to Amnesty.
The oil firm says it has not seen the report, but condemns rights violations.
It also insists that formal mechanisms exist whereby credible allegations of potential wrong-doing are taken to the government of Chad - with a request that they be investigated.
But Amnesty claims in its report; Contracting out of human rights: The Chad-Cameroon Pipeline Project, that the legal framework for the £2.6bn World Bank-backed scheme, means the oil company is "de facto unaccountable in the pipeline zone".
Both Chad and Cameroon are obliged to regulate the conduct of third parties, including companies, to ensure they the respect human rights of the thousands of farmers and fishermen living in the zone being developed.
But as they face financial penalties, even if they intervene to protect rights and enforce laws that apply elsewhere in their countries, Amnesty says it would make it very difficult for governments to proceed against the consortium for malpractice, for example.
The oil pipeline is supported by $200m in World Bank loans
Amnesty is particularly concerned because this type of "host government" or "state investment" agreements are used in many large-scale development projects.
It says that although such agreements are ultimately designed to ensure projects run smoothly in troubled areas, they should not be interpreted to carve out "a corridor" where individuals enjoy lesser protection.
It is calling on the World Bank not to support projects based on these contracts.
Professor of Law at Essex University Sheldon Leader said: "Disturbingly there may be hundreds of such agreements in existence around the world, drawn up to a similar template, weakening the capacity of states to protect human rights."
Andrea Shemberg, the report's author and Amnesty legal adviser, said tensions between the human rights obligations of Chad and Cameroon and their contractual obligations lie at the heart of the report.
"States' obligations under international human rights law cannot be superseded by the signing of a private agreement for an investment project.
"However, in practice such agreements could be used to sidestep states' human rights obligations and companies' human rights responsibilities."
The report warns that human rights protection is already compromised in Chad and Cameroon and that these legal agreements may exacerbate the situation.
It claims there is a prevailing climate of fear and intimidation around the pipeline, some of whose critics have already been arrested and intimidated.
It is essential therefore that human rights protection is of the highest standard, Amnesty says.
According to the report: "Thousands of people across a wide area have been or will be affected by the pipeline project and oil development.
The pipeline will boost Chadian government revenues by up to 50%
"Individuals may be aggrieved because the land they use or own is expropriated, or because the pipeline is buried under large areas of land where they fish, farm or use forests."
The pipeline project has already been hit by allegations of human rights abuses, with farmers in the Doba region of Chad claiming they were denied access to their land or compensation.
And fishermen in the costal area of Kribi in Cameroon claimed fish stocks on which they depended were destroyed.
ExxonMobil said it could not comment on these claims, but did say in a statement that it condemned human rights violations in any form.
It said it regretted that Amnesty International had not shared its report or consulted with it during its preparation, although Amnesty said it had held several meetings with the firm and given it a copy of its recommendations.
An ExxonMobil spokeswoman said: "The government of Chad and the World Bank put in place an unprecedented Revenue Management Plan (RMP) for the allocation of oil revenues, including monitoring by representatives of civil society.
"RMP sets in place a mechanism whereby a portion of the revenues generated from the oil development project are used for social development projects.
"Since first oil in 2003, more than $270m of oil revenues have been managed in accordance with the Chad Revenue Management Plan.
"No other country has established such a rigorous and transparent plan for managing the revenues from a natural resources project."
Fishermen on the Cameroon coast claimed they lost the fish stocks
A spokeswoman for the World Bank said it was still reviewing the detail of recommendations but said it shared Amnesty's concerns about the human rights record in Chad and Cameroon.
She said it had helped finance the project as a way of providing revenues to support economic development and reduce poverty in Chad.
"Under an agreement between the government of Chad and the World Bank, oil revenues are directed toward poverty reduction priorities such as health clinics and schools, and to communities living in the oil production area of the country."
The project is overseen by a formal body which incorporates civil society organisations including a representative of a major human rights group in Chad.
She added: "The World Bank, in collaboration with counterparts in the government and independent bodies, monitors the social and environmental impacts of the project and insists on adherence to stringent policies in these areas."