Commercial fisherman are blamed for the ever-decreasing fish stocks
Plans to force sea anglers to buy a £22 annual sea fishing licence are being considered by the government.
Commercial fishing firms could also be made to pay up to £1,000 for the benefit of fishing in British waters.
In return the government would safeguard sea fish stocks and look after sea anglers' interests, it says.
The Department of Environment, Food and Rural Affairs (Defra) is consulting on the plans, which were drawn up by the prime minister's strategy unit.
A Downing Street spokesman said: "This is just one of the proposals set out in the review of enforcement and charging in marine fishing."
"No decisions have been taken," he added.
If the licence was introduced, it would mean sea anglers were charged the same as their fresh water counterparts.
It is thought such a licence could raise £3m a year and this would then be put towards maintaining and managing fish stocks.
But sea anglers fear that they may just be forced to pay the licence fee without getting anything in return.
Development officer at the National Federation of Sea Anglers, David Rowe, said: "If in the end what we are saying is that the fish stocks will be managed in a sustainable manner then that is welcome.
"But we don't see anything in the review which would deliver that. What we want is bigger and better fish," he said.
A spokeswoman for Defra said there would be a number of advantages for sea anglers including representation on the soon to be set up regional advisory councils.
These bodies would make representations about fishing stocks to the European Union which sets the quotas for international fishing.
They might also get improved facilities such as fishing platforms and habitats created from artificial reefs.
An estimated 1.1m people take part in angling in the UK, contributing £1.3bn to local economies, according to Mr Rowe.