A technical problem led to several suspects receiving September phone bills with charges for a connection to an unknown voice-mail number.
German authorities can only use wiretapping in serious cases such as murder, money laundering, kidnapping or treason.
When the suspects called the number, they were told that they did not have access to the voice mailbox.
This was because police were using the mailbox to record their phone conversations.
Thousands of taps
"The technical fault arose when we were installing new software," a spokesman for the mobile phone company O2 said.
Only a handful of cases were involved, he added.
Telecommunications authorities said that nearly 20,000 lines were currently being tapped.
The Frankfurter Rundschau newspaper said the incidents had involved both police and intelligence services.
The daily Sueddeutsche Zeitung said the federal crime office (BKA) had sent out an urgent warning to regional offices about the security problem.
The BKA particularly wants to know how many suspects learned that the police were at the other end of the line during the calls in question, the paper reports.