Further incidents involving US troops posted in Japan could "shake the foundation" of US-Japanese ties, Japan's defence minister has said.
A recent rape allegation involving a young girl has sparked protests
Shigeru Ishiba also urged the US to take concrete prevention measures.
The warning comes after two rape allegations in 10 days against US servicemen on the island of Okinawa.
Meanwhile, Foreign Minister Masahiko Komura announced new surveillance measures agreed with the US to guard against crime around US bases.
Last week, a US marine living off base was arrested, accused of raping a 14-year-old girl.
The marine admits forcibly kissing her but denies rape. US military and civilian officials have apologised for the incident.
Speaking during a parliamentary meeting, Mr Ishiba said promises to improve behaviour in the future were not enough.
"I don't think an alliance is possible unless the US shares the view that if incidents like this continue to happen, it could shake the foundation of the Japan-US alliance," Mr Ishiba said.
Also on Friday, Mr Komura said Japan was looking into installing security cameras and conducting joint patrols.
The US and Japan had agreed to review security criteria for off-base residency, he said.
A 24-hour curfew has already been imposed on US troops, their families and civilians working for the military on Okinawa.
On Friday, the military held a "day of reflection" to encourage troops not to commit crimes.
Okinawa is a key hub because of its proximity to the Taiwan Strait.
Nearly 45,000 US troops, non-military staff and their families were in Okinawa as of the end of January, according to Japan's foreign minister.
Of those, 10,748 were living off base.
This week a second allegation surfaced against a US serviceman accused of raping a Filipino woman at a hotel in Okinawa.
As well as the rape cases, there have been allegations of other, more minor crimes including drink-driving and break-ins.
Are you in Okinawa? Send us your views on this story, some of which may be published.
The BBC may edit your comments and cannot guarantee that all emails will be published.