Ugandan rebel Lord's Resistance Army leader Joseph Kony has moved to the Central African Republic from his Democratic Republic of Congo base.
Joseph Kony is wanted on war crimes charges by the ICC
Ugandan Minister of State for Defence Ruth Nankabirwa said that by doing so, he was violating a ceasefire deal.
A peace agreement is due to be signed at the end of the month to end the 22-year rebellion in northern Uganda and she expressed fears he may not attend.
The brutal conflict has left thousands dead and nearly two million displaced.
"We are concerned because the CAR is far away from the gazetted area where they are expected to assemble," Mrs Nankabirwa told the BBC's Focus on Africa.
Some LRA rebels are stationed in CAR, but she expressed concern that they were continuing to violate the ceasefire and were not serious about peace.
The LRA say this is an attempt to discredit them.
"Let's wait for the D-day when we are expected to sign the peace agreement and see whether they will come to sign," said Mrs Nankabirwa.
Last week, President Yoweri Museveni said LRA leaders should face local justice.
The LRA insists that the war crimes indictments placed on its leaders at the International Criminal Court are lifted before they sign the deal.