The New Zealand government could ban the Zimbabwe cricket team from touring in December.
Fleming said he will tour Zimbabwe
But they are powerless to stop their own team from playing in Zimbabwe in August despite urging them not to.
Foreign Minister Phil Goff said a side representing Zimbabwe was not welcome because of the country's "appalling abuses of human rights".
He added: "The Zimbabwe cricket team should not anticipate that it will be granted entry to New Zealand."
New Zealand are set to head to the strife-torn African country, run by Robert Mugabe, in August for five-week tour featuring two Tests and a one-day series.
Chief executive Martin Snedden said they had no other choice and could be fined by the International Cricket Council if they did not fulfill their fixtures.
They could only pull out if there were genuine player safety concerns but an independent security report released this week gave Zimbabwe the all-clear.
And there has not been one player who has threatened to boycott the tour.
Snedden said: "If you've entered into a contractual arrangement with nine other countries and you try and pick and choose later, then you expose yourself to ridicule.
"The players have been through this situation a number of times in recent years. It's an issue we've met head on and done pretty well.
"My understanding is the players are reasonably confident in the way we handle it and are prepared to trust us to do it right."
But the reciprocal tour in New Zealand in December could be cancelled if the government does not let the team in - and the New Zealand cricket team would not then be liable to pay compensation.
Goff said he had not yet brought up the matter at cabinet level nor taken legal advice.
But added: "Given the current appalling abuses of human rights and the relentless trend of the Zimbabwe government towards a dictatorship, the New Zealand government would not welcome a visit from a side representing Zimbabwe at this time."
England were in a similar dilemma last year but a one-day tour did go-ahead although the Test series was postponed as Zimbabwe had had their Test status suspended at the time.