The Zimbabwe Cricket Union (ZCU) has agreed to tour England next month, subject to approval from the country's Sports and Recreation Commission.
But England's refusal to play their World Cup game in Zimbabwe created "ill feeling" between the two countries.
ZCU managing director Vince Hogg told BBC 5 Live that the African country had been "hurt" when England withdrew from their group match in the World Cup in February.
"We were hurt by their non-arrival and there was some ill feeling. We gave every assurance about security and we felt let down," Hogg said.
"We don't believe that they were justified. We thought it was the wrong decision.
"England have learned from the experience as well and they expressed regret for not fulfilling their fixture here. I'm sure they don't want to repeat that.
We are not in the game of suits and counter-suits
"England are important to us and we are important to England. We are always looking at the cricket issues and we think we made the right decision (to tour England)."
The England and Wales Cricket Board (ECB) had estimated a cancellation could cost them £10m because television rights and sponsorship deals have already been finalised.
But ZCU chairman Peter Chingoka said he does not expect the Zimbabwe government to enforce a boycott.
"The suggestion of retaliation was aired and was looked at seriously but at the end of the day the board was unanimous to do the right thing," he said.
"We are not in the game of suits and counter-suits, we are in the game of developing Zimbabwe cricket."
ECB chief executive Tim Lamb confirmed Morgan had given a qualified assurance England would tour Zimbabwe in 2004.
They have also agreed to make a "goodwill" payment to cover part of the costs of the cancelled World Cup fixture.
"Subject to safety and security concerns it is fully our intention to undertake that tour to Zimbabwe in November 2004," Lamb told the BBC Sport website.
"We recognise that ZCU had incurred losses as a result of the late cancellation of the World Cup match. We have agreed in principle to make a contribution towards those losses."
Zimbabwe are due to play two Tests, including the first to be played at the Riverside ground in Chester-le-Street, and also take part in a triangular one-day series with South Africa.