The men are being detained in Colombia
Richard Haass said if the IRA was engaged in activities that supported terrorism, it could have "potentially serious consequences for the role of the United States in the peace process".
Speaking at the US State Department, Mr Haass said he believed that three IRA suspects detained in Colombia last month were "not just on their holidays".
Three Irishmen, Niall Terence Connolly, Martin McCauley and James Monaghan, were arrested on 11 August as they left an area controlled by the FARC.
They are accused of training FARC guerrillas and travelling on false passports.
But the men denied they were terrorists. They said their visit had been misunderstood and that they were simply looking at the Colombian peace process.
Mr Haass said the United States was still awaiting further details about what the men were doing in Colombia.
But he added: "My understanding is that they were not there for vacationing."
Their discussions, he said, could only be associated with activities "under the rubric of terrorism".
He said no connection between the IRA and the FARC would be tolerated.
Mr Haass added that the FARC met "anyone's definition of a terrorist organisation and called for a total cessation of any relationship".
He also said the current environment in the US underscored the importance of organisations getting out of the business of terrorism.
Mr Haass has discussed his concerns over the arrests with Sinn Fein.
'No case to answer'
However, Sinn Fein's Gerry Kelly denied that his party had a case to answer.
He said: "Richard Hass can have his own opinion.
"But the facts are that three men were arrested who have not been charged and should not be tried in the media."
Mr Kelly also said it was important for people to realise how the Colombian issue was being used to frustrate the political process in Northern Ireland by those who opposed the Good Friday Agreement.
Collapse
Earlier this month, Sinn Fein president Gerry Adams again denied that one of the IRA suspects was a member of his party.
His comments followed confirmation by Cuba's foreign minister that Niall Connolly had been Sinn Fein's representative in his country for a number of years.
The men have now been moved from the main prison in the capital, Bogota, for their own safety.
Meanwhile, Mr Haass said he did not rule out the possibility of a breakthrough in the Northern Ireland political process before the latest deadline - Saturday 22 September.
The political parties have to reach agreement on the issues of IRA arms, policing and British Army demilitarisation or the political institutions could collapse.
"I've not given up hope that we could have a breakthrough of sufficient size which would once again recreate the conditions for a working partnership across the political divide," he said.
Mr Haass said IRA disarmament was only part of the package.
But he added: "No single action would have greater impact on the confidence of the other parties in Northern Ireland than a substantial act of decommissioning."