World Karate Federation chief Antonio Espinos was furious after his sport was denied an Olympics debut in 2012.
Karate and squash were briefly chosen to replace baseball and softball before being rejected, meaning the earliest they can now compete is 2016.
"For our athletes, it's an insult to talk about 11 years from now.
"The whole career of the karate athlete is eight to 10 years, I can't tell my athletes to keep going until 2016 due to five minutes of what happened here."
He added: "I can tell you it's very, very bad, so disappointing - how can I explain what happened to my members?"
Squash legend Jahangir Khan said: "As a player, one never gives up. We'll try again for 2016. We'll get in one day," he said. "Obviously, I'd have liked to see my game there. But we'll never give up."
The claims of rugby sevens, golf and roller skating to be included were also rejected.
Many of the administrators involved with these sports, together with baseball and softball chiefs, were left surprised and disappointed by the decision.
Don Porter, the American president of the international softball federation, said he was devastated by the vote.
"We thought that we had a lot of support," he said. "The members told us we were getting support, but obviously we weren't.
"I don't want to say it's an anti-US thing, but they are two native American sports."
Softball was introduced in Atlanta in 1996 and has been dominated by the United States, who have won all three gold medals to be contested.
Aldo Notari, the Italian president of the international baseball federation, acknowledged that the absence of major leaguers in the Olympics had been the crucial factor in the sport's exclusion.
"The problem with baseball is the best players are not going to the Olympics Games," he said.
"But baseball is still in Beijing and it is still necessary to work for the future in 2016."
Baseball made its first appearance in an Olympics in the 1992 Games, although it was only in 2000 that professional players were allowed to take part.
Rugby's governing body, the International Rugby Board, said the IOC had missed a "golden opportunity to modernise the summer Olympic Games sports programme".
IRB president Dr Syd Millar, said: "In terms of broadcast, commercial and ticket sales opportunities, the IOC's own evaluation process [has shown] that Rugby Sevens is among the top five in comparison to the existing Olympic sports."
"The result of this decision is also very disappointing for the smaller nations of the world and African nations who often have little opportunity to compete at the highest levels in the Olympics.
"Rugby Sevens, with Fiji as the reigning world champion, provides these nations with the opportunity to be very competitive."
The modern pentathlon was also tipped to lose its Olympic status but survived.
The event made its Games debut in 1912 and had a women's competition added in 2000 and sees athletes compete in shooting, fencing, swimming, running and show jumping in the same day.
International modern pentathlon federation president Klaus Schormann said: "Everybody was nervous."
He added that the federation had revamped the competition and were launching a new head-to-head format in September to make it faster and more exciting for television audiences.
"If you're working hard, the IOC members are following what you're doing. They could see our activities."