Former Australian swimming champion Dawn Fraser has even called for the Commonwealth Games to be moved.
Wales will send an estimated 150 athletes to the event in October 2010 and the Commonwealth Games Council for Wales wants evidence as much as reassurances that Delhi will be safe.
Ellis, the former captain and coach of both the Wales and Great Britain hockey teams, admits: "Security is an obvious concern to anybody who is about to take a team abroad, particularly to that part of the world where we've seen some horrific incidents.
"We have to worry about the security because it would be irresponsible of us in Wales to take anybody if didn't think we could keep them safe during competition.
"The Games committee are taking pre-Commonwealth precautions on the construction sites and there's background checks of personnel being employed.
"For the Games period, we're told there will be comprehensive security at the venues and all over the city and security levels will be at their highest.
"So I am somewhat reassured that they are taking it seriously.
"But these are words, we must get some sort of proof that this is going to happen.
"We will have one or two reconnaissance missions and that will be crucial for us to make up our mind and make sure we are going to a safe place for our athletes so they can enjoy an experience of a lifetime.
"We must make sure we do not put our athletes in a position where they might not be safe, we have to be 150% certain."
Ellis, who won 138 caps Welsh caps between 1963 and 1980, understands the athletes and will sympathise with anyone who decides to remain at home.
"I wish I could guarantee everybody's safety," she said.
"But having seen what happened this week, I don't think we can guarantee anyone's safety.
"I'm absolutely sure that there will be people who have reservations about going and all we can do is monitor the situation carefully for them and keep the athletes up to date.
"We will not expect an athlete who feels it is unsafe to go to India.
"If the Commonwealth Games does go ahead in Delhi, we will make sure the athletes are safe but they have a choice whether they want to go."
Weightlifter Michaela Breeze, who has won back-to-back gold medals at the 2002 and 2006 Commonwealths, believes that athletes should make a stand against any potential terrorists at the 2010 Games.
"The fact that someone makes a threat, the world shouldn't come to an end," said Breeze.
"If anything, we need to make sure that we are there to show these people that they can't use sport for a way of dictating the things they want.
No one is safe from terrorism and we must realise we live in a very unsecure world
Commonwealth Games Council for Wales president Anne Ellis
"There were risks before the 2008 Olympics in China and there were no security scares where we were.
"And I would be very surprised if athletes don't go to the Games, as athletes won't train for so many years just to pull out because of something that might happen.
"Of course it is of great concern that there are security issues and athletes are being targeted.
"But I'm confident the Delhi organising committee will be putting on the highest levels of security during the Commonwealth Games."
Ellis insists the Commonwealth Games Council for Wales will continue to take advice from the British Government and its security services before taking any team to India.
"I remember the Munich Olympics in 1972," said Ellis, referring to the deadly attacks on Israeli athletes by the Palestinian militant group Black September.
"Sport hasn't been hit by terrorism for some time.
"But no one is safe from terrorism and we must realise we live in a very unsecure world."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.