British swimming chief David Sparkes has defended the team's record in Athens following criticism of head coach Bill Sweetenham's tough regime.
Stephen Parry's bronze in the 200m butterfly is the only medal so far.
Former British Olympian Mark Foster criticised Sweetenham for treating experienced swimmers like juniors.
But Sparkes, chief executive of the Amateur Swimming Association, told BBC Five Live: "Bill's got it right. We're going in the right direction."
Britain failed to win a single medal at the last Olympics in Australia so Sweetenham was appointed performance director to turn around the nation's fortunes
And Sparkes said: "The swimmers are much fitter than they were in Sydney and we're challenging in the finals."
Sweetenham's demanding training schedules have improved performances but have also come under criticism.
"
It's not like rowing where you've got about 30 nations in the world doing it. You can't compare swimming to rowing
"
David Sparkes
Foster, who was left out of this year's team after failing to make the tough qualifying standards, also claimed British swimmers were forced to peak too soon in the year to ensure they came through the trials in April.
But Sparkes denied the team was underachieving, claiming that it is much harder to win a medal in the pool than in most other sports as there are more countries battling for honours.
"We've got 150 countries competing for medals," he said. "It's not like rowing where you've got about 30 nations in the world doing it. You can't compare swimming to rowing.
"They actually get twice as much funding as us but you've got to remember you're not competing against the world, you're competing against a quarter of the world.
"There are 200 nations in swimming and 153 here in the pool, that's a heck of a difference to the 55 nations that will be paddling up and down in boats."