A report by the Royal National Institute for the Blind Cymru has blamed ignorance on the part of travel companies and the public for compounding difficulties faced by visually impaired travellers.
The study called Travellers Tales is calling for action to improve the safety of visually impaired travellers whether they are on foot or in a vehicle.
The charity said it fears somebody could suffer a serious accident if more is not done to improve the situation.
According to their research, the part of a journey people with sight problems fear the most is walking along the pavement because of hazards along the way.
Wheelie bins, rubbish bags and untrimmed hedges can cause major difficulties for the blind and partially sighted.
Stephen Green, who is visually impaired, is a regular traveller on buses and trains in and around his home town of Pontypridd in south Wales.
He said: "A lack of information means it's difficult to know where to catch my bus or train, what time it is supposed to arrive, or when I've arrived at my stop.
"However, what really frustrates me is when thoughtless people park their cars at bus stops, meaning I have to struggle into the road in order to get my bus.
"Walking is a major hazard too. Shop signs, people's rubbish, untrimmed hedgerows and broken or uneven pavements make my life much harder than it need be.
"I just wish people would have more consideration."
'Change law' call
At present, a loophole in the Disability Discrimination Act prevents people from taking action for discrimination when they are travelling on a bus or train.
The RNIB Cymru is calling on the government to alter the legislation.
RNIB Cymru's policy officer Gavin Cox said: "People contact RNIB to express their concerns when they leave their homes
"We're talking about smple things like overgrown hedgerows, litter left in the street, cracked paving stones and illegally parked cars.
"When they have to use public transport, we're finding that access is not available whether travelling by bus or train.
"We need more public consideration, but we need the government to take action.
"The Disability Discrimination Act does not cover you once you set foot on a bus.
"We're calling on the government to cut this loophole so people are no longer discriminated against when they travel on public transport."