They found that the lotions stopped the skin from burning, but failed to prevent the rays from going through the skin.
When the rays go through the skin, they can damage our DNA, which can cause cancer.
They say that further research is now needed to see if any other creams on the market are more effective.
They told people that the safest way of keeping out the sun's rays was to cover up.
A skin cancer expert backed them up, saying: "People should stick on a hat, a t-shirt and stay in the shade."