People who want to become British citizens should be expected to learn the language, Gordon Brown told MPs.
The population is projected to rise to 71m by 2031
The prime minister told MPs people applying for citizenship or permanent residence "have got to accept responsibilities".
He also said "tougher rules" would help Britain restrict immigration to skilled workers who could boost the economy.
But he refused to put a number on how many people Britain could take and said annual caps could harm the economy.
Mr Brown was being questioned on a range of issues by the chairmen of all the House of Commons' committees, in his first session since becoming prime minister.
He said it was possible to define a set of British rights - as opposed to universal human rights - saying rights and responsibilities were different from country to country, citing US citizens' right to carry a gun.
"Becoming a citizen is an important act, because they are getting rights and in return for that they have to accept responsibilities," Mr Brown said.
"If someone comes to our country and is applying for citizenship or permanent residence they have also got to accept responsibilities.
"You should be able to speak the English language, you should be able to understand and speak about British cultural traditions."
Asked about the impact of immigration on Britain - Mr Brown said figures compiled using only small snapshots had to be treated with caution - and the reliable figures were those of the 10-year census.
But he said there had been a "recognition" that some areas were under pressure, in announcements of extra funding for some schools and areas.
Tory MP John Whittingdale asked about recent population forecasts - which suggest the UK may house 71 million people by 2031 - and asked the prime minister how many people Britain could take.
Mr Brown did not give a figure and said annual caps would end up excluding skilled workers Britain needs and would not include EU citizens and dependents.
He told MPs 200 million people a year were looking to move to different countries - and it was incumbent on Britain to look again at the rules.
He said the new points system for skilled workers and a tougher approach to people coming to Britain illegally were part of the "biggest changes to immigration policy for 30 to 40 years".
He said there were about 200,000 people from other countries working in the City of London, who had helped boost the economy.
But he added: "When more and more people are travelling around the world looking for a country of choice, we as a country have a responsibility to set the rules by which we want to offer people the chance to come and live here."