The former Danish prime minister, who took over as Nato's head in August, also called for Moscow's co-operation on Afghanistan and Iran.
A revitalised Nato-Russia Council would provide "a forum for serious dialogue", he said.
Mr Rogozin welcomed Mr Rasmussen's address as "very positive, very constructive", but added that it needed careful scrutiny.
"Co-operation with Russia is not a matter of choice but of necessity," Rogozin said.
Pressure on Iran
Nato-Russia ties improved after the end of the Cold War but deteriorated as the 28-nation alliance expanded to take in former Soviet bloc countries, and suffered greatly after Russia's brief conflict with Georgia in 2008.
Mr Rasmussen called on Moscow for a "genuine new beginning in our relationship, in our own interests and that of the entire international community".
There is no reason to fear that these [missile defence] plans will weaken the defence of any ally
Referring to the US missile defence rethink, he said "the new plans will make capabilities ready sooner than the previous plans and will provide us with broader coverage".
"There is no reason to fear these plans will weaken the defence of any ally.
"Improved relations between Nato and Russia will also be to the benefit of our eastern allies," he said.
Russian Prime Minister Vladimir Putin on Friday welcomed President Barack Obama's shift in US missile defence strategy as a "correct and brave" decision.
Russian President Dmitry Medvedev said the move would make Moscow "attentive" to US concerns, but rejected any suggestion of forming "compromises or primitive deals" in response to the US announcement.
US officials say Mr Obama and Mr Medvedev will hold talks on the sidelines of the UN General Assembly meeting in New York on Thursday.
Russia had long objected to plans pursued by the administration of former President George W Bush to base a missile interceptor system close to its borders.
Mr Rasmussen said tackling the proliferation of ballistic missile technology was in the fundamental strategic interests of both Nato and Russia.
BBC Europe correspondent Jonny Dymond
In the medium and long term, the new secretary general wants to reinvigorate the Nato-Russia council and carry out a joint review of the strategic security challenges.
In the short term, he wants to deepen co-operation in a wide range of areas.
Perhaps most eye-catching was the prospect Mr Rasmussen raised of the US, Nato and Russia working together on missile defence.
When asked for detail after his speech he demurred, saying that the specifics were beyond him.
Russia should put "maximum political and diplomatic pressure on Iran to stop its nuclear aspirations", he added.
There should also be more co-operation over policy on Afghanistan, which was important for both Moscow and Nato, he said.
Asked about Nato expansion, Mr Rasmussen said Georgia and Ukraine would become Nato members as and when they satisfied the necessary criteria, but added that Moscow should not see that as a threat.
"We have to provide an atmosphere and security environment within Europe within which the open-door policy can continue, while at the same time ensuring Russia does not feel threatened."
His speech showed that Nato realises it needs to come up with a new strategic concept and essentially reinvent itself, says the BBC's defence and security correspondent, Nick Childs.
"In that sense it's generally agreed that having Russia as a partner on broad global security concerns is better than having new divisions," says our correspondent.
"But just how to put that into practice, and overcome the obstacles that are clearly still there, is a question that still has to be answered."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.