Ms Rosen, who has been with RIAA for 17 years, said she wants to spend more time with her family - partner Elizabeth Birch and their twin four-year-olds.
Her work has seen her lead the group's successful copyright infringement battles against song-swapping services Napster and Aimster.
"
This has been the most exciting job I can imagine
"
Hilary Rosen
Under her leadership, the RIAA became a major player in the debate over copyright protection.
And earlier this week the music industry was successful in its battle against internet piracy.
A US court ordered a telecommunications firm Verizon Communications to identify a customer who allegedly illegally downloaded music, complying with a subpoena issued by the RIAA.
The profile of Ms Rosen, 44, has been high for several years; in 1998 she was named one of the most powerful women in Hollywood by trade paper Hollywood Reporter.
Resignations
She said the decision to leave the job had been "extremely difficult", but that it was "the right one for my family".
A successor has not yet been revealed for the position. The RIAA represents the music industry's major labels including Bertelsmann AG, BMG and EMI Group plc.
Ms Rosen's resignation comes close behind two other big executive resignations in recent weeks.
Jay Boberg, president of Vivendi Universal's MCA Records, resigned just days after Sony Music chief Thomas Mottola stepped down to be replaced by industry outsider Andrew Lack, an NBC executive.