The postal service is "chronically poor" in parts of the UK, while compensation schemes for customers make it too hard to complain, MPs have said.
Sorting larger letters is more expensive, says Royal Mail
The Commons Public Accounts Committee found some "black spots", such as London, were particularly badly served.
The fact 15 million items a year were being lost, stolen or damaged meant customers were not always seen as the "absolute priority", the MPs said.
But Royal Mail said its service was "among the very best in Europe".
The committee called for industry regulator Postcomm to look into levels of service and to push harder for improvements.
It also criticised compensation schemes for lost, stolen or damaged post, claiming they did not offer customers adequate protection and were often too complicated.
Items lost: 13.1m
Items stolen: 1.2m
Damaged or unlawfully interfered with: 1m
Source: Royal Mail 2004-05
Royal Mail must be monitored to ensure it "is not abusing its dominant position to hinder the development of competition", the MPs said.
The committee's report comes in the same week as the closure of thousands of post offices looks set to be announced, with rural loss-making outlets likely to bear the brunt of the cuts.
The Royal Mail accepted a new regulation and pricing regime earlier this year.
The committee's chairman, Conservative Edward Leigh, said: "You would expect that a new regime for the quality of postal services would aim to improve collection and delivery times and reduce the number of pieces of mail lost by Royal Mail. If so, you would be mistaken.
"Neither of these aspects of performance is targeted under the current regime introduced in April of this year.
"Given that people are now receiving their mail later in the day and that over 15 million letters and parcels are being lost, stolen or damaged in a year, it is pretty clear that the interests of ordinary users of the post are not an absolute priority."
He added: "Freeing Royal Mail as far as possible of the burden of regulation is crucial.
"The answer is to use the power of existing regulation to improve the black spots of postal service and that means having detailed information on what those black spots are.
"What Postcomm must get right is to encourage healthy competition in the postal market as a way of driving up the quality of the postal service without damaging the interests of users of the post."
Committee member Richard Bacon said: "By comparison with many countries in the world, the Post Office is not doing too badly here, but there are problems."
He said Postcomm failed to measure issues that were important to customers - namely collection and delivery times, and the number of items lost.
He said changes made recently by the Post Office had been "painful" but "at the same time it's important that they balance the interests of customers with the interests of having a profitable post office."
Under the regulations agreed earlier this year, the price of a first-class stamp can increase by six pence by 2010 and that of a second-class stamp by five pence.
A Royal Mail spokesman said: "It is absolute nonsense to suggest anything other than Royal Mail's quality of service is at record levels.
"Its performance is among the very best in Europe and its prices are among the very lowest."
More than nine out of 10 first class letters were delivered on time, while parts of London had some of the best performances in the country, he added.