The bodies of Mr Kennedy, his wife Carolyn and her sister Lauren were recovered on Wednesday, five days after their plane crashed into the ocean off the coast of Massachusetts.

The ceremony was conducted on the USS Briscoe off Martha's Vineyard, opposite a house that used to belong to Mr Kennedy's mother, Jackie, and about three miles from where the bodies were found.

The BBC's Stephen Sackur: "The USS Briscoe fulfils its melancholy mission"

A public memorial service is under way at Old St Patrick's Cathedral, a Catholic church in New York, sponsored by the city's Irish community.

The service began with a procession of musicians playing bagpipes, followed by a moving performance of "I Will Arise Now and Go to Innisfree".

At the funeral ceremony off Massachusetts, there was an honour guard of sailors dressed in white.

Two navy chaplains and a Roman Catholic priest performed a simple ceremony before the ashes and three wreaths with red, yellow and white flowers were cast into the waves.

Three wreaths were cast into the sea for the dead

The mourners included Mr Kennedy's sister Caroline, his uncle Senator Edward Kennedy, the Bessette sisters' mother Ann Freeman, stepfather Dr Richard Freeman and sister Lisa Ann.

Boats and helicopters were ordered to stay three miles away from the Briscoe during the proceedings.

(Click here to see a map of the flight route)

Historian Michael Beschloss said Mr Kennedy had told his sister Caroline of his desire to be returned to the ocean, where he used to play as a toddler with his father, the late President Kennedy.

After the crash, JFK Jnr's parents-in-law intimated that they wanted John and Carolyn to go to their resting place together and to watch over Lauren.

Sister Caroline (right) and cousin Maria Shriver (left) leave for the burial

The New York Times also reported that the Kennedy family was keen to avoid having a spectacle made of Mr Kennedy's final resting place.

Mr Kennedy never served in the military, but US regulations allow the navy to authorise burials at sea in the case of "an outstanding contribution to society".

President Clinton and his wife Hillary will attend a memorial service for the Kennedys on Friday in New York.

New York's Irish community is planning a public ceremony for Thursday night at Old St Patrick's Cathedral in Manhattan.

And a private service for Mr Kennedy's sister-in-law will be held on Saturday night at Christ Church in Greenwich, Connecticut, where the Bessette sisters grew up.

Post mortem

A post mortem has found that all three died instantaneously from "multiple traumatic injuries" resulting from the crash.

Floral tributes outside the Kennedy's New York home

Mr Kennedy's body was found inside the wreckage of the light plane in which they were killed on Friday. The bodies of his wife and sister-in-law were discovered nearby.

The autopsy was conducted at a hospital after the bodies were brought ashore. They were then taken by hearse to a funeral home for cremation.

Intensive search

The wreckage of the single-engine Piper Saratoga HP plane was located in about 100ft (30 metres) of water.

Rear Admiral Richard Larrabee of the US Coast Guard said the major part of the fuselage had been recovered.

The BBC's Malcolm Brabant: "Nobody saw the plane crash into the sea ... few saw it return to the surface."

The plane's radar traces show the final descent may have exceeded 5,000ft (1,600 metres) per minute.

After levelling out at 2,300ft, Mr Kennedy turned to the right, climbed 300ft, turned to the right again and went into a steep dive.

Some aviators said Mr Kennedy could have been suffering from ''spatial disorientation'' - a common experience among less experienced pilots.

On Wednesday, Mr Clinton praised those involved in the search for the bodies for their ''fine work under extremely difficult circumstances" and said his ''thoughts and prayers'' were with the family.

Across the US, thousands of Americans have been laying flowers, offering prayers and lighting candles for the Kennedy couple and Miss Bessette.

Many have also queued at Boston's JFK library to sign a condolence book.

• Farewell at sea: Picture gallery
• The final moments of JFK Jnr's flight
• Clinton defends extended search
• JFK Junior: Born into fame and tragedy
• What's the future for George?