"It's not so much the age, but trying to survive these celebrations."
The Queen was speaking at a private lunch attended by royals from across Europe during a day of celebrations.
A special thanksgiving service was held at St George's Chapel at Windsor on Sunday morning, followed by a private reception and family lunch at the castle.
It was the biggest gathering of royalty since the wedding of Prince Edward and Sophie Rhys-Jones in 1999.
The Queen wished her husband, the Duke of Edinburgh a happy birthday and said: "I can't believe you're 80."
At the Windsor Castle birthday party, she said: "I speak for all the family and everyone here, thank you from us all."
Prince Philip said: "I'm not sure that I recommend being 80.
"It's not so much the age, but trying to survive these celebrations."
During the thanksgiving service, the Right Reverend David Conner, Dean of Windsor, paid tribute to the duke for his "loyalty, encouragement, inspiration, example and sheer stickability".
"We can be grateful for the example of someone who, on his birthday, chose to bring his family and friends to church," he said.
European royals
The birthday lunch at Windsor Castle was for up to 30 members of the Royal Family as well as six nephews, four great-nephews, one niece and six great-nieces from Germany.
Prince Harry, 16, arrived at St George's with one of the German nieces, Princess Xenia of Hohenlohe-Langenburg.
The Queen Mother, who will be 101 years old in August, the Prince of Wales and Princess Margaret all attended the celebrations.
Margaret, who was pushed in a wheelchair by her son Viscount Linley, is still suffering the effects of a series of recent strokes.
The British royals were joined by ex-King Constantine of Greece and Prince Paul of Yugoslavia.
Prince William, 18, was absent as he is in Africa during his gap-year before starting university at St Andrews in Scotland this September.
Busy schedule
Although many aspects of the duke's life have been well documented, some things about the Queen's husband are perhaps less well known.
The youngest child and only son of Prince Andrew of Greece and Princess Alice of Battenberg, Philip's grandfather was a prince of Denmark who became King of Greece.
But Philip renounced his Greek royal title in 1947 and became a naturalised British subject.
Philip is related to kings of Prussia and emperors of Russia, and had four elder sisters, only one of whom, Princess Sophie, is still alive.
And although the Duke of Edinburgh's only constitutional role is as a Privy Counsellor, he has always taken his royal duties seriously.
He has fulfilled more than 18,000 official engagements, excluding those accompanying the Queen, an average of 370 a year.
He is also known for taking a hands-on approach to the organisations he represents and has chaired more than 1,500 meetings.
Since 1956, more than four million young people from over 90 countries have taken part in the Duke of Edinburgh's Award scheme.
The duke is also a prolific writer and has had several books published on environmental, technological, equestrian and other issues.