Speaking to BBC Wales' Scrum V rugby programme, the chief executive said:
"The Olympics can be the pinnacle of Sevens rugby and we will decide whether we will keep World Cup sevens, or not, if we get into the Olympics."
The IRB has launched a high-profile campaign to end rugby's 92-year Olympic exile, with star names Danny Cipriani, Bryan Habana and Jerry Collins appearing in a promotional video.
One of Cipriani's rivals for the Lions fly-half jersey, Wales's James Hook, first made his international mark in the 2006 Commonwealth Games Sevens tournament in Melbourne and gave his backing to the 2016 bid.
"I would not mind being a part of that and I think it could be hugely beneficial," Hook, who will be 31 in 2016, told Scrum V.
OLYMPIC SELECTION TIMETABLE
Nov 2008: IRB and the six other sports present to IOC Programme Commission
March 2009: Programme Commission and other IOC members attend men's and women's Rugby World Cup Sevens in Dubai
June 2009: IRB presents to IOC executive board. Executive board may or may not make recommendation to IOC members
Oct 2009: IOC members vote on which new sports will be included in 2016, two will be chosen
"For me the Commonwealth Games was a huge experience and my first opportunity to play against well-known players like Matt Giteau of Australia.
"I think lots of us found that teams you had never even heard of were capable of beating or coming close to beating the bigger sides, which can only be good for the game."
Rugby bosses have opted to pursue Sevens rather than the 15-a-side rugby because of the length of time needed to stage a 15-a-side tournament, and to keep the focus of the full game on the Rugby World Cup, Six Nations and Tri Nations.
In the meantime, the IRB will spend their time making a series of top-level presentations to the International Olympic Committee in the run up to the selection vote in Copenhagen next October.
Rugby missed out on selection for the 2012 Games in London and this time is up against six other sports - golf, roller sports, baseball, softball, squash and karate - with two set to get the nod from the 111 IOC members.
According to the IRB's own figures, rugby union is in rude health.
The 15-a-side World Cup in 2007 in France achieved record profits, was watched by a cumulative TV audience of 4.2 billion, and saw two-and-a-quarter million people go through the turnstiles.
A record number of unions have also bid to host the 15-a-side World Cups in 2015 and 2019.
To win an Olympic gold medal is a big dream and why shouldn't rugby players get to do that?
But becoming an Olympic sport, it is argued, would bring even bigger benefits.
"We are trying to grow the game in new territories and are spending US $50m a year," said Miller.
"But we are being told by member unions in countries such as Russia, China and the United States that if rugby became an Olympic sport everything would change.
"The government would see the potential for the Sevens teams to win a medal and as a result they would have government funding and better access to facilities.
"The smaller nations [in an Olympic context] such as Fiji would also have the chance to win a medal, as would Welsh rugby players as part of the Great Britain team.
"Rugby players do not often get the chance to mingle with other sports.
"To win an Olympic gold medal is a big dream and why shouldn't rugby players get to do that as well as other sportsmen and women?"
The IRB are also keen to stress the shared values held by the Olympic movement and rugby union, but ultimately any decision rests in the hands of the 111 IOC members in Copenhagen.
According to Miller, rugby would also make good commercial sense for the IOC.
"The Olympics has a huge stadium for the opening ceremony and a week later it has the athletics, so there is nothing in the stadium for five or six days," he said.
"We could fill that stadium for two or three days and we will bring rugby fans to the Olympic Games in droves. They would support not just the rugby but also the Games as a whole.
"That is something we have, that a lot of the other sports trying to get in do not necessarily have."
*Scrum V, BBC Two Wales, Sunday 1455 BST
Watch live EDF Energy Cup action of Ospreys v Harlequins, followed by a full round-up of the other matches and a report by former Wales captain and BBC Olympic commentator Eddie Butler on rugby's Olympic bid
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.