His appointment has broken the Vatican tradition of conferring the honour on the Archbishop of Armagh - the leader of the Catholic church in Ireland.
Archbishop Sean Brady has held that position since the retirement of his predecessor Cardinal Cahal Daly.
Religious commentators say the appointment is a very unusual step which will raise questions about where power lies in the church in Ireland.
But Archbishop Connell, who said he was deeply moved to be chosen, has been quick to play down any suggestions of division.
"The president of the episcopal conference who will continue to speak on behalf of the episcopal conference, is the Archbishop of Armagh," he said on Sunday.
"I will be with the Archbishop of Armagh and there is no rivalry between us. There is no conflict between us. It is the one voice of the church," he said.
Nevertheless his appointment is the first time in many decades the see of Dublin has been recognised before that of Armagh.
He is only the third Archbishop of Dublin to be appointed cardinal, and the first since 1885.
Archbishop Brady has welcomed Dr Connell's appointment and has congratulated him.
He said: "He has given outstanding leadership to the Catholic church in Dublin over the last 13 years."
"I am confident that he will provide excellent assistance to the holy father in his new role."
37 new cardinals
Archbishop Connell has been described as Rome's favourite churchman in Ireland.
He has been considered a conservative cleric who is not afraid to defend traditional church views, even when it has proved unpopular.
Several years ago he was critical of a decision by the Irish President, Mary McAleese, to receive communion at a Protestant Church of Ireland cathedral.
But on Sunday the President welcomed his appointment as an honour for Dublin. John Paul II has appointed 37 new cardinals altogether, heavily putting his stamp on the body that will elect the next Pope.
The ceremony to install the cardinals is to be held in Rome on 21 February.