Senators voted by an overwhelming majority to unseat Senate President Evan Enwerem on Thursday following months of allegations and rumours that he had lied about his age and academic qualifications.
Chuba Okadigbo, 58, formerly a political adviser to civilian President Shehu Shagari, was elected as the new Senate president on Thursday.
Lower House Speaker Ghali Umar Na'Abba, was among several MPs to welcome the outcome of the vote, calling it a "vindication of the principled stand of the house" and a good omen for democracy.
Mr Na'Abba himself took office after the previous Lower House speaker, Salisu Buhari, stood down in July when he admitted to having lied about his qualifications.
Series of scandals
Nigeria's National Assembly has been hit by a series of scandals ever since the country returned to democratic rule in May.
Mr Enwerem has always denied the complicated and various accusations that he has lied about his age, his schooling and his legal qualifications.
But he has become politically isolated and the scandals surrounding him have badly weakened the National Assembly at a time when it is struggling to establish its authority in the new democracy.
Last week the visiting Canadian Prime Minister had to cancel an address to the Assembly when members of the lower chamber, the House of Representatives, refused to attend a joint sitting presided over by Mr Enwerem.
This caused embarrassment to the Nigerian Government, which had originally supported the Senate president but had gradually come to realise that he was a liability.
Nigeria correspondent Barnaby Phillips says public disillusionment with the National Assembly is beginning to grow. It has passed only a handful of bills in the past six months and has spent much of its time engaged in arcane political feuds.