Soldiers detained Abdallahi on Wednesday, took control of state broadcast outlets and announced the creation of a state council led by the commander of the presidential guard, Gen. Mohamed Ould Abdel Aziz, The Washington Post reported.
The bloodless coup occurred after Abdallahi fired Aziz and three other generals believed to have supported lawmakers in opposition to the president. Those lawmakers accused Abdallahi of corruption and reaching out to Islamist radicals.
Condemnation of the coup came swiftly from the international community.
Washington demanded Abdallahi be restored to power and threatened to cut off aid, as did the European Union.
A statement from the African Union said it "demands the release of the president ... and other personalities who were arrested, the preservation of their safety, security and dignity, as well as those members of their families," reported Reuters.
Mauritania's last coup in 2005, which involved some of the same members of the military, was wildly popular within the country and silently applauded abroad because it ended the 21-year reign of a dictator and led to the 2007 elections, according to the Post.
Aziz referenced the earlier coup when he led thousands of supporters in a march Thursday through the capital of Nouakchott.
"It's the army that brought an end to dictatorship in 2005. And today it's once again she that brings an end to dictatorship, to nepotism, to chaos and disorder," he yelled, according to the Associated Press.
The junta leaders did not specify a date for elections to replace the 15-month-old government.
Until elections are held, the West African nation will be governed by an 11-member council, which will ensure government institutions continue to run normally, according to a junta statement read on national television.
