Zuma received 2,329 votes, ahead of incumbent Mbeki's 1,505 votes, according to the Associated Press.
Zuma, who was fired by Mbeki in 2005 over corruption allegations, is wildly popular among the party members. And the party so dominates South African politics that its nominee is almost certain to win in the 2009 presidential election. Mbeki is barred under the constitution from running for a third term.
Rivalry among backers of the two candidates created the worst rifts in the ANC's history. The vote was delayed two days because of the political wrangling and raised concerns among veterans, such as former president Nelson Mandela, that government infighting would distract from issues such as battling AIDS, crime and poverty.
Chaos erupted in the conference room when the results were announced, and then Mbeki and Zuma mounted the stage together and embraced, reported the AP.
Uncertainty over Zuma's policies and his strong left-wing backers has caused unease among many investors, according to Reuters.
But Intelligence Minister Ronnie Kasrils said Tuesday that ANC policy was unlikely to change after the election.
"The economy is a very central issue. That is where there will be thorough discussion," he told reporters.
As for the corruption allegations, Zuma defeated an initial round of charges in court, which stemmed from a multi-billion-dollar arms deal in 1999. But observers say prosecutors could revive the charges.
Zuma also was acquitted last year of raping a family friend infected with HIV. He was widely ridiculed for saying in court that he showered after the encounter to minimize his possibility of infection, which showed poor understanding of how to protect against the disease ravaging his country, reported the Washington Post.