Local official Lev Dzugayev said the release was "the first success" and expressed hope for further movement, the AP reported.
The insurgents first released three women and three children and then sent out another 20 people.
Casualty reports were varied, but an official in the joint-command operation for the crisis said under the condition of anonymity that 16 people have died and 13 were wounded, including some children under the age of 14.
Dzugayev said seven people have died.
Just before the release there was a brief burst of explosions when militants fired two rocket-propelled grenades, about ten minutes apart, into an apartment complex roughly 200 yards from Middle School No.1 in North Ossetia. The explosions frightened family members who have gathered outside the school and sent a black cloud of smoke into the sky.
Russian negotiators spent the night in telephone discussions with the roughly 15 to 24 insurgents who stormed the school Wednesday morning just after opening day ceremonies that included parents, teachers and students.
The militants herded more than 350 hostages into the school's gymnasium and placed a sniper on every floor of the school to keep guard.
Leonid Roshal, a prominent pediatrician who helped with negotiations in the Moscow theater takeover by Chechen militants in 2002, is participating in talks with the militants.
The hostage takers refused an offer to replace adults from the outside with children, and safe passage out of the school if they released the captives.
Offers to deliver food and water were also denied by the militants. Food and sanitary conditions in the school are not known.
Security forces surround the school and the militants have threatened to blow up everyone inside if the police attempt a rescue.
"There is no alternative to dialogue," said Valery Andreyev, the director of the local branch of the Federal Security Service, according to ITAR-Tass news agency. "One should expect long and tense negotiations."
In Moscow in 2002, rebels seized a theater that held hundreds inside. About 129 hostage deaths resulted when police pumped an unidentified gas into the building just before troops stormed in and killed the militants.
Andreyev also tried to reassure worried relatives gathered by the school.
"Do not listen to panicked rumors," Andreyev said. "No force action is being considered."
In his first public statement on the crisis, President Vladimir V. Putin said Russia's first priority was the hostages in the school.
"We understand these acts are not only against private citizens of Russia but against Russia as a whole," Putin said during a Kremlin meeting with Jordan King Abdullah. "What is happening in North Ossetia is horrible."
Wednesday's hostage takeover follows Tuesday's suicide bombing outside of a Moscow subway station that killed nine people and a nearly simultaneous explosion on two airplanes in Russia that killed 90 people over a week ago.
Officials say the attackers are part of the Chechen separatist groups that have been fighting Russian forces for over a decade, according to the New York Times.