Unable to broker an agreement on major points of contention, the heads of the council moved Thursday vote on parts of the constitution.
"We've had enough speeches," said council chairman Sibghatullah Mujaddedi. "Let's start voting."
Most of the Pashtun delegates, the same ethnic group as President Karzai, began voting at several polling stations around the large tent where the deliberations have been taking place.
But some 200 other delegates mainly representing ethnic Uzbek and Tajik groups remained seated, refusing to participate in the balloting.
For more than two weeks, 502 delegates have debated the shape of a new Afghan government, but failed to find consensus on a series of critical issues, including the power of the central government, the role of the presidency and the protection of ethnic minorities.
The inability to bring the debate to a final vote accentuates the apparent ethnic divisions within the assembly that threaten to prolong or derail efforts to build a new government.
The holdouts have criticized attempts to create a strong central presidency, saying they fear the presidency could become a dictatorship and lead to more violence.
The Associated Press quoted Mahsa Toyie, a Tajik delegate from Herat, as saying Karzai's government was trying to push through a document that ignored the needs of minorities.
"This constitution is not for one tribe, it is for the whole country," she said.
But members of the majority Pashtun group and those running the loya jirga said the boycott was the work of Islamic extremists and those loyal to powerful warlords in the northern part of the country.
"There are several fundamentalists at work here," Mirwais Yasini, the loya jirga's deputy chairman, told the AP. "The jihadi groups all want a share of the power."
Observers said the Thursday boycott was troubling, but added they hoped it would only delay a final agreement.
"I am concerned that there is an ethnic polarization that could be, if allowed to continue, very damaging," the European Union's envoy to Afghanistan Francesc Vendrell told the BBC, adding he hoped it was a "temporary explosion".
Karzai appears to have a majority to pass most of the key components to the constitution, but analysts say he will need more than a bare majority if the new government expects to exert control in the tribal nation.