Dozens of Serbs had been occupying the courthouse since Friday. When U.N. police stormed the building just before dawn Monday to arrest the Serbs, other demonstrators reacted by firing guns and throwing Molotov cocktails and hand grenades.
The demonstrators surrounded three U.N. vehicles, pulling out and freeing about 20 of the 53 protesters who had been arrested.
At least 42 U.N. and NATO troops and 70 protesters were wounded in the violence, the Associated Press reported.
Machine guns could be heard firing and NATO helicopters flew above the town until midday.
The city of Kosovska Mitrovica is populated by Kosovo's ethnic Albanian majority and a Serbian minority who opposes Kosovo's independence. The protesters have been trying to take over institutions, such as the courthouse, which have been operated by the United Nations since the end of the war in Kosovo in 1999.
Kosovo was thrown into turmoil in 1999 when NATO forces, including American troops, launched a 78-day bombing campaign to stop Serbian forces from killing Albanians.
Since 1999, Kosovo has been administered by the United Nations with NATO and EU forces remaining as peacekeepers. A U.N.-led process began in 2005 to determine the province's future. Kosovo declared its independence from Serbia on Feb. 17.
NATO and the U.N. condemned "lethal violence, including direct fire by a mob," news agencies reported. Serbian President Boris Tadic accused the international forces in Kosovo of "using excessive force," and warned of "escalation of clashes in the entire territory" of Kosovo.
Alexander Ivanko, spokesman for the U.N. mission in Kosovo, said U.N. staffers were ordered to relocate to regional headquarters in the southern part of the town, according to the AP.
He said they would return once the security situation improved.
U.N. Secretary-General Ban Ki-moon pledged that the U.N. will continue "to take measures required to implement its mandate" to administer Kosovo, spokeswoman Michele Montas said, according to the AP.
Ban "deplores the violent attacks" against U.N. personnel" and "urges all communities to exercise calm and restraint," she said.
Serbia's ally Russia called for restraint by NATO, and Serbian officials said they were consulting Moscow on joint steps that could be taken to protect Kosovo Serbs, Reuters reported.
