Taylor's assigned lawyer Karim Khan also left the courtroom, saying the former leader had withdrawn permission to have Khan represent him and was instead applying to serve as his own defense attorney.
The judge then ordered the trial to continue, and Chief Prosecutor Stephen Rapp began his opening statement.
The trial was due to adjourn after the prosecution's presentation and is set to resume June 25.
In a letter to the court that Khan read earlier, Taylor said he was not allowed to meet with his preferred lawyer and that his one court-appointed attorney was at a disadvantage to the extensive prosecution team.
"At one time I had confidence in this court's ability to dispense justice," Taylor's statement said. "Over time, it has become clear that confidence has been misplaced. I will not receive a fair trial."
Rapp countered Taylor's claims, saying that the former leader had been assigned a lawyer, assistant attorneys, a special investigator and court funds. "Everything that can be done is being done," Rapp said, according to the Associated Press.
Presiding Judge Julia Sebutinde of Uganda repeatedly cut short Khan's address and Taylor's letter, demanding an explanation of Taylor's absence.
Sebutinde ordered Khan to stay and represent Taylor through the first day of the trial, but he said he no longer had Taylor's authority and left.
Taylor, 59, has pleaded not guilty to 11 counts of war crimes and crimes against humanity at the Special Court for Sierra Leone. Approximately 50,000 people died in the West African nation's civil war from 1991 to 2002.
The conflict was marked with brutal killings, mass rape, the amputating of civilians' limbs, and the recruitment of child soldiers as young as 8.
Taylor was president of neighboring Liberia at the time and is accused of supporting the insurgency.
His defense does not dispute that the atrocities took place but says he did not orchestrate the fighting.
In Mumbai, Tensions Remain One Year After Deadly Attack
