The goal of spreading "jihad" against Western governments and other perceived enemies of Islam is perhaps the most profound and, for al-Qaida in Iraq, the most scarring legacy of Zarqawi.
"There is a division separating now Sunni insurgents who see a future Iraq in the Middle East that we can recognize verses extremists who are intent on redefining the entire face of the Middle East," Evan Kohlmann, creator of the Web site globalterroralert.net, told National Public Radio.
Even at the height of his power, Zarqawi's statements made it clear that his goals were not the same as Sunni insurgents looking to protect their communities or former Baathists hoping to destabilize the Shiite government in Baghdad.
"We are not fighting our jihad in the name of nationalism. Our jihad is purer and higher. We fight so that Allah's word becomes the highest," Zarqawi was quoted by the PBS series Frontline as saying in 2003.
In its public statements and internal documents, al-Qaida leaders claim and celebrate that Iraq has ensnared the Americans in a no-win war and brought Sunni and Shia closer to civil war. They also have focused on training terrorists there for operations in other countries, including Jordan, Algeria and Morocco, hoping to fill the gap left after the Taliban collapsed in Afghanistan.
"Iraq became a big laboratory to train kamikazes and warriors," Hamida Ayachi, editor of the Algiers-based daily Djazair News, recently told the Washington Post. "They are trying to take young people from here to Iraq for training so they can use them later in North Africa."
This broader vision of jihad flared in 2006 when Zarqawi, frustrated by the lack of support from Sunni tribesman, threatened to launch new attacks against them if they did not do more to support the al-Qaida vision of jihad.
Zarqawi's threats so worried al-Qaida leadership along the Afghan-Pakistan border that they sent a letter to their top man in Iraq cautioning him against driving a wedge between Sunni tribes and al-Qaida.
The group's senior leadership represented by a relatively unknown al-Qaida leader, Atiyah, cautioned Zarqawi to not launch bombing attacks in neighboring Jordan, to be patient in his attacks in Iraq and to focus his jihad more on American forces and Shiite Iraqis.
"We warn against all acts that alienate, from killing to any sort of other treatment. Even insofar as the corrupt ones and traitors from among the Sunnis, we shouldn't kill them unless the people would understand and think that it was a good thing," Atiyah wrote in a letter. "However, if we come and kill some people whom we know to be corrupt and treasonous, but who are respected and beloved by the people, then this leads to great trouble."
For regional experts, the letter, which was released by Iraqi and American forces who said it was discovered in the house where Zarqawi was killed, appears to be a pivotal moment in the al-Qaida and Sunni relationship.
"In order to understand this letter one has to see the circumstances of when this letter was released,'' says Rita Katz, the director of the SITE Institute, which is devoted to tracking Islamist militant groups, told the Christian Science Monitor in October 2006. "This followed after Zarqawi had an audio message ... in which he threatened the tribes of the Sunnis who wouldn't cooperate with him."
Some experts have even said the split between Zarqawi and the Sunni leaders contributed to the militant leader's death since it was reportedly Sunni sources who told American officials that the Jordanian was hiding in the restive town of Baquba. But after the missile strike killed Zarqawi, al-Qaida in Iraq appeared to try and mend fences with their Sunni counterparts.
The man who replaced Zarqawi, Egyptian Abu Ayyub al-Masri, made overtures to Sunni leaders, saying in a statement that the two sides needed to work together to force the occupying American troops to leave Iraq.
Violence between the two sides briefly subsided, but al-Qaida was not standing by silently, instead moving to create a single umbrella organization, the Islamic State of Iraq, aimed at unifying the Sunni insurgents.
Brian Fishman, a senior associate at the Combating Terrorism Center, said the ISI is part of a larger strategy that is only partly focused on Iraq, pointing to the recent book, "Informing the People about the Birth of the Islamic State of Iraq," written by the ISI's main spokesman, Sheikh Uthman Bin Abd al-Rahman al-Tamimi.
"Believers will unite jihadi factions in and outside of Iraq by giving them an institution to rally around," Fishman wrote in his report, "Fourth Generation Governance." "Outside Iraq, the ISI provides al-Qaida sympathizers a firmer intellectual structure through which to imagine an Islamic government and the reestablishment of the Caliphate."
As ISI and al-Qaida worked to create a more cohesive insurgency, they moved to actually take control of areas of Iraq, an effort that set them at odds with local Sunni tribes and more nationalistic groups.
"In order to appease its foreign backers, especially the ulema of senior jihadi-salafi thinkers, [ISI] needs to consolidate real control in parts of Iraq," said Fishman in a later interview. "It cannot do that without challenging other militant groups."
By early 2007, the confrontation between Sunni groups flared into even more intense violence. Issuing a blanket warning to anyone who cooperated with the coalition or refused to support the ISI agenda, the group spread messages that it would "use every possible means to strike at the infidels and renegades."
The al-Qaida fighters have reportedly killed key insurgent leaders of Sunni militias in a series of clashes in March and April, hoping to shatter those more nationalistic forces.
In early April, the Sunni extremist group the Islamic Army of Iraq responded with a statement against the ISI, openly calling on Osama bin Laden to "bring in the line" of his followers in Iraq and accusing them of killing 30 Sunni fighters "with no justification."
In late April, renewed fighting erupted north of Baghdad and on May 2, anti-al-Qaida tribesman claimed they killed al-Masri, a claim the militant group denied.
Regardless of the state of al-Masri, the violence between the tribes and al-Qaida intensified, and Western officials have said they hope the cycle of violence dries up local support for the largely foreign-led al-Qaida.
"[Al-Qaida members] hope that if they murder random groups of women and children, the tribes will fall back in line," wrote Bing West and Owen West in the Wall Street Journal. "These tactics have locked AQI [al-Qaida in Iraq] in a fight to the death against the tribal leaders. It reflects an enemy who has lost popular support for his jihad, clinging to fear alone."
The apparent al-Qaida effort to intimidate Sunni leaders is pitting the militant group against its fellow Sunnis as well as Shiite and American forces. And the internal struggle is unfolding as al-Qaida in Iraq continues to struggle in the wake of Zarqawi's death.
"[Al-Qaida in Iraq] doesn't have that distinct command-and-control structure it used to," Fishman, of West Point's Combating Terrorism Center, told the Council on Foreign Relations. "Once you take away that figurehead, no one has been able to come in and wrangle all the pieces together."
That decentralized system, combined with an increasingly angry local Iraqi population, may threaten al-Qaida's goal of using Iraq as a training ground.
Recent attacks in the western Baghdad Amiriyah neighborhood are an example. In late May, local Sunni leaders accused al-Qaida militants of executing Sunnis without reason.
"I think this is going to be the end of the al-Qaida presence here," mayor of the Amiriyah neighborhood, Mohammed Abdul Khaliq, told the Washington Post.
"But if the Americans interfere, it will blow up, because they are the enemy of us both, and we will unite against them and stop fighting each other," he said.
Regardless of the role the United States chooses, experts foresee continued fighting among the insurgent forces as they jockey for a leadership role after American forces leave.
"[T]he insurgents sense victory and are positioning themselves to claim leadership of the resistance for the post-U.S. phase which is the overthrow of the sectarian government [in Baghdad]," said Mohammed Hafez, a political science professor at the University of Missouri and author of the upcoming book, "Suicide Bombers in Iraq: The Strategy and Ideology of Martyrdom."