Predator unmanned aerial vehicle at Kandahar Air Field, Afghanistan, September 2009 | DoD photo by Master Sgt. Jerry Morrison, U.S. Air Force/Released

Visual Interventions

Visualization can add important context to help the public follow this inquiry and to assist policymakers in their careful evaluation of current and future programs and policy responses. Last week, I blogged on MapBox about a site I created to visualize one particular drone strike database, maintained by the Bureau of Investigative Journalism.

U.S. Drone Strikes in Pakistan, 2004-2013.

I explained how our new MapBox.js library makes it easy to visualize complex and dynamic datasets in real time. Over the past few months, we've been refining the mapbox.markers features of MapBox.js, which allow you to add markers to your MapBox maps using client-side JavaScript, rather than rendering them as separate map layers. In the post, I dive into the Google Fusion Tables API, which provides the data feed powering the drone strike map.

Revisiting Pakistan

In the week since we published the blog post, I received feedback -- some good, some critical, but all ultimately constructive -- about my drone strike microsite. I'm going to go over some of the key criticisms, which provide good guidance for future visualizations, as well as paint a more vivid picture of the current state of drone strike data, which is of especial importance, given the current UN inquiry.

Method behind the map

The Bureau of Investigative Journalism provides a live-updated database of U.S. covert drone strikes in Pakistan. There are other sources for this information, including New America Foundation and The Long War Journal, each of which has its own advantages and disadvantages.

To get a better idea of the differences between the Bureau of Investigative Journalism's dataset, New America's and the Long War Journal's, see pages 43-44 of "Living Under Drones" and pages 27-33 of "Counting Drone Strike Deaths." The Bureau's data is sourced from local and international news accounts of the strikes and credible field investigations in drone strike areas. An important factor to account for in using drone strike aggregators is that the dataset is a derived one, coming from third-party sources rather than traditional official channels. Please see The Bureau of Investigative Journalism's Methodology for additional information regarding their sources and methodology.

The map site visualizes four aspects of the Bureau of Investigative Journalism's drone strike data:

1. civilian deaths
2. children killed
3. number injured
4. total number killed

On the map, each marker represents a drone strike and has been scaled to size based on the number of casualties. Because the number of casualties and injured are estimates, the Bureau's dataset sometimes gives a range, rather than a single number. In those cases, the high and low values were averaged to scale the marker.

Data-driven design

Drone strike data is difficult to obtain and is generally problematic in nature. This is due to several reasons, which are outlined in "Counting Deaths from Drone Strikes," published in October 2012 by The Human Rights Institute at Columbia Law School. After carefully evaluating the estimates provided by three different drone tracking datasets, the Columbia Report concludes:

We -- the public, the analysts and experts, and the policymakers -- still do not know the true impact or humanitarian cost of drones; the estimates, though well-intended, may provide false assurance that we know the costs and can fairly assess whether to continue drone strikes.

A similar uncertainty is expressed in the "Civilian Impact of Drones" report, though directed toward the U.S. Government for failing to act as an official source:

The information included here is not comprehensive, as despite public and repeated allusions to covert drone strikes by Obama administration officials in 2011 and 2012, most official materials related to the drone program are classified. Even the existence of a CIA drone program remains classified, although government officials have repeatedly leaked information to the media.

Thus, one might summarize the problems with drone strikes datasets as deriving from insufficient information about the justification for the strike -- the secrecy of which may be driven by operational security concerns -- as well as the existence of only third-party estimates, rather than comprehensive official totals, for drone strike impacts.

Calls for Analysis and Visualization

There are additional dimensions of the impact of drone strikes -- besides the raw numbers of casualties and injured persons -- which deserve further analysis and visualization. Two dimensions of importance for those conducting the UN inquiry are first, the relative mortality of drone strikes; and second, the relationship between civilian deaths and unmanned versus manned aerial strikes.

The map microsite does not address the relative civilian mortality rate of particular drone strikes or of drone strikes in aggregate. Using the Bureau's dataset, an analysis of civilian mortality would be difficult, if not also methodologically questionable. This is due first to the Bureau's use of numeric ranges, rather than single values, for casualty and number injured estimates; and second, that not every strike has values for the number of civilians killed.

A third obstacle to a relative civilian mortality analysis relates to the distinction between civilian, non-civilian, combatant, and targeted populations, which would require thorough and consistent operational definitions for membership in each group. It would also require that these definitions were agreed to and followed throughout -- from the initial data collection, reporting, and eventual analysis and visualization. Such consistency is extremely difficult when working with third-party sourced data. Without this consistency in group membership definitions, a cursory analysis of the number of civilians killed versus total number of deaths from a particular strike might fail to address the lived humanitarian impact.

Another avenue for visualization that could provide meaningful insight into the effects of drone strikes might look at relative mortality of unmanned strikes in comparison with manned aerial strikes. A researcher might ask: "Do unmanned aerial strikes cause greater/fewer civilian/non-civilian casualties than do manned aerial strikes?" While I did not have the opportunity to explore this concept with this visualization, it appears to be an important avenue for careful analysis.

The report, "The Civilian Impact of Drones: Unexamined Costs, Unanswered Questions" from the Human Rights Clinic and the Center for Civilians in Conflict at Columbia Law School, critically examines claims related to drones' battlefield precision and could provide a good starting point for questions regarding unmanned versus manned strikes.

There are nevertheless many potential pitfalls associated with this sort of analysis, stemming from the faulty comparison of fundamentally incommensurable situations, which might be due to conflict theater, target, time of strike, type of weapon/targeting capability, altitude of pilot, and, among others, atmospheric composition. Military publications about the use of unmanned aerial vehicles in lethal combat could also prove helpful here. A few reports from the Army, Navy, Air Force, and Department of Defense include:

• Brave New Warfare: Autonomy in Lethal UAVs, Lieutenant Matthew S. Larkin, Master's Thesis, Naval Postgraduate School
• The Children of Aphrodite: The Proliferation and Threat of Unmanned Aerial Vehicles in the Twenty-First Century, Major Darin L. Gaub, School of Advanced Military Studies, United States Army Command and General Staff College
• Unmanned Systems Integrated Roadmap FY2011-2036, United States Department of Defense
• Robot Wars: Legal and Ethical Dilemmas of Using Unmanned Robotic Systems in 21st Century Warfare and Beyond, Major Erin A. McDaniel, U.S. Army Command and General Staff College
• Establishment of Models and Data Tracking for Small UAV Reliability, Marinos Dermentzoudis
• Air Force UAVs: The Secret History, Special Assistant to the Chief of Staff of the U.S. Air Force Dr. Thomas P. Ehrhard

My inclusion of these reports here is meant to ease the burden of those wishing to research the topic further, and should not be taken to imply implicit or explicit support for the messages contained therein.

Reflections on an Inquiry

U.S. MQ-1 Predator unmanned aerial vehicle (UAV) at Camp Dwyer in Helmand province, Afghanistan, May 2011, U.S. Marine Corps photo by Lance Cpl. Robert R. Carrasco/Released

One entity that could substantially improve the current drone strike data situation -- a situation in which we "do not know the true impact or humanitarian cost of drones," according to "Counting Deaths from Drone Strikes" -- is the U.S. government. Greater transparency -- more openness -- concerning the targeted individuals, and the civilian and non-civilian populations collaterally damaged by the U.S. government's drone strikes in Pakistan (as well as Yemen and Somalia), would enable greater accountability and careful analysis in assessing the program's humanitarian impact and updating international laws to ensure international human rights and legal safeguards are met.

Head over to MapBox to read the original post, "Using Google Fusion Tables to Add Real-Time Feeds to MapBox Maps," and be sure to check out my map, U.S. Drone Strikes in Pakistan.

Chris Herwig is a data analyst at MapBox, where he uses open-source tools to wrangle and visualize complex data in innovative ways. His tools of choice include R, TileMill, GDAL, and PostGIS. Chris is passionate about open data, civic engagement, and governmental accountability. In his spare time, Chris engages the larger open data community through Occupy Data and other data-centric civic organizations and hackathons.

Journalists will be able to embed these maps directly into their online coverage, zooming in on certain areas and letting users pan around to see the crowds. They'll also be able to compare ceremony attendance from 2009 with this year via a slider comparison feature on the map itself. We're working with GeoEye to process their 2009 imagery and prepare the web maps ahead of Inauguration. To do this after post-processing required us to manually georeference the imagery, which we did using GDAL, QGIS, and MapBox Satellite. For a step-by-step walk-through on how we did that, check out this blog post.

On Monday watch for satellite maps of the Inauguration, and be sure to check out the panning and comparison features.

Chris Herwig is a data analyst at MapBox, where he uses open-source tools to wrangle and visualize complex data in innovative ways. His tools of choice include R, TileMill, GDAL, and PostGIS. Chris is passionate about open data, civic engagement, and governmental accountability. In his spare time, Chris engages the larger open data community through Occupy Data and other data-centric civic organizations and hackathons.

Brawley, California | MapBox Satellite

Lago O'Higgins, Chile | MapBox Satellite

South Korea | MapBox Satellite

I spent the past four months acquiring dozens of terabytes of high-resolution imagery for the continental United States from each state's GIS agency, the USDA Farm Service Agency, and the U.S. Geological Survey to clean, process and render the images into an aerial map of the United States. Eventually, our aerial map will cover the entire world.

The government -- at the local, state and federal levels -- provides some amazing sets of public data. Unfortunately, many public data sets are difficult to access and download, especially for bulk users like myself, because of the data portal sites. Two exceptional examples of great AND accessible data are the Arkansas GeoStor and the Utah Automated Geographic Reference Center, both of which allow users to access raw data through maps -- better for casual users -- and FTP directories, which are better for bulk data users.

Making open data more open

Open data is not truly "open" if it is inaccessible. Open data projects should include bulk data access and retrieval options that are as free and open as the open data itself, along with access options for more casual and programmatic users like data interfaces and APIs. We should be discussing best practices for making open data more open and accessible in our conversations about open government data. I recently blogged my takeaways from accessing bulk aerial data for MapBox Satellite, highlighting some of the successes and failures in providing geodata I saw at the state and local levels, and providing some recommendations to make this process easier and more accessible.

It's time we move beyond just talking about the merits of open data, and start talking about the reality of how data needs to be made accessible to make it a truly open and public resource.

You can read my full article "The Trouble with Geoportals: Why open data should be free, open, and accessible" on the MapBox blog. Also, read Open Aerial: The Data Behind MapBox Satellite for more information on the public data powering MapBox Satellite.

Chris Herwig is a data analyst at MapBox, where he uses open-source tools to wrangle and visualize complex data in innovative ways. His tools of choice include R, TileMill, GDAL, and PostGIS. Chris is passionate about open data, civic engagement, and governmental accountability. In his spare time, Chris engages the larger open data community through Occupy Data and other data-centric civic organizations and hackathons.