Patrick Ball on the Perils of Misusing Human Rights Data

Belated post at Foreign Exchange on a talk I attended last week about problems in human rights data collection. A snippet:

In his lecture, Ball presented a number of cases like this, from Kosovo, Guatemala, Sierre Leone and Timor-Leste, where sound and verifiable data was used effectively to answer a small question, then stretched to answer a broader question for which it was not suited. The researchers in each case, Ball argued, had confused ‘what was observable’ with ‘what was true,’ failing to acknowledge the existence of all the data they hadn’t collected or hadn’t thought to ask for.

The second half of Ball’s talk focused on ways to work around this, starting with some quick math. A short, 7th grade example: Two NGO projects in a country report widely differing figures for killings over the same period, call then result-set A and result-set B. They have some overlap in the list of names, call that subset M. What’s a rough estimate for the total number of killings, N? [Solution at the bottom of the post.]

More on Ball, and the solution, in the post.