Global Investigative Journalism Network (https://archive.gijn.org/category/methodology/page/6/)
The latest tools and resources from the Research Desk: new world of drones databases available, reports from the European Parliament Research Service, Top Green Companies in the World 2015, a handy free extension to download entire pages or individual files, and more.
$8 billion in just a few hours earlier this year? It was because of a web scraper, a tool companies use—as do many data reporters. A web scraper is simply a computer program that reads the HTML code from webpages, and analyze it. With such a program, or “bot,” it’s possible to extract data and information from websites.
We’re back with another Research Desk post. We’ve curated a collection of two new research databases and thirteen new or updated research reports. All of the resources listed and linked below are free to access and use.
Confiscati Bene, released in mid-December in Europe, is a pioneering data journalism collaboration that digs into the $4 billion of goods in the EU confiscated from criminals by European authorities. An international team of journalists and their allies sought to create a European database of seized assets and answer troubling questions about the accountability of the process. Confiscati Bene (literally, Well Confiscated) received support from GIJN member JournalismFund.eu; the main project can be seen at http://eu.confiscatibene.it.
“Hello,” wrote the anonymous source to a German newspaper, “this is John Doe. Interested in data?” Thus began what would soon become an international financial investigation into what are being called the Panama Papers—an investigation so massive that even whistleblower Edward Snowden, on Twitter, called it the “biggest leak in the history of data journalism.”
The story published in the Washington Post’s Wonkblog ended up being about race, but it didn’t start out that way. Nick Diakopoulos, who leads the lab, wrote for the Wonkblog last year with a story on how surge pricing motivates Uber drivers to move to those surging areas, but does not increase the number of drivers on the road as Uber claims.
The trove of files that make up the Panama Papers is likely the largest dataset of leaked insider information in the history of journalism. For ICIJ’s Data and Research Unit, it offered a unique set of challenges. The overall size of the data (2.6 terabytes, 11.5 million files), the variety of file types (from spreadsheets, emails and PDFs to obscure and old formats no longer in use), and the logistics of making it all securely searchable for more than 370 journalists around the world are just a few of the hurdles faced over the course of the 12 month investigation.
In an extract from his book Finding Stories with Spreadsheets Paul Bradshaw explains how to use basic cleaning functions in spreadsheets to make it easier to combine data, including a case study where the same functions were used to speed up a research process for a story.
Mexican newspaper El Universal has put a face to the 4,534 women who have gone missing in Mexico City and the State of Mexico over the last decade: Ausencias Ignoradas (Ignored Absences) aims to put pressure on the government and eradicate this situation.
At First Draft, we frequently receive emails from a whole range of people asking how they can start doing the sort of online open-source investigation and verification that they’ve seen us doing. The skills and methodologies used are all something that can be learnt through a little persistence, but here are a few pieces of advice to get you started.