Português | 中文 | বাংলা | Français| العربية | Español | Русский | Türkçe

BBC Africa Eye

Protesters hide from live fire at a pro-democracy demonstration in Sudan.

It’s an increasingly common sight: a protester or refugee lying prone on the ground, after a crowd has fled or the tear gas has cleared.

And the official response is almost as common: a government or police department claiming to have no responsibility for the injury or death.

But what really happened to these people, amid all the chaos?

Increasingly, open source investigative reporters are using visual forensic methods to find out. The waves of protests against police brutality and racism, sparked by the U.S. killing of Minnesota resident George Floyd, illustrate the need for reporters in newsrooms around the world to develop these niche skills. And the protests may just be starting. Experts warn that both climate change and economic fall-out from the COVID-19 pandemic could bring crowds and security forces into even more frequent conflict in the future. 

The visual forensics approach is not about a smoking gun document, or a whistleblower’s testimony, but rather about assembling pieces of a visual and time puzzle, where, taken together, they have the weight of evidence to have impact.

This raw evidence can not only expose deliberate attacks by security forces, but can also reveal to police that procedures they designed to be nonlethal can have lethal outcomes.

For instance, in June 2020, The New York Times used time markers on video clips to show how the use of pepper ball projectiles to enforce a curfew around George Floyd protests in Louisville, Kentucky, led to the killing of a man in his home.

The approach has also exposed massacres.

Last year, BBC Africa Eye grabbed Facebook livestream clips of a protest in Sudan in real time, and later pieced together evidence of the killing of at least 61 protesters by the Rapid Support Force militia, hired by members of Sudan’s ruling military council. The team carefully reconstructed events from 300 mobile phone videos — mostly shaky clips shot by protesters as they fled.

BBC Africa Eye

Some of the 300 mobile phone video clips BBC Africa Eye gathered and mapped to describe a Khartoum massacre in 2019.

In April 2020, Nick Waters, an open source investigator with nonprofit investigative group Bellingcat, needed to figure out exactly where a Pakistani migrant, Muhammad Gulzar, had died in Turkey, but only had social media video showing apparently featureless farmland in the background. So he used the direction of the furrows beneath the man’s body, and the crop patterns behind, to find the exact field via Google Earth. His team then investigated the gap in time between the supersonic “crack” of a bullet and the “bang” of its muzzle blast to find the proximity of the gun heard in the footage. Together with four media collaborators, Waters’ team  determined that — despite government denials — Greek security forces had used live rounds against refugees near their border that likely killed Gulzar, and wounded six others.

“It generally starts with trying to get absolutely every bit of visual information of an incident,” Waters explained, “and systematically using any key words that can find it. But it’s mostly about a problem-solving mindset.” 

Open source investigations of violent public incidents were once considered a form of fact checking in journalism. After all, they rarely name the individual police officer or soldier who pulled the trigger, and have tended to meet the “balance of probabilities” standard of evidence, rather than hard facts.

However, new tools and forensic methods — together with the global ubiquity of mobile phone video — have increased accuracy to the point where these investigations are having a direct impact.

In 2018,the visual investigations team at The New York Times traced a bullet that killed a volunteer medic, Rouzan al-Najjar — standing in a dense crowd of Palestinian protesters — to an Israeli sniper positioned well over 120 yards (110 meters) away. The team’s task was made even tougher by the fact that the time was incorrectly set on several of the mobile phones that shot the critical footage. But they carefully recalibrated the metadata for each, and commissioned a spatial investigation of the scene, with the help of UK-based research agency, Forensic Architecture, which exposes human rights abuses, often in collaboration with civil society organizations. The NYT team then learned from a ballistics expert that an Israeli bullet likely missed its target, skimmed off the ground, and struck al-Najjar above her chest.  

“We’ll always try to get to the standard of beyond a reasonable doubt,” said Malachy Browne, a senior open source reporter at the NYT. “We are always trying to nail it. For instance, our series of investigations into the bombings of hospitals in Syria last year would not have had the impact if we had said, ‘It’s likely Russia did this because witnesses say so, and they heard Russian aircraft.’ Doing the work to verify the moment of those attacks, and assigning it to particular pilots, had far greater impact. But there are times when we can’t be judge and jury — when we need to simply present the facts as we’ve found them.”

Alexa Koenig, co-founder of the Human Rights Investigations Lab at the University of California, Berkeley, said the next generation of open source reporters in the United States is being led by women journalists. Women represent about 70% of the student investigators at the lab — which partners with major media, and does much of the legwork on tasks like geolocation — and that the entire lab leadership is female.

In March, the Washington Post collaborated with student investigators at the lab to reveal how people were beaten by police at an event celebrating the release of a political prisoner in Western Sahara. “I think these methods are going to become critical and central for reporting in the 21st century,” Koenig observed. “It’s going to be increasingly difficult for reporters to get on the ground, especially in situations of conflict.”

Forensic Architecture

Built from video and drone footage, this 3D model shows how an Israeli sniper round struck and killed volunteer medic Rouzan al-Najjar. Image credit: Forensic Architecture and The New York Times

Any Reporter Can Use Forensic Methods

Leading open source investigators stress that all reporters, even those without technical skills, can investigate the causes of harm to protesters through basic visual means, and that they should not be intimidated by the array of advanced tools used in the field. Visual evidence garnered from just a few different angles can suffice.

In 2017, the same NYT team that used drones and 3D modelling to expose al-Najjar’s killing revealed how security agents for visiting Turkish President Recep Tayyip Erdoğan beat up protesters in Washington DC. This time the journalists used only TV and social media footage. That evidence was largely built in just two days, and some of the key footage was obtained from ordinary, on-the-ground reporting. Here, Browne simply asked a witness if he had video recorded the melee, and received the file on a Google Drive link. The team then pored through the various clips, frame by frame, from five angles, and positively identified 24 men behind the mass assault.

“Ordinary, shoe-leather reporters can do stories like these,” said Browne. “The ultimate tool is your eye, and it comes down to looking at imagery carefully. A lot of our work is rooted in open source reporting and forensic audio-visual analysis, but it’s wrapped in traditional reporting — it’s hammering the phones and talking to experts.” 

In interviews with leading reporters and editors, GIJN identified 12 techniques and 12 tools that proved effective in recent major investigations that traced harm by security forces.

Techniques for Uncovering Incidents Within Crowds

• First focus on discovering material — downloading and archiving as much raw video and imagery as possible — and then work on verification, geolocation, and, finally,  chronolocation (establishing and synchronizing timelines). Keep your initial search broad using everything from Facebook and TikTok to CCTV and audio recordings, and archive the pages from your search history. Typically, the Where, When, and How will lead you to the Who.
• Track individuals through video clips by using markers, like distinctive clothing. One man’s bald spot proved to be an important marker for the NYT investigation into the beating of protesters by that Turkish security detail.

Bellingcat

A tear in the knee of Muhammad Gulzar’s jeans helped journalists track him in another clip where no other part of his body was visible.

• Visit or contact businesses in the vicinity of protests or other incidents, and ask to see their CCTV or security footage. If they initially decline, show them your early coverage of the event, and ask again. Also, show your first story in a series to witnesses who were initially reluctant to speak, as many change their mind after seeing evidence of a reporter’s intent. Defense attorneys are also a good source for both CCTV and police body cam footage.
• Organize your video clips side-by-side, and examine them frame-by-frame. For collaborations, consider organizing the materials on platforms that make it easy to share with remote partners, like Google Sheets.
• If timestamps are unreliable, synchronize the videos by looking for temporal markers visible in all of them — like a car door closing, or a light turning on.

Bellingcat

The blue lines show how Bellingcat synchronized the timing of video clips from refugee shootings on the Greek border, using markers like gunshots and car horns. Image: Bellingcat, Forensic Architecture, and Lighthouse Reports

• Remember the familiar tools you use every day. If you need to show that a protester or police officer could not have walked from A to B in a certain time, just enter those addresses into Google Maps, and it will automatically generate a walking time, together with car and public transport times. These methods have the additional benefit of being known and trusted by audiences.
• Focus on, and identify, groups responsible for harm, rather than individuals, unless you have a compelling reason — and strong evidence — for naming an individual officer definitively linked to a wound. Blaming individual security force members may require layers of additional evidence, including facial recognition and even ballistics matches, which are rare, and which may not be necessary in making the case for the cause of harm.
• Use the ‘intersection’ method to identify the position of a person or object relative to distinctive features in their surroundings, or to know where the camera was. For instance, one imaginary line can be drawn from a mast and a church spire, and an intersection point can be found with a second line between a path and a building corner. Google Earth Pro allows you to draw straight lines between these objects on the screen.

Bellingcat

With the help of Google Earth Imagery, Bellingcat fixed the position of the camera that shot the top image by tracing one line through a minaret and building pillar (the yellow line), and another through a distant roof and a nearby restaurant (the red line), and finding where they intersected, from above. This established the bomb craters from an airstrike on a market in Yemen.

• Don’t automatically discount video with timestamps that don’t match your event timelines. Investigators have found that mobile phone times are often inaccurately set, and the metadata can be recalibrated to synchronize with an accurate clock.
• Don’t rely on eye-witness accounts of events in space and time when starting your reconstruction, as memory of traumatic events is often severely flawed. The NYT found that several medics on the protest field where Rouzan al-Najjar was killed had misremembered where they were standing, and that some were certain that two shots were fired at a particular point when there had only been one shot.
• Interrogate initial reports by assessing visual evidence or by using a professional translator. Bellingcat understood from initial reports that Muhammad Gulzar was “near the gate” at the border, but could find no evidence of this from several videos. Instead, it turned out that Gulzar was near a hole in the border fence. “Hole” and “gate” may have been confused in the initial translation.
• Avoid manipulating footage, unless it is essential for audience understanding — and, if you do, explain why you’ve altered it. However, artificially highlighting an object, like a weapon, is appropriate, when presented within the context of original video.

Casting a Wide Net 

Marc Perkins, editor of BBC Africa Eye, said the Sudan Livestream Massacre story — which won a Webby Award in May 2020 — involved many elements of open source reporting, but that it was built on three pillars: pioneering coding work, shoe-leather reporting, and, above all, the courage of the filming protesters.

“A lot of videos that are shot aren’t really out there, so what Ben Strick did — our open source investigator — was to develop his own script, his own code really, that was ripping off all these livestreams,” said Perkins. “There were code words in Arabic and English plugged in. So on the day the protests happened, he had the process already in place, so he could [grab] all these streams as they were happening. Often, these streams disappear. He had spent months training YouTube to find videos you and I wouldn’t find with a simple search, in what was a brilliant technical effort.”

Perkins said the team spent most of the time during the investigation geolocating all 300 videos, using tools like satellite imagery and Google Maps. They then chronolocated when events happened with audio, shadows cast by people and buildings, and other markers.

Strick worked with a Sudanese producer and two Sudanese journalists, as well as his own open source team, Arabic translators, and external collaborators.

“In a way, this is the opposite to how we reporters used to work, where you’d say, ‘OK, I’ve got this great video’, I hold on to it, I don’t tell anybody, and I run to my editor,” said Perkins. “That’s not the way open source works. It needs a lot of people and a lot of collaborative work to piece together the puzzle.”

Perkins’ colleague Bertram Hill recently developed and mapped out a comprehensive, accessible list of more than 200 open source and forensic tools, including several that are specific to African databases.

“This dashboard is Africa-specific, although, frankly, it’s global,” Perkins said. “This is something I’m happy for GIJN to share. If even one reporter in every newsroom could learn a little about these tools, it would make a real difference on the [African] continent.”

That resource, divided into two dozen categories, can be found at this link.

There were similar strands from the Sudan story in an investigation by Venezuela’s Ultimas Noticias, which analyzed video footage to reveal that the deaths of two students in a 2014 march in Caracas were linked to live fire by four members of government intelligence services.

Fixing Video Clips In Space and Time

Bellingcat’s Waters said collaboration was key to the Gulzar investigation — as it is in many open source investigations — with Dutch nonprofit Lighthouse Reports co-ordinating the project, video analysis and special investigation by Forensic Architecture, and Der Spiegel and Sky News doing on-the-ground reporting in Turkey and Greece.

Bellingcat

Waters’ team had to locate the place and time of each of the clips that showed all six refugees who were wounded near the Greek border in March.

He said Bellingcat now used the word “discovery” for the video collection phase of its investigations, as the material is sometimes used in legal proceedings, such as the trial on the downing of Malaysia Airlines Flight MH17.

“When you’re geolocating, you’re trying to take every piece of information within that image and find places where it is consistent [with other data],” said Waters. “We already had the general area, and we could see the [border] fence line in the images, so we had kind of a box that [Gulzar] was in. We could see the furrows that faced the fence, which narrows down the fields he could possibly be in. In the background, there were two features — a mound and a tree line —  which narrowed it down to two fields with perpendicular furrows. And in only one of them did two other fields meet in the background, and that’s where he was — the southwest corner of the southern field.”

With the place fixed — and visualized with satellite imagery — the team then had to fix the time of events. Waters said the pattern of gunfire — automatic bursts, double-taps, and single shots — proved crucial in this chronolocation. 

“Those rounds came from a rifle consistent with an M16, or some other 5.56mm assault rifle — not a pistol or revolver,” he said. “Live rounds were fired by Greek forces into Turkey that day. And in not a single video or still image did we see any Turkish forces close to that scene.”

Tools to Unpack Protest Incidents

• Timeline-based video editing tools, like Adobe Premiere Pro or Avid Media Composer. Use them to organize your materials on a single screen, and audio tools like Adobe Audition to synchronize sound markers. Open source tools like WatchFrameByFrame can help the analysis.
• Advanced search functions on Google and Twitter to track video. Leading open source investigators rely more on these than bespoke tools, and note that they are far more powerful than many reporters realize. Search other domains on Google with phrases like “site:youtube.com” for more accurate results.
• Satellite images. TerraServer allows you to browse existing images without charge, while Maxar DigitalGlobe is valued by many investigators for the high resolution of its images. Planet Labs has also played a central role in several investigations.

Planet Lab

A satellite image helped fix the position of refugees under fire near Turkey’s border with Greece. Image credit: Planet Labs and Bellingcat

• Google Earth. ”Google Earth is amazing — by far, the best geolocation tool we use,” notes Waters. “And it also provides historic satellite imagery, and has very liberal licensing. It also has a tool called Landscape within it, which is fantastic. To show that Syrian military officers were [there], we could go to where they were standing, and you could see their viewpoint. It was absolutely intuitive.”
• Suncalc. This tool allows you to know the position of the sun in the sky at a particular time and place. This enables investigators to use shadows to work out the time a pictured event took place.

Bellingcat

The shadows cast by these soldiers helped Bellingcat determine that senior Syrian generals were observing a town shortly before it was attacked with chemical weapons.

• Try to develop your own program to search YouTube precisely by date, or collaborate with organizations with their own internal programs, like Bellingcat. Investigators have found that YouTube’s internal search functions are poor, and even good advanced search functions like Montage are not always adequate.
• Archiving tools, like Hunch.ly. They not only capture and save webpages automatically — eliminating the need for saving screenshots and URLs — but can help demonstrate that the evidence you use in a story was not altered. Waters said: “It’s not a 100% solution, but it’s the best we have.”
• AI-driven image sharpening tools, like Topaz. While this technology can sharpen blurred images dramatically, it is important that reporters inform readers that the new image represents the tool’s interpretation.
• Spectrograms. These visual representations of frequencies can be used, for instance, to estimate the distance between a victim and the shooter.

Beck Audio Forensics

This spectrogram shows the gap between the “crack” of a supersonic bullet, in red, and the trailing “bang” of a rifle (in yellow) used near the Greek border. Image: Courtesy Beck Audio Forensics

• Invid is a key tool to help verify video material, by pulling out key frames from video on platforms like Facebook and YouTube, so you can search for their cover images in history, and find the original version.
• People-finding apps like Pipl and Spokeo can identify and verify witnesses and protagonists in almost every country. Knowing a person’s email address makes these apps especially powerful in finding alternative contacts and background details.
• Tin Eye, a reverse image search tool, and Google Image search, can help you find previous images on the web, including uncropped images with more background detail. 

Waters said that inviting a person new to the material to look through it, late in the process — a “fresh set of eyes” — could reveal details previously missed. For the New York Times story on the point-blank shooting of a Hong Kong protester on October 1, 2019, Waters was invited to check pre-analyzed video. He found that the primary reporting team had already nailed the story, but he was able to point out one small final piece in the puzzle: that one of the protesters was wielding a hammer in the skirmish with police.

The Importance of Being Organized

While journalists don’t need to be tech experts for this work, they do need have their visual materials organized.

On June 1, 2020, American restaurant owner David McAtee was shot and killed by police enforcing a protest curfew in Louisville, after his customers had taken shelter inside his home.

To investigate what happened, the NYT’s Malachy Browne opened his Adobe Premiere Pro edit program and divided his computer screen to show four videos.

New York Times

Malachy Browne synchronized CCTV footage (left) with Facebook livestream footage to understand how people in David McAtee’s house reacted to pepper rounds fired at the front door.

On the left, he had silent CCTV footage from three angles, provided by local police, and he compared them to a Facebook livestream with audio, from a different angle, on the right. He decided that he couldn’t use video timestamps, because they appeared to be unreliable. So Browne’s team synchronized the videos by using three markers seen in each: when car lights switched on, when the person filming the livestream closed the door of his truck, and when a man seen walking in the video planted his right foot.

“Then we had all four frames lined up, and we could hear from the livestream every action on the CCTV, and see who fired first, and what led to this tragic death,” said Browne. “We could see whether the police breached their own guidelines on how to disperse non-violent crowds, which was our conclusion on what happened.”

While the police knew they were using nonlethal pepper ball projectiles, the story’s video recreation of the scene revealed that McAtee, the restaurateur, had to rely on other visual cues to understand the threat: he sees a bottle exploding on a table, and something strike his front door within inches of his niece’s head, as men in dark uniforms approach with compact weapons. In his confusion, McAtee fires off a shot with his gun, and is then killed by live fire from police.

Meanwhile, non-lethal violence against protesters can easily lose attention in today’s media emphasis on body counts.

French filmmaker David Dufresne has demonstrated how a systematic reporting focus can expose systematic police repression, with his project to investigate violence against France’s Yellow Vest demonstrators. Dufresne uncovered more than 800 cases of police violence or wrongdoing toward protesters — including 24 cases of blinding and 279 head injuries — in an investigation that was recognized with France’s Grand Prize for Journalism in 2019. 

Dufresne decided to cast his net wide, rather than deep, in assessing so many injuries across so many protests, but verified evidence for each picture, X-ray, or video that he obtained online. He found the metapicz tool to be effective in verifying the metadata for these. Dufresne has kept his investigation going, and by mid-June 2020  had recorded a total of 950 cases of police violence against protesters, including 340 head injuries.

Open Source Work Can Be Exhilarating

For investigators like Waters, there is a special excitement about presenting raw evidence to audiences, and a creative challenge in choosing a presentation that people can intuitively understand.

Bellingcat went so far as to apply different standards of fact for its findings in the same story on the shooting of refugees near the Greek border, depending on the weight of evidence for each incident.

“On the balance of probabilities, we can say that Gulzar was likely shot by Greek security forces,” Waters said. “However, regarding an incident at 10:57 am, we can say it’s beyond a reasonable doubt that Greek forces shot and wounded a [different] person we call Casualty Four.”

And while the field is defined by assembling pieces of puzzles, journalistic eureka moments do still happen.

“It can be an adrenaline rush,” said the NYT’s Browne. Referring to his team’s report on the bombing of a Syrian hospital, he added: “I think the best was — and this was for two members of our team — when we heard a Russian pilot read out the coordinates for a hospital we were investigating. A source we were talking to had the audio of the Russian pilots, and we dug a little deeper, and someone had set up a system just like a police scanner. Then the pilot bombed those exact coordinates. 

“That cracked the case.”

Rowan Philp is a reporter for GIJN. Rowan was formerly chief reporter for South Africa’s Sunday Times. As a foreign correspondent, he has reported on news, politics, corruption, and conflict from more than two dozen countries around the world.