Facebook’s Safety Check: what is it, and is it helping?

London suffered its first, large-scale terrorist attack since the 7/7 bombings this week. Tragically, three innocent lives were taken, and the attacker was shot and killed by the police.

At the time, no one knew if another attack was on its way, or if this was a one-off. Amid the confusion, it was clear that London was facing a crisis. And Facebook was quick to respond, activating their Safety Check feature for the first time in the UK.

The communication system was introduced by Mark Zuckerberg in October 2014 “to serve everyone in the world”, though it was designed with Japan’s devastating 2011 earthquake and tsunami in mind.

In the two and a half years since its implementation, the Safety Check feature has been activated 35 times, giving users near an incident the chance to ‘mark’ themselves safe for their friends and family to see.

It could not have been anticipated that just seven of the activations would turn out to be for natural disasters, with most of the other crises ‘man-made’.

In fact, the majority of system activations were catalysed by some sort of attack, often terror-related, with shootings, bombings, and even hand grenades causing the feature to be activated.

The chart below shows how many times Safety Check has appeared after different types of disaster.

The ‘other’ category includes miscellaneous events such as a building collapsing in Tel Aviv, Israel, and a train crash in New Jersey, US in September 2016.

77% of system activations took place in 2016 alone. As journalists, we’re not supposed to speculate, but this figure certainly lends power to the theory that 2016 was one of the worst years in history.

It’s not uncommon for events that are not globally-recognised to trigger the safety check. It was activated after a fire in Massachusetts in December last year. No one was hurt, but the community were concerned after 60 people were displaced.

Ultimately, Safety Check is a community tool, and it’s no surprise Facebook wants to err on the side of caution when it comes to declaring a crisis.

The social network has been criticised, however, for its system’s excitability. This week, the Guardian’s Tim Burrows claimed Facebook only created “another avenue for fear” by activating the Safety Check during the Westminster Attacks because it was confined to a small area.

As well as informing the user which friends have checked in as safe, the system also lists those who haven’t, potentially evoking further worry.

But this is the nature of a crisis situation: events are ultimately unpredictable. And in a crisis, it is not better to have the option of reassurance?

Alberto Cairo: To succeed in data journalism, steal from the best

Alberto Cairo thinks you can solve almost all your problems with data journalism in two steps.

“Journalists,” he tells me, “in my opinion, tend to oversimplify matters quite a lot.

“Let’s say that you are exploring the average height of people in an area. If you only report the average, that may be wrong or right depending on how spread out the data is. If all people are more or less around that height then reporting that average is correct.

“But if you have a huge range of heights, the average is still the same but you may not be reporting how wide the spread is. Or if your distribution is bimodal, the average will still be the same, but you have a cluster of short people on one end and a cluster of tall people on the other end, that’s a feature of the data that will go unnoticed if you only report the averages.”

His other complaints about journalists attempting to write about data will be familiar to despairing members of data desks in newsrooms around the world. Speculative extrapolation, inferring causation from correlation, a lack of understanding of probability and uncertainty, and many other journalistic foibles all fall under Cairo’s fierce scrutiny.

“The thing is,” he insists, “all the things I’m mentioning are easily solved, so it’s not that you need to take a course on advanced correlation and regression analysis, it’s just a matter of learning Stats 101 and then read two, three books on quantitative thinking and quantitative reasoning.

“That’s how you avoid 80 per cent or 90 per cent of the problems, and the other 10, 20 per cent will be avoided if you consult with experts every time you do a story based on data, which is something we need to systematically do.”

Even most students of data journalism probably can’t say that they’ve read two or three books on quantitative thinking and quantitative reasoning, but if we’re serious about our pursuit of the truth, perhaps next time a disgraced politician publishes their memoirs we should Google ‘best books on statistics’ instead.

A decent place to start

Though Cairo is of course prolific in data visualisation in his own right, he is clearly a teacher at heart. After two decades working in infographics and data visualisation, he could be forgiven for losing an ounce of enthusiasm, but the Knight Chair in Visual Journalism and author of two books on data visualisation still has a twinkle in his eye as we discuss his recent work.

He’s just finished teaching an online course on using data visualisation as more than just a method of communication. Instead, he is focusing on using it “to find stories in data”.

“There’s a whole branch of statistics,” he explains, “which was defined around the 60s, 70s and the 80s by a statistician called John Tukey.

“He wrote a book titled Exploratory Data Analysis. The whole field of data visualisation in computer science and statistics focuses mostly not on communication, it focuses on exploration, how to explore data, how to discover features of the data that may remain unnoticed if you don’t visualize them.”

Alberto Cairo is a member of the jury for this year’s Data Journalism Awards. Unfortunately he won’t give me a direct road map to victory, but for students hoping to enter this year (the first year in which a student category has been included), his advice is surely invaluable.

“Steal from the best.

“This is advice I give my students every semester: we all learn by copying other people. By copying I don’t mean plagiarising, but getting inspiration. Look at work from ProPublica or the Washington Post or The New York Times, and copy their style, copy their structure, copy the way they present information.

“Don’t try to think of graphics as if they were embellishments to your story, but as analytical tools and communication tools within your story. They should never be afterthoughts when you’re developing your story. They’re an integral part of your story and an integral part of its communicative power.”

Even with his experience, Cairo says, he still does this himself. Though many journalists seem addicted to credit, and are unlikely ever to admit to anything short of completely original works of genius, in data journalism, collaboration is endemic.

“Nobody works inside of a cocoon,” notes Cairo. “The community of data reporters and investigative reporters is very open. I just came back from the NICAR conference, and some of my students who attended were amazed that they could approach – I don’t know – Scott Klein from ProPublica and ask him questions directly. They believe there’s some sort of hierarchy, but there’s not.”

This lack of hierarchy should lend confidence to aspiring data journalists. To slightly amend Alberto Cairo’s steps to success: all one need do is get a decent grounding in statistics, consult with experts, and join the data journalism ecosystem.

How to make the perfect podcast

Starting a podcast can be a daunting prospect, but if ‘Data Day’ can teach you anything, it’s that the barrier for entry is incredibly low.

On this final episode of ‘Data Day’, Bridie Pearson-Jones joins Luke Barratt as they discuss what makes some of their favourite podcasts great, why podcasting is such a compelling format for modern journalists, and the difference between podcasts and radio programmes.

Plus, special guest and longtime fan of the show Faye White joins the team to discuss some funny podcasts, because apparently our podcasting experts are really boring.

This is the end of season 1 of Data Day, and Luke’s last episode. It remains to be seen whether future Interhacktives take up the mantle.

Facebook Live polls are great, here’s how to make one

reactions

You’re a dog person, let’s face it, and It’s because you appreciate a pooch that you share a Facebook Live poll asking users to indicate their preference, between cats or dogs, with a heart or a like.

And why not? Facebook is for sharing. Live is the big push, and these polls are interactive, simple enough to produce for news organisations, and have been well received by the general public.

Like any social innovation, there were teething problems when media outlets first picked it up. Polls were lazy, gimmicky and irritatingly ever-present on newsfeeds.

Facebook identified a problem and they were banned. Matt Navarra, head of content for tech site The Next Web reacted:

After some reflection, Facebook disagreed (or bowed to public pressure) and reinstated the technique – with a caveat or two. They include this direction on their branding resources page.

facebook reaction rules

Interested?

Before you think about making one, consider the guidelines of use outlined above. Make sure your video follows this simple criteria

  1. Don’t just make a poll. Reactions can’t be the ‘most prominent feature’ so add another layer. Stream something important for people to vote on, like a news conference, or report the news first and foremost, using voting to supplement what you’re doing.
  2. Don’t use simple visuals. Add a live video element, or work on what’s shown in the background. The rules aren’t written, but use the medium to its fullest.
  3. No associating reactions irresponsibly. An angry face for one candidate and a heart for another, during a debate? That’s not democracy.

Want to make one yourself?

You should. Here’s a quick guide on getting started – using OBS studio, and an open source tool for creating polls. There are a few caveats though.

  1. Streaming video is intensive. Your computer may suffer and, even with the video settings used in this guide, your machine may splutter a little.
  2. The tool used to make the polls is a shortcut. It’s quick and easy, but customization options are left wanting. The creative freedom to make some more impressive interactives will require a knowledge of the Facebook API and some developing skills.

But this will do. Here are the basic steps. If you want a TLDR click here.

Step 1: Download and prepare your tools.

We’ll be using the following in this guide:

  • OBS studio. Open Broadcast Software will send the video content you create on your desktop to your Facebook live stream and is the command center through which you’ll operate the controls. Download here.
  • Your image creating software of choice. Be it Photoshop, or open-source alternative GIMP, you’ll use this to create any background assets you may wish to include.
  • This online tool for creating Facebook polls. This is an unofficial generator, created by Hayden Ryan, and will connect to our live video.
  • Might also be worth opening Facebook, too. You’ll need to be broadcasting live from a page and not a personal profile.
Step 2: Make your assets

For this guide, let’s add some interactive elements to POTUS’ congressional address. We’ll broadcast the speech (a live replay stream, but could just as well be actually live), add some data visualisation, or statistics, on the side, and ask viewers to vote for their preferred spending priority below the video. A little like this:

mock up of live video

Using Photoshop/GIMP, design your background asset – considering the video player size and ensuring everything is nice and visible. Facebook recommends an aspect ratio of 16 x 9. Here’s one we made earlier. 

first asset: background image

You’ll also need the link to your video, which you can copy into OBS. The broadcast software will let you add multiple assets, so if you wanted to move images around during the stream, add them as different sources.

Step 3: Generate your video and your polls.

Now it’s time to create your video and get your streaming keys. Head over to your Facebook Page and visit ‘publishing tool’. Press the button to post a live video.

where to find live button

You’ll be presented with this window.

getting your stream key

You’ll need to input the Server URL and Stream Key into OBS, so take a note of these. It’s also advisable to take down the number underlined in red in the above image. This is the Video ID and is used to connect the poll to the video.

To make the poll, head to this link and complete the form, making sure to copy and paste the right ID into the first box.

The form is relatively customisation: you can select polls with two, three or six reactions. If you need four options, simply make two different polls of two. Set the background colour to white and OBS will convert to transparent.

polling form for live

When you’re finished tweaking, take a note of the poll ID, copying the address presented so you can add it into OBS.

getting your poll id

Step 4: Prepare OBS and tweak your video settings.

Head over to OBS and start arranging your workspace. The studio version of the software adds some handy tweaks, allowing you to copy transitions to make sure assets are lined up, and the ability to work on a scene privately whilst the broadcast is up and running.

You’ll need to be familiar with two different concepts.

Scenes are your workspaces, where your different video setups will be saved. Sources are your assets, and can be video links, feeds of web pages and still images from your desktop.

Add a new scene and name it as you wish, then click the plus by the source box to add your background image.

opening obs and adding image

adding browser source to obs

Follow the wizard and position the background to fill the video preview. Add your poll through the same steps, selected ‘Browser Source’ instead of image and adding the URL for the poll. Position your poll next to the voting instructions.

arranging scenes in obs

In this example, we’re using a live video feed. For live, use another ‘Browser Source’ and crop the window to fit the video. Fiddle around with the transformation until your scene is to your liking. If it’s not a live video, use ‘Media Source’ and select the local file.

adding the video to our live

If everything’s moving, it’s almost time to go. The final step is to make sure your video settings are right.

Hit ‘settings’ in OBS, head to stream, select ‘Facebook Live’ from the drop-down and add the stream key we saved earlier.

The video settings we use will be dependent on your own computing capacity, and how much your setup can handle. Use this guide from OBS, lower the bitrate and set the CPU usage to ‘ultrafast’ if your machine isn’t so punchy.

Step 5: Start streaming, enjoy democracy.

Once you’re happy with the settings and the look of your stream, hit start streaming and jump back to Facebook. The stream will load and you’re ready to go!

posting the video on facebook

TLDR
1) Download OBS and load up the poll making tool.
2) Make your assets with photoshop etc.
3) Start a FB live on your FB page, through publishing tools.
4) Make your polls using the online generator, and save the link.
5) Set up OBS, adding the sources we’ve made
6) Get your video settings right, and hit stream.

You can now make a very simple Facebook Live voting interactive. Have a go and tweet @interhacktives with your attempts, we would love to see.

The secrets of Facebook Live

In this week’s podcast, Luke Barratt and Jasper Pickering talk about the growing importance of live-streaming on Facebook for media organisations and the average Joe alike. As Facebook grows its video platform, Zuckerburg is pushing users to interact with each other via live video feed.

No longer are the broadcasters telling us what to do from atop their ivory towers. Now you (yes, YOU!) can produce live coverage from the comfort of your handheld device.

The intrepid duo tackle examples of live-streamed news like Donald Trump’s disastrous press conference and the suspense of watching a watermelon explode under the pressure of a thousand elastic bands on Buzzfeed. While future coverage will pale in comparison, users are still becoming more engaged with online videos.

Gone are the days of panda sneezes and laughing babies. Now audiences demand more from their social media influencers, as outlets like Vice produce high quality documentaries that can be watched from the comfort of our bed/toilet.

Our cup runneth over.

CNN Trump Press Conference.

A stressed watermelon that forgot it had a test today.

Vice videos

Interview with Eliot Higgins of Bellingcat

Verifying missile launchers, tracking down ISIS supporters and holding worldwide governments to account is just a day’s work for 36-year-old Eliot Higgins.

Last time I met Higgins, an independent intelligence analyst, he was giving a talk about his work with Bellingcat, the investigative news network he founded in 2012. It was this network that trawled the internet’s vast and polluted reservoir of publicly accessible material to track down the Russian-owned missile-launcher that took down the Malaysia Airlines Flight 17 over Ukraine in 2014.

This time, it’s me doing the tracking, struggling to find Higgins on the hectic roundabout at Old Street station. I eventually spot him standing next to a telephone booth squinting through his glasses, in a Matrix-style black coat.

We only have fifteen minutes but there’s an excitement in Higgins’ eyes as he talks about his work, while easing into his chair. Ordering a Coke, he laughs at how he’s been trying to avoid caffeine.

Bellingcat formed after Higgins’ personal blog, Brown Moses, attracted huge attention as he was able to uncover war atrocities, such as the use of cluster bombs, from the comfort of his home in Leicester. It was the readily available nature of open source tools that prompted Higgins to start Bellingcat and form a network where others were able to learn how to use such tools in their own investigations.

I ask him what his proudest moment is with Bellingcat since his formation. He screws up his face a little, “Hiring people is a lot of fun”, he says. “If it wasn’t for what we did, we would have had this whole narrative of [the] Russian government [claiming to intervene in Syria only to fight ISIS, and not prop up President Bashar al-Assad’s regime] that wouldn’t have been challenged,” he explained.. “And, you know, there are families involved who are being lied to by the Russian government and without us, there would have been no push back.”

“For me, a lot of what we do is about accountability and justice and working with international organisations on that.”

Though the website states it is by and for “citizen investigative journalists”, and many news outlets, including the Financial Times, call its founder a “citizen journalist”, Higgins himself is uneasy about the label.

Shuffling in his seat, he explains: “It’s not citizen journalism. It’s not just about conflict or journalism. It’s about all kinds of different areas. From my perspective, the work we do is not about journalism: it’s about the research and getting it [the findings and tools] to people that can actually do something with it.”

“For me, a lot of what we do is about accountability and justice and working with international organisations on that.”

While Higgins wants to distance Bellingcat from being purely journalistic, the network’s handful of contributors definitely shares a hack’s mind-set, utilising publicly available tools, such as Google Earth and social media, to investigate atrocities abroad.

Credit: SKUP 2015, Marius Nyheim Kristoffersen

Three years after the network shot to fame by solving the MH17 mystery, it now covers all corners of the Earth and is fast becoming a force to be reckoned with. This was made clear last November, when Higgins quashed the Russian government’s denials over the bombing of a hospital in Syria. By comparing satellite and on-ground photographs from 2014 to 2016, he was able to show specific areas that were in fact damaged by bombing.

Bellingcat also drew huge media attention after using social media to track down ISIS supporters. Most recently investigators used archived Facebook profile and geo-located social media photos to hunt the Berlin Christmas market attack suspect.

“We thought it would be impossible. Within an hour we had the exact location”

When I ask him about how Bellingcat uses social media in their investigations, he blushes, admitting that they recently caused a “minor panic” in Holland, after the network asked its Twitter followers to geo-locate a photograph found on an online community consisting of ISIS supporters. He laughs, shaking his head as he notices my eyes widen: “It’s nothing urgent or scary. We had one photograph [and] we just wanted to know where it was because it looked like it was in Europe. So we put it out on Twitter, asking if people could help geo-locate it.

“We thought it would be impossible. Within an hour we had the exact location: in a holiday park in Holland. The police showed up at the holiday park and the poor manager had to come out in the middle of the night.”

This brings our conversation to online privacy, as I noted that day he asked his 49k strong Twitter followers about Donald Trump. He says, with a cheeky glint in his eye: “My Twitter page looks like I do a lot online. But if I’m away, I won’t share when I’m actually away. If I post a picture of my time abroad it’s often a week after I’ve actually been there.”

He adds, laughing: It amazes me that people keep their Instagram profiles public. Who needs likes that much?”

I keep my own settings to myself as he stands up to leave, shaking my hand and plonking the Coke can on the table. At that point, I sadly decide it’s time to change my Instagram settings to private.

BBC ‘Trafficked’ producers: ‘VR is laid bare, you can’t hide’

trafficked from bbc

The idea of virtual reality isn’t new: Sega made a headset in 1991. VR as a storytelling device, however, is picking up traction in newsrooms and becoming more complex as the medium is explored. How many journalists can say they’ve had a bite of the cherry?

Interhacktives spoke to Charlie Newland and Owain Rich, Producers at the BBC and the filmmakers behind “Trafficked”, BBC World Service’s first foray into virtual reality. The eight-minute film introduces viewers to Maria – a single mother trafficked from Nicaragua to Mexico, and forced into the sex trade.

In an interview with the Interhacktives, Charlie and Owain discussed the story’s inceptions, whether the technical skills they have gained are essential for working journalists, and the first thing everyone says when they remove their headset and return to reality.

What came first: the story or the idea of doing something with VR?

O: The journalist that brought the story to us was from Mexico. VR is so new that certain stories work and certain stories don’t, and we’re trying to learn the rules of it. We wanted to find a personal story from a female perspective, and find something appropriate. [Filming in VR is] different to a standard filmmaking, where you’re going out with a camera and you get an idea of what’s actually possible: your levels of access, how you’d actually shoot everything. All these ideas pop into your head and you get an idea of what you’re making as you film it.

With VR, however, we didn’t know where the trap doors would be.

“With traditional filmmaking you can choose different camera angles, you can cover shots with audio, you can show the curtain blowing in the breeze, then cut to the door handle etc. But with VR you’re seeing all things around you at all times.”

C: We had a few different ideas. Considering the subject matter with “Trafficked,” it was really tricky because we couldn’t actually show anything explicit. We had lots of ideas before we could choose what we went ahead with and we were still conditioned by what we had access to, so we had to find the best avenue and to drill down on the best way to film

O: We were slightly overwhelmed with the idea of telling Maria’s story. How are you going to get a sense of her whole perspective? Who are you? Are you Maria? Are you a third person? It’s just an endless field of options.

C: You really have to pick where you focus the attention: Is it the fact she’s transported around [or] all of the eight years she suffered through? You try everything out and eventually you see what works.

O: The other difficult thing is choosing what to show and what not to show. Basically, there were certain things that were too graphic to show. With traditional filmmaking, you can choose different camera angles, you can cover shots with audio, you can show the curtain blowing in the breeze, then cut to the door handle etc. But with VR you’re seeing all things around you at all times.

C: VR is laid bare. You can’t hide.

How do you convey the seriousness of the subject best, in VR?

O: We didn’t need to show that much to get that sense of menace, and to connect to people emotionally. We don’t show any violence beyond Maria being struck near the end.

C: Which is what got us our “over 18” certification.

O: We could have shown all sorts of things, but we choose to have the guy just getting slightly too close for comfort [The first scene of the film introduces you to one of the traffickers.]

C: It was the one thing, given the medium, that we could play with: having someone breaking into your personal space, so we really exaggerated his presence.

O: That’s one of the challenges of VR: what to show, what not to show. We’ve got loads of options and the rules aren’t really written. We mocked up scenes, blocked out scenes, and tried lots of different scenes. But in the end, we had to scale things back.

C: We had a dream sequence scene for Maria panned for the start too, and were experimenting with 360 audio, but to tell the story we didn’t need that much. You don’t need to make it super complicated. We did a workshop before we started, and what came out was that it needed to be something that people wanted to watch. It’s not for entertainment, as such, but it needed to be informative. We ran it as an installation — it needed to be an intro to the subject.

“I don’t know whether it [VR] will go mainstream the way that people hope, but I think we would like to be there –  just because of how it connects you on an emotional level.”

How important are the typical journalistic values when filmmaking in VR, with regards to ethics and accuracy?

O: Some of the dialogue was rewritten from the transcript to fit the timeframe, but the actual content of what is shown is based on pure fact. We conducted rigorous interviews with Maria herself several times, as well as people involved with her rescue and people who worked on the trafficking route. We had a lot of contextual information to work with.

O:  Essentially, what we were making was a dramatised film based on real accounts, so in terms of the narrative, everything was there.

C: Even down to Maria being hit on the head, it was a dramatised version. We gathered as much reference material as possible. When you’re creating something, you don’t want to embellish the facts to make things seem more dramatic and dismal, but the film is still a stylised version of events, factually accurate.

How do you see the industry progressing, and what do you hope your role would be? In an ideal world, would newsrooms invest in these kinds of things?

C: Definitely — I think [VR] offers a new perspective. We forensically recreated a scene and you can walk around, and the emotion combined with the interactivity is definitely something we, as an industry, should be looking at. But it’s all down to the budget.

O: The strategy has lots of pockets of people experimenting. As a filmmaker, anything that gives me a chance to connect with an audience in a different way, that gives a different perspective, makes me pretty excited. I don’t know whether VR will go mainstream the way that people hope, but I think we would like to be there —  just because of how it connects you on an emotional level.

Image: BBC Trafficked https://www.youtube.com/watch?v=5jBPqveBdfk

How long did “Trafficked” take from start to finish?

C: About three months.

O: In terms of actual hours, it was probably much more than that. it was like six months work in the space of three.

C: The size of the operation and the work needed really depends on the story. If you’re thinking more short form, you can just create an environment, add a voiceover and have a little bit of interactivity. It is quite a scalable medium. Do you want to make a 60-second piece when the viewer has to rig up the whole headset to watch? You do earn an appreciation for both the pitfalls and the shortcuts.

“It’s not about having the coders in one corner and the journalists in another — everyone will have to meld together to make the best stuff.”

What was the feedback like at the installation?

O: People are often most surprised about the emotional aspect.

C: We had a 15 year old girl and a 60-year-old man experiencing the film together, and they were able to take the headset off and have a conversation at the installation. That’s really something. I don’t think there’s been something people have said repetitively, just that it was unexpected.

Have you gained any technical skills during the process? How important are these skills to journalists?

O: If VR does go mainstream, at least a cursory overlapping of knowledge with developers.The gaming industry is a huge industry – the tech has come out of kid’s bedrooms and into newsrooms.

When we started working with the Unreal Engine [ a tool used to create video games] we had to go pretty deep. It’s a whole other world of knowledge. There’s one leap to video, and another to interactive content.

O: And it is the future. It’s not about having the coders in one corner and the journalists in another — everyone will have to meld together to make the best stuff.

C: But should journalists be coming out of school with all the skills to make big VR projects? It’s certainly possible, but I think that it’s still quite unusual. You don’t have to be instantly trained in 3D modelling and sound design, but having an awareness of these skills will help you out in conversations with the developer.

You can read more about the ‘Trafficked’ project here.

This interview has been edited and condensed for clarity.

Email newsletters for journalists: a guide

Despite numerous technological innovations — from 360-degree video to social media live-streaming to robot journalists — the trusty old email seems to be increasing in importance in the newsroom.

Quartz, for instance, has several journalists based across the world work on its daily newsletter, which releases at about 6am across time zones in Asia, Europe/Africa and the Americas, every day.

This shouldn’t come as a surprise. The email inbox is, for many people, the first app or webpage they open in the morning and one that they return to multiple times over the day. Amid the noise and barrage of social media posts, the email newsletter may be the easiest and quickest way to reach a reader directly.

In the podcast, we compare our favourite email briefs, such as the Times Red Box and Politico Morning Media tipsheet, and why we like them. We also discuss the ethics of email newslettering.

If getting access to a reader’s email inbox is like gaining a private audience with him or her, how much should publications use it for their own marketing or political purposes? If publications, political parties and companies alike use people’s information for political or marketing purposes, does this count as an abuse of big data and reader’s trust?

Listen to the podcast to find out. Don’t forget to sign up to Interhacktives’ newsletter here, too.

Using data in sport journalism

In this week’s Data Day, Luke Barratt is joined by Matteo Moschella to discuss the use of data in sport journalism.

Data is omnipresent in the reporting of sport, particularly recently. The closing of the Barclays Premier League January transfer window has prompted a glut of visualisations on the month’s top stories.

Check out some of the code used by the Guardian on their Github.

Athletes and sports teams are using more and more data nowadays to optimise their performance. But crucially for journalists, the vast audiences drawn by sports demand extensive data.

Opta provides detailed data feeds on a number of different sports.

While providing this data to users in raw format is common, there is also great scope for journalists to use data to analyse issues in sport.

Here, Rob Minto uses data to defend a potential increase of the number of teams taking part in the FIFA World Cup.

One crucial area where this kind of journalism has flourished is in predictions. Nate Silver, now renowned as a polling expert, made his name using data to predict the results of baseball games.

Visit his site, FiveThirtyEight, which still applies its methods to sport.

Similarly, the Financial Times has built a complicated statistical model to predict the outcome of the 2016/17 Premier League.

Daniel Finkelstein has a weekly column in The Times using similar methods to analyse football. Here, he uses sport to teach his readers a lesson about probability through a parable about the likelihood of giant-killing in the FA Cup.

We’ve also seen data used for in-depth investigations into sporting issues. Buzzfeed used data from betting markets to uncover indications that certain players had been guilty of match-fixing.

The Sunday Times, meanwhile, in a more traditional piece of data journalism, made use of data from a whistleblower to find evidence of doping throughout the world of athletics.

How to improve your social videos to tell compelling stories

As Facebook founder Mark Zuckerberg told investors of his ‘video first’ strategy, content creators are trying to find ways to optimise their video output and attract larger followings.

Social video has been around for many years but is now  considered the dominant vessel for consumption on social media. Last year, Facebook video uploads increased by 95% from the previous year and these numbers look set to rise again in 2017.

Here are a few tips on how you can improve the quality and watchability of your videos on Facebook.

Keep them short and sweet

Even though Mark Zuckerberg himself has expressed an interest in opening up Facebook to longform and episodic videos, he wants to focus on shorter-form content just to start. While you might be excited to produce an expensive Pulitzer-winning documentary, start small. The optimum length for social videos is between 30 – 90 seconds. Don’t worry too much if your video is slightly over. Use your own discretion to figure out what works for you.

Work without sound

Now, you don’t have to go full Buster Keaton when making a video for social, but make sure that your video still makes sense without audio and doesn’t become just a sequence of footage without context. Most viewers who come across your video will do so because it played automatically. If they’re interested enough, they might turn the sound on to find out more. Use captions to let viewers know what the video is about and use subtitles if subjects are talking so people can still ‘hear’ what is being said. The captions should be able to drive the story without breaking the flow of the video.  

Think about your first Impressions

For many people, your social presence will be your first port-of-call, so you want that first impression to stand out. Come up with something succinct that doesn’t give too much away to the viewer. You want them to stay with you to the end of the video but you also don’t want to bore them.

Avoid using still images/stock photos

When I first started making videos for social, I was told to avoid using still images. “If the story can be told with images then tell it with images.” In other words, video should only be used if the story can’t be told in any other way. If you absolutely must use an image for a video, then try to create the illusion of movement with zooms and pans. This is known as the ‘Ken Burns Effect’ and it’s a widely used technique. You will often see it in war documentaries to create the illusion of a battle, for example. You might also need to use photos or screenshots that others have taken to tell your story (we will get to that later on).

Be professional

This might sound obvious but treat your social video exactly the same as you would any news piece: as professional as possible. If your audio isn’t properly synced or you’ve captured all of your footage on your old Nokia then people will be turned off and go to the next item. You don’t need to invest in a lot of equipment to achieve this. All you need to do is take extra care. Make sure that your audio matches what’s being said on screen, remember to adjust focus, and keep your camera steady.

Ask for permission to use other people’s photos/videos

A common problem with social video is that it can be easily downloaded and uploaded on another channel without giving credit to the original author. This is known as ‘freebooting’ and it is heavily frowned upon. If you want to use footage that you’ve found from another source to help tell your story, try and contact the author and they may be happy to let you use it as long as you give them credit in the video. Some people might say no,  so you’ll need to find something similar elsewhere.

Avoid cliches

This. Just this.

 

The Bureau of Investigative Journalism’s Megan Lucero: it’s time for data journalism to go local

Megan Lucero has seen it all. The former head of data at The Times and The Sunday Times is now directing her attention to local data journalism as the head of the Local Data Lab at the Bureau of Investigative Journalism. In a (second) discussion with the Interhacktives, Megan talks about her decision to leave The Times, how she envisions the Lab’s future, the importance of collaborative data journalism at a local level.

“I left purely from the idea of it [the Local Data Lab] is really what journalism needs and what data journalism should be contributing to,” she says.

“I left the Times really happy with how we got on there,” she continues. “We brought data investigations into the heart of the newsroom. We went from being a sort of Excel help-desk to actually being integrated into news investigations, big-time front pages […] I was very happy to leave knowing that I was leaving a really strong legacy. But I left because I believe that this is really, really important”.

Megan is referring to the Local Data Lab, an arm of the Bureau of Investigative Journalism, who have set out to “fill the voids” after many newsrooms cut provisions for investigative journalism in the wake of the 2008 economic crisis. “Investigative journalism is expensive: it takes time it takes a lot of people and a lot of [economical] resources,” she explains.

Local newspapers bore the brunt of these cuts, so the “idea was to try to help to solve that,” Megan continues. The Lab will focus on data journalism, which she believes is a form of investigative journalism at its core. For her, data journalism hinges on “the idea that you harness data to find stories, and using data to find stories is in itself journalism.”

Data journalism is an exercise in finding stories in the large amount of data that the digitised world offers today:the journalist has to be able to swim in a sea of information,” Megan says. “In order to do that, there is often technical innovation that is needed, and that’s where I see the [data journalism] coming in. The computational method — the means in which to programmatically query and build databases, automate the process — all of that comes together in digital journalism.”

One of the Lab’s more challenging tasks, Megan says, will be to bring data journalism to the local level: “It’s an ambitious challenge —it’s a really daunting one — but I absolutely [think] that’s right where my next step had to be.”

“There is a time and moment to listen to our communities and to find out what it is that each member of those communities is trying to say,” she says.

Megan maintains that the introduction of computer-assisted journalism to regional newsrooms will not affect regional reporters: “We want to make sure that we are not trying to put local journalists out. A lot of local journalists use the Office of National Statistics or data.gov.uk We are not going to try to change that because that would potentially harm their jobs,” she says. “We are after the gaps in the industry, right? Local reporters don’t have the means, the time or the resources to do the computational work […] My team essentially would be coming in to try and provide this”.

Megan insists that the new Lab will focus on unearthing local stories and issues with the help of regional journalists: “There is a time and moment to listen to our communities and to find out what it is that each member of those communities is trying to say,” she says.

How will she carry out this goal? “I am going to listen more than I am going to talk,” she explains. “What are the stories that need to be told at a local level? What are the stories they want to tell? What are the datasets that are not open? What are the challenges to covering local beats?”

She explains that the Lab’s role will empower local newsrooms, and stresses the need for transparency and accountability at a local government level. A failure of local governments to provide information and data surrounding its work is “a problem for democracy and it’s a problem for the free press unless we address it”.

She then discusses her vision for the Lab: “I don’t want to be too prescriptive at this stage, but my goal is to find datasets that are not in the public domain. Already we have a few stories that we want to be bringing out: things that NGOs, charity groups [and]activists have obtained via Freedom of Information requests, a lot of datasets that haven’t actually been brought out nationally or locally. We are going to merge lots of datasets to find analyses that maybe we hadn’t [found before].”

But searching for datasets is not her sole objective, she says. The Lab also hopes to change the way local stories are told and highlighted in the media: instead of having national newspapers dictate the media agenda of the day from the top down, local news outlets will shine: “Our idea is to put the power directly into the local stories, so every dataset that we worked on has to scale on a national [newspaper]”.

The Lab will work together with local newsrooms by collaborating on local issues identified by the partner and discovering whether it might be a story suitable for a national audience. They will do that by using resources not typically available for the local newsroom (it could be complicated datasets that can’t be accessed from a local newspaper, or advanced analysis techniques): “So the idea is that we would scale it [for national coverage]”.

The Lab and the local newspaper will then break the story together, one (the Bureau) as a national news outlet and the other locally. Cooperation is a fundamental journalistic value for Megan: there is little space for competition as “it’s very unlikely that anyone will ever scoop you or steal your story out of the back of it,” she says.

Megan says the creation of the Lab was inspired also by other major nonprofit newsrooms, such as ProPublica. ”Their Electionland project, their data lab, their data stories — they are everything that we were looking to do”, she explains. The Bureau of Investigative Journalism frequently communicated with Scott Klein, ProPublica’s editor, while it was developing the Lab.

Finally, the Lab comes together in one role: “the idea of independent journalism breaking down important complex datasets to very small levels, that’s where we are hoping to do”.

Does the idea of working for the Data Lab interest you? The Bureau will be accepting candidacies until the 1st of February, you can find applications details on Megan’s Medium.

Correction: February 2, 2017
An earlier version of this article misspelled the name of the Lab. It is the Local Data Lab, not the Local News Lab. Some other amendments were done to the quotes in accordance with Megan Lucero.

Why you shouldn’t make a 360° video podcast

360° video is an emerging format and, like all emerging formats, must go through an uncertain period of experimentation. As journalists begin to push the boundaries for this storytelling device, the Interhacktives discuss the merits, challenges and funny side of all things 360°.

It has to be seen to be believed.

In order to fully enjoy this 360° experience, strap on a cardboard viewer and be transported into the studio with Interhacktives podcast team. This week we discuss the BBC podcast pilot ‘No Small Talk,’ as well as our favourite examples of 360° journalism.

Check out Within, who provide a virtual reality experience that has the feel of the magazine. FIlms include trips into the deep blue to hear the clicks and whistles of pacific sealife.

You’ll find the Flint police ride-along film on the New York Times Virtual Reality site, alongside many more docs and experiences.

Finally, check out the RT 360° app and be transported to space every time you open your phone.

Follow Luke on Twitter at @lukewbarratt, follow Ryan at @ryanleewatts, and follow Interhacktives at @Interhacktives.

You can also find this podcast and previous episodes on iTunes and on Soundcloud.

Thanks to Podington Bear for our theme song, ‘Am-Trans’.

3 ways journalists can deal with Donald Trump’s ‘alternative facts’

White House press secretary Sean Spicer appeared on Fox News on Tuesday evening, comparing his “alternative facts” regarding Donald Trump’s inauguration crowd to reading different weather reports.

“The press was trying to make it seem like we were ignoring the facts when the facts are that sometimes… in the same way you can look at a weather report,” Spicer said. “One weather report comes out and says it’s going to be cloudy and the next one says there’s going to be light rain. No one lied to you.”

Spicer had told a press conference that the number of people who had attended and tuned into Trump’s was “the largest audience ever to witness an inauguration, period” — despite multiple side-by-side shots of Trump’s 2017 and Obama’s 2009 inauguration crowds indicating otherwise.

Spicer justified this claim by citing public transport rider statistics, which, he said, indicated that more people travelled on underground trains in Washington on Trump’s inauguration day than on Obama’s (WMATA, Washington’s public transport provider, later tweeted figures that proved him wrong).

Obviously, telling untruths to the public and reading differing weather reports are apples and oranges. Weather reports concern uncertain events in the future, whereas Spicer was reflecting on a past event for which there were photographs and viewership ratings.

With this in mind, here are three useful tips for journalists who wish to produce verified reports amid the proliferation of fake news.

1. Remember that not all data is gospel
The New York Times’ forecast on the night of the US elections.
The New York Times’ forecast on the night of the US elections. (Source: The New York Times)

A key reason the EU referendum and US election came as such a surprise was because journalists and pundits had, quite simply, misused data.

Forecasters, such as FiveThirtyEight, The New York Times Upshot and HuffPost Data, had put Hillary Clinton’s chances of winning the US election between 70 to 99 percent — because many of them had relied too heavily on opinion polls, and forecasters had failed to explain fully the concept of margin of error.

In Spicer’s case, it seems obvious that WMATA data on rider statistics cannot be enough to prove that Trump had the largest crowd in the history of inauguration ceremonies, even if the numbers he stated were true.

What if more people had chosen to take the underground on Inauguration Day because half of the city had shut off its roads for the ceremony? And, even if those people had gone to Trump’s inauguration that day, where did the thousands more people photographed attending Obama’s inauguration come from?

2. If a news source says something fishy, find another one
Donald Trump signs executive orders surrounded by his nearest and dearest.
Donald Trump signs executive orders surrounded by his nearest and dearest. (Source: Karl-Ludwig Poggemann via Flickr)

Journalists are arbiters of truth, not political mouthpieces. Reporting claims by the White House, 10 Downing Street or any other powers-that-be is insufficient. If official sources won’t provide truthful quotes, journalists should feel free to punish them by ignoring the quotes and going elsewhere for the truth.

After Trump invited Sheri Dillon, a self-appointed federal tax lawyer to justify the then-president-elect’s plans to solve conflicts of interest over his business conglomerate, The New York Times invited government ethics experts to rebut Dillon’s remarks. The experts, which included former White House ethics lawyers and the current director of the Office of Government Ethics, found at least 15 flaws in her argument.

CNN also refused to air a White House press conference one day after Trump’s inauguration, revealing the cable network’s misgivings about broadcasting false statements to its viewers.

3. Call out untruths

On 2 January, veteran US journalist Dan Rather published a lengthy Facebook post calling for journalists to call out the Trump administration’s untruths as bald-faced lies. As the media industry blog Mediaite noted, major publications don’t tend to refer to inaccurate statements as lies, but rather as “unsubstantiated claims” or similarly euphemistic language.

Many major news outlets have also adhered to this line of thinking. Some have even gone as far as referencing Trump’s lies in their headlines, perhaps as a reflection of a 2016 study that estimated that 59 percent of URLs shared on Twitter had never been clicked.

On 21 January, The Huffington Post published a headline reading “Trump And His Press Secretary Flagrantly Lied On Their First Full Day In Office. That Matters.” A few days later, CNN published a column by Dean Obeidallah, a political satirist, titled “Dear team Trump, ‘alternative facts’ are lies.” The New York Times, on the same day, published a headline that referred to Trump’s statements as “lies”.

And, in perhaps the boldest headline of all, New York Magazine published a story back in December investigating Trump’s “War Against Facts”.

Of course, there has been wide debate among major news outlets regarding this new journalistic policy. Earlier this year, Wall Street Journal editor-in-chief Gerard Baker advocated against referring to Trump’s statements as “lies”, noting:

To refrain from labeling leaders’ statements as lies is to support an unrelenting but not omniscient press, one that trusts readers’ judgments rather than presenting judgments to them. If we routinely make these kinds of judgments, readers would start to see our inevitably selective use of a moral censure as partisanship… We must be seen to be objective to continue to earn our readers’ trust.

A screenshot of New York Magazine's headline, "Donald Trump's War Against Facts".
A screenshot of New York Magazine’s headline, “Donald Trump’s War Against Facts”.

Ultimately, it’s up to individual journalists — and subeditors — what language they should use to frame statements that just aren’t true. But whatever you choose to do, it’s increasingly important to call out fake news and hoaxes, and to implement rigorous verification practices to do so. To quote Rather:

A lie, is a lie, is a lie. Journalism, as I was taught it, is a process of getting as close to some valid version of the truth as is humanly possible. And one of my definitions of news is information that the powerful don’t want you to know.

In other words, it’s time to call a spade a spade.

Featured image by Gage Skidmore via Flickr under Creative Commons licence (original image adapted)

How to scrape tweets using R for journalists

What’s a pirate’s favourite programming language and statistical computing environment? Why, it’s R, of course. Jokes aside, R is the language of choice for data miners, scrapers and visualisers – as well as journalists looking to manipulate datasets that Excel just can’t handle.  

R can be used to create stunning and modern interactives, plot cats instead of points in weird feline data visualizations, and it can help journalists scrape data from Twitter.

Twitter data has the potential to inspire important stories. Blogger David Robinson analysed the tweets of President Donald Trump to find that tactful posts sent from an Iphone were composed by his staff, and those angrier messages from Android were typed by Trump himself.

Journalists need a tool to filter tweets and to find trends among them, R helps by grabbing that data and making it usable.

In this guide we’ll be getting set up with Rstudio on Windows, an open-source program for working with R, and we will learn  the basics of twitter scraping. This is a basic how-to, with little assumed knowledge, so should hopefully translate for OSX users too, with a few tweaks. Let’s get started:

Note: If you want this guide distilled into 24 words, head to the TLDR at the bottom of the page and just follow the links to download what you need. If you have the patience, read ahead for more detailed instruction.

Step 1: Prep, downloads and installing R

You’ll firstly need to gather your tools. Head here and download the latest R package, currently R-3.3.2, and install it to your computer. You’ll also need to download Rstudio, the software we’ll be working within.

Your final download: save this script to your computer. R can use scripts (basically text files) to save commands and save you having to type them out every time.

Once you’ve followed the installation wizard, open up Rstudio to be greeted by this nice blank canvas.

Step 2: Open R and load your script

You want your screen to be divided into four sections. On the screenshot above, you can see the console panel on the left: which shows the code that you run, like a timeline of what you’ve done so far. You also have the Environment panel, with your list of elements, databases and variables (currently empty), on the right and, in the bottom right, a simple file manager.

Now, press the folder button under ‘edit’ in the main menu. Alternatively, press Ctrl-O.

Navigate to where you saved the script we downloaded earlier, and open it up in Rstudio. The program will now show  a panel for scripts in the top left.

Your script is loaded, and everything you need to migrate tweets from the internet to a spreadsheet are in the top right of your screen.

Before we start scraping, let’s make sure Twitter lets us in when we knock.

Step 3: Getting your Twitter access.

To do this, we’ll need access codes from Twitter. You’ll need to head to: apps.twitter.com and create your own application (A Twitter application in this sense is just a way of connecting to the API. Hit the ‘create new app’ button and fill in the form. This is purely for personal access to your own twitter credentials, so fill in the fields with your info.

After that’s completed, head to the ‘Keys and Access Tokens’ tab in the menu of your new app, and copy the four codes you find into R.

These are the Consumer Key and the Secret Consumer Key, and the Access Token and Secret Access Token.

Once these four strings of text and numbers have been copied into the R script you downloaded, you’re good to go and can follow each stage of the script until you have the data you need.

Step 4: Running and merging data

There are three stages to the actual process of grabbing data from Twitter. These are:

  1. Loading the packages you need.
  2. Running the code to access Twitter.
  3. Searching tweets and saving them to file.

The first time you attempt this process, however, you’ll need to install the packages you plan to use. On the script you downloaded this is flagged as step 0 and by highlighting this and pressing Ctrl-R, you’ll install everything needed for twitter scraping.

install.packages("stringr")
install.packages("twitteR")
install.packages("purrr")
install.packages("tidytext")
install.packages("dplyr")
install.packages("tidyr")
install.packages("lubridate")
install.packages("scales")
install.packages("broom")
install.packages("ggplot2")

You only need to do this the first time you attempt a twitter scrape, and can jump to step 1 in all subsequent attempts.

Mini-Step 1: Load your packages.

Those shiny new packages you added to your R setup now need to be loaded so that you can use commands associated with them. Run this:

library(stringr)
library(twitteR)
library(purrr)
library(tidytext)
library(dplyr)
library(tidyr)
library(lubridate)
library(scales)
library(broom)
library(ggplot2)

Mini-Step 2: Access Twitter.

With your own personal codes copied into the script, you can run the following few lines. (Don’t forget to save your script so you won’t need to repeat the copy/paste process next time around. Your codes will remain the same unless you generate new ones:)

consumerKey = "INSERT KEY HERE"
consumerSecret = "INSERT SECRET KEY HERE"
accessToken = "INSERT TOKEN HERE"
accessSecret = "INSERT SECRET TOKEN HERE"
options(httr_oauth_cache=TRUE)

setup_twitter_oauth(consumer_key = consumerKey, consumer_secret = consumerSecret,
access_token = accessToken, access_secret = accessSecret)

Mini-Step 3: Searching and scraping (the fun part)

The script we’re using gives you the options to search for three different things (parts 3.1, 3.2 and 3.3). You’re able to search for the last 3200 tweets of any individual account. You can search for the last 3200 tweets to use a hashtag of your choosing. Finally, you can search for the last 3200 tweets directed to a certain user aka tweets ‘@ed’ to someone else.

In each instance, add your chosen phrasing to the lines that contain the search terms, and follow it through, updating the variable names as you go.

To best demonstrate this, here are some examples:

To create a list of Barack Obama’s tweets sent whilst he had the POTUS handle. Use this:

obamatweets<- userTimeline("potus44", n = 3200)
obamatweets_df <- tbl_df(map_df(obamatweets, as.data.frame))
write.csv(obamatweets_df, "obamatweets.csv")

The function, “userTimeline”, adds the tweets of a user of your choice to the database. In this instance, the handle is POTUS44 and is written between the speech marks, and the first word on the line is the name of the value where the tweets will go.

The next line sends them to a database, and the final line writes that database to file.

To create a list of tweets containing a certain hashtag, use this: 

yeswecan <- searchTwitter("#yeswecan exclude:retweets", n=3200)
yeswecan_df <- tbl_df(map_df(yeswecan, as.data.frame))
write.csv(yeswecan_df, "yeswecan.csv")

To create a list of tweets sent to a user, use this:

tweetstoobama <- searchTwitter("@potus44 exclude:retweets", n=3200)
tweetstoobama_df <- tbl_df(map_df(tweetstoobama, as.data.frame))
write.csv(futureexwife_df, "tweetstoobama.csv")
Step 5: Your finished sheet and where to go next

Head to the the folder where you saved your CSV file and open her up. Congratulations, you have successfully scraped your desired tweets. What you do next is up to you.

One idea is to count the instances where a word appears. Download this sheet containing a template for totalling some key words, and check out David Robinson’s guide to running a sentiment analysis on your newly collated data.

TLDR
  1. Download R, Rstudio and this script.
  2. Get your Twitter access by creating an app here.
  3. Open Rstudio and load the script
  4. Follow the instructions

If you try this out for yourself, make sure to let us know. Tweet @Interhacktives (or me, @ryanleewatts) with whatever you find, we would love to see. Happy scraping!

4 social videos from Iraq’s battle with ISIS

Snapchat and Semtex might sound like something of an oxymoron. However, as the bombs and mortars continue to rain down in the battle for Mosul, journalists covering the conflict are taking to social media to document the final days of a war that has so far cost hundreds of thousands of lives.

Now, mobile phone chargers, hard drives and selfie-sticks are as germane to the intrepid foreign correspondent as the flak jackets and combat helmets they wear.

This collision has led to social media platforms becoming home to some of the most powerful and hard-hitting footage of the conflict, with Western and Middle Eastern journalists capturing some key moments.

Here are a few video highlights from some of the brave, intrepid journalists covering the fight for Mosul.

Josie Ensor – The Bells of Bartella

Recorded by the Daily Telegraph’s Beirut-based Middle East Correspondent, Josie Ensor was one of the first western journalists to enter the town of Bartella after it was liberated by the Iraqi Golden Division from Isis.
The town, which had been under ISIS control since August 2014, had once been home to 30,000 people, the majority of them Assyrian Christians. However, when the Iraqi Golden Division captured the town, Ensor was one of the first journalists to report on the assault, and her footage of its church bells ringing went viral.

 

Ayman Ogyhanna – Entry into Mosul

Embedded with Iraqi Special Forces, freelance video journalist Ayman Oghanna kept filming even as the vehicle he was travelling in came under sustained RPG and machine gun fire to provide one of the first reports from a Western journalist to make into the city after it had been liberated from 444 years of ISIS occupation. Love the beard too.

 

Iraqi Popular Mobilization Units – Reunited families

The conflict in Iraq has displaced more than filmed by the Iraqi Popular Mobilization Units – this footage captured the moment that an estranged family was finally reunited after almost two years apart.

 

Danny Gold – Celebrating fighters

The LadBible may not be best known for its foreign coverage, but these images were filmed by its intrepid roving correspondent, Danny Gold, on the outskirts of Mosul. Depicting celebrating Iraqi fighters, it’s becoming something of a viral hit.

Have your say on government open data

On this week’s ‘Data Day’, Ayushman Basu and Luke Barratt discuss the opening of a survey for journalists by the Government Statistical Survey. The Government is looking for feedback on how to improve their provision of open data.

You can respond to the survey here.

The main focus of the survey is on the possibility of creating a single outlet for releasing data from the government, and on this podcast, we discuss some of the inconvenience of the current system. Datasets have to be sourced from various different portals and subsequently combined, which creates significant delays for journalists.

The survey is not especially focused on data quality, but we discuss the importance of this issue, which is made more serious by the worrying fact that the government has no centralised policy on data quality.

Finally, since Ayushman Basu has specific experience in this area, we discuss how some of these issues present themselves in India. The government there has a central data portal, but the quality of releases is very poor, with PDFs often used instead of Excel spreadsheets. India’s large population also makes data collection very difficult.

India’s government data portal can be viewed here.

6 lessons for digital journalism from the New York Times’ strategy report

The New York Times has released a report, written by a group of seven of their journalists called ‘The 2020 Group’, outlining “the newsroom’s strategy and aspirations”.

This is part of the paper’s ambition to dramatically increase its number of subscribers and build a true digital business by 2020.

For any journalist hoping to work for the Times, or even just embody its values in their own journalism, there are some crucial lessons to be drawn here.

1. We must defeat the single block of text

Despite the leaps and bounds that have been made in attitudes towards digital journalism, the report acknowledges that too many articles on the website are still “dominated by long strings of text”.

Indeed, it notes that only 12.1 per cent of stories include deliberately placed visual elements. With the proliferation of digital tools available to the modern-day journalist, there is no excuse for this lackadaisical approach to storytelling.

A journalist ought to care about the readability of their copy, and few things encourage readers to click away more than an intimidating wall of text with nothing to break it up. One reader, the report says, mocked the Times for not including in a story about subway routes even a simple map of the disputed train line.

You don’t even need a picture to break up text — a block quote will do the job.

The report bemoans the lack of expertise in this regard in the Times newsroom. Aspiring journalists should perhaps take note, and take it upon themselves to build these skills.

2. Digital means constant innovation

The potential of digital techniques for distributing journalism is huge, and yet the instincts of many journalists are still overwhelmingly traditional.

Most reporters seem focused on writing up and filing 300-word pieces to the exclusion of all else. But if a journalist’s focus is on their story, their choice of medium should not be so rigid.

Instead, we should allow the medium to serve the story. This is the thinking that has led to successful podcasts, email newsletters, and social video.

Multimedia, then, should not just be part of a young journalist’s arsenal, but should be the primary way they think about stories, always with an eye out for new formats.

3. Expert knowledge is digital gold dust

In the Internet Age, experts abound. If I want to find an authoritative voice on a subject, any subject, I can do it fairly easily. What is more, that voice is probably on Twitter, and doesn’t even need me to find it.

This development of human interaction means that journalists can no longer get away with rough knowledge of a topic, or the social media reaction will be brutal.

With that in mind, the report recommends that the Times hire more journalists with a high level of authority on a specific beat. This has been reflected already in the hiring of an editor to cover gender issues.

Therefore, an aspiring journalist should find a beat they can become an expert in, and make it their own. Whether that means keeping up with healthcare journals, following the stock market, or reading every policy briefing on the environment, being a leader in one field is now a key task for any journalist.

Note: This man is not Prime Minister.

4. The paywall is working

The report trumpets the success of the Times’ digital subscription model, with a graph indicating that their revenue from consumers is growing, and continuing to exceed advertising revenue.

How much this indicates the value of subscribers and how much it merely reflects the decline of advertising is an open question, but it is clear that about a third of the Times’ lost advertising revenue has been replaced by subscription fees.

Not only that, but the model helps to fight the fall in advertising fees. “Advertisers,” the report explains, “crave engagement: readers who linger on content and return repeatedly.”

The path to creating engaged readers is very different to the modern urge to create clicks and pageviews. A reader who feels betrayed by a headline they perceive as clickbait is less likely to return.

This is good news. Journalists now have a reason beyond their own professional integrity to pursue quality, and an effective retort to an editor complaining that their story isn’t attention-grabbing enough.

5. It’s time to think about success

The availability of detailed and sophisticated analytics of audience engagement, pageviews, and so on, means that success can be defined and vigorously pursued.

Pageviews, of course, should not be the defining metric, but it is important to have a sense of important analytics.

The Times report encourages news desks to set themselves tangible goals, so that journalists know what success looks like, and can pursue that.

This is not about making journalism subservient to business, but about creating content that gets read: effective journalism.

6. We should all learn to code

Good visuals don’t just come from nowhere, and presenting a story well online is sometimes about more than just an embedded YouTube video.

The report boasts that the Times has more journalists who can code than in any other newsroom in the world.

There will be some journalists who don’t understand why this is something to boast about. These journalists are still creating articles dominated by big blocks of text.

It may be intimidating, but perhaps the time has come for journalism students to sacrifice the time spent learning shorthand and get some lessons in coding.

It is 2017, after all.

The Coral Project’s Sydette Harry: ‘People feel that journalism doesn’t seem to listen’

We love the Internet because it’s a treasure chest of information, and a place where we can join groups to communicate with people who share our interests, no matter how niche the subject or how scattered about the world we might be.

As journalists, social media and online communities are places to discover new stories and trends, find experts and learn more about our readers.

But in the age of political division, trolling and fake news, we can also get frustrated on the Internet sometimes because open platforms — like Facebook and Twitter when left on public settings — can leave us vulnerable to abuse and hatred that which we never signed up for.

With this dichotomy in mind, what should be the relationship between journalism and community? Interhacktives’ Alexandra Ma chatted with Sydette Harry, community lead at the Coral Project, a joint collaboration by Mozilla, the Washington Post and New York Times to provide open source tools for newsrooms to engage with their readers.

Why do you think communities that allow comments are important for journalism?

Our focus isn’t that every community allows comments, but that some sort of feedback, some sort of interaction is necessary for good journalism. That can be comments. That’s important because it gives you a way to get a more complete and continuous relationship with your readers.

What we are constantly thinking about is how journalism now, more than ever, needs everyone, and good journalism needs to be open and transparent to people, and needs to be verified. The way you do that is by having a continuous dialogue and discourse.

Now that can be comments — we would like it if it were comments — but most importantly, we would like people to really consider what it means to have a community and to plan on it. Because too often people say, “we will let social do it” but they don’t also say, “what does ‘let social do it’ mean?”

What can these other interactions between journalists and readers be, and how can journalists learn from that?

Some of the great research fun that I have is talking to people about the different ways they connect. There is one website I love called Bitter Southerner, which publishes content dedicated to the South [of the US]. They don’t have comments, but you can become a card-carrying “Bitter Southerner”. You can pay to support their journalism and have meetups, get discounts with southern artisanal makers and concerts, and get books — and they have no comments. But it makes being part of the community a tactile thing that focuses on your interests. So if you were concerned enough to be a Bitter Southerner, you get to participate in southern crafts and southern concerts.

It’s not always in comments, but the journalism is supported. The community is created. And with that, people tell them things. People think about things and go, “I’m going to go here first because even if it’s not comments, I know that these people have invested in making connections with me. When I have a story, I talk to people because it’s obvious in the way they have set up the connection to their business. They care about what I think.”

Also, comments aren’t always the best way to get information from people. Marginalised populations, specifically women and people of colour, don’t like comments because comments have historically been so awful and racist. But they will respond more directly to direct solicitations: “Tell me about an experience of racism you’ve had.” “Tell me about your mother’s favourite recipe.” “Tell me about your immigrant stories.” “Tell me about what you are excited about college.”

People will notice suddenly they have so many more comments, so many more interactions. They will even get people to say: “I don’t comment but I don’t mind talking to you.” It’s about opening up the ways and letting people know that you are open to the ways you want to talk to them.

“Why are there so many layers between you [journalists] and your readers?”

Why don’t you think journalists can rely on social media to get feedback and interact more with their readers?

Think about the way we do social media, even as journalists. Sometimes we’re thinking out loud, sometimes we’re super directed, sometimes we might be angry. A lot of that is on social, where we may not always be in our most linear of thoughts and focused. And that’s fine — that’s what people use social for.

But how do you start connecting your readers to each other around similar topics? A lot of that on social now has been things that readers have modified social for, like hashtags. Hashtags were something readers developed to be able to follow conversation and this is all information that lives on social. These important things aren’t on your site. They’re not a relationship you’re building with your community — they are things that you are going to a third party to see and then bringing it back to your platform because your readers are commenting on things that they found on your platform.

Why are there so many layers between you and your readers? Is that what you actually want? Your data is also on a third-party platform — it should be your data. It’s your content. And even though it’s on a third-party platform — Twitter or Facebook or whatever the third-party platform is — readers’ opinions of their experiences on these platforms is transferred to journalism. It’s transferred to the newspaper or the website or the network that they are talking and interacting with. And that’s the kind of relationship we think you [journalists] should own.

The Coral Project’s products, as outlined on its website. (Credit: Coral Project)

 

The tools of Coral are “Trust”, “Ask” and “Talk”. They handle what we think are three very important sections of community building.

[Editor’s note: Check out the Coral Project’s website and public Trello board for more information about its tools and goals.

Not everybody should have comments. But we want to improve tools that will allow you to get to the core of it, which is: how do you honestly and transparently represent and provide good journalism, and get good feedback and integrate that into journalism as part of a growing and continued relationship with the community you claim or with the audience you are searching for?

 

What is the relationship between journalists and readers like now?  What would you want to change?

I think it depends on the site, it depends on who you are, it depends on what you feel. I know a lot of people feel that sometimes journalism only shows up when they’re having the worst moment of their lives.

I’m an immigrant from a very tiny South American country, Guyana. People don’t often talk about it, and when they talk about it, it’s usually, “Hey, it’s flooding.” Or that there are lots of deportations, or corruption, or something like that, rather than “oh it’s a beautiful, we have a nearly-1,000-year history and it’s geographically biodiverse, and we have had communities in New York, Philly and Canada for quite some time.”

Journalism doesn’t show up for those things — it shows up for the horror. And people often feel that journalism doesn’t seem to listen. Journalists will say, “I never ever read the comments,” and some of them have perfectly good reasons. Comments have been terrible to them. If you are a person of colour or a woman journalist, comments in some places are horrific for you. They’re utterly horrific and not reading them does you a form of self-preservation.

“A lot of people feel that sometimes journalism only shows up when they’re having the worst moment of their lives.”

On the other hand, there are people who have fallen under fantastic communities from their comments, who have gotten book deals, who have been able to help people with healthcare, who have been able to help people with legal aid, who have supported funerals from the comments.

I remember that a friend introduced me to Bitter Southerner. She’s from Atlanta, I am not Southern in any way, shape or form. I am a first-generation American, so a lot of the South is not personally [related] to me, but I like the way the community [writes], and when they said “hey, you have to pay or we may not survive,” I paid. It was worth it to me to sustain the model.

The community, monetarily, can sustain you. It can also allow — when members are interested and willing to contribute — for different types of fascinating journalism. Bitter Southerner did a wonderful piece about coal refuse that I don’t think I’ve seen anywhere else. They did pieces about the origin of hot fried chicken or music streets that were mostly supported by their community.

One of the big things that we’ve been talking about post-election is the creation of echo chambers and filter bubbles. Is that a risk that comes with building communities?

A filter bubble isn’t a community.

When you think of communities, and when you think of them outside of journalism, we think of them in our own lives. They don’t consist of like-minded people: they’re bonded people, they’re people who have chosen or have, by circumstance, are together, but that doesn’t necessarily make them all the same. One of the big jokes I always make about communities is: think of your last big family dinner.

How well did that go? You’ve got maybe 30 minutes before the lifelong battle between Auntie M and Auntie G came up again because both of them had a glass of wine at decided they wanted to have that fight again. You would prefer that they not have the fight. Somebody will separate them before dishes go flying, and everyone’s doing their job. But it’s reality.

We don’t have to sit here and have discussions where everyone agrees. We do have to have discussions where people’s humanity is respected. We read everything here at Coral. How horribly people talk about each other! Depending on what side of the spectrum you’re on, [the reaction] goes from, “well that was really mean,” to “that is some of the most dehumanising, racist, homophobic language I have ever seen.” And communities, I think, should be for the expression of views where humanity is respected.

“One of the big jokes I always make about communities is: think of your last big family dinner.”

I have my political views. If anyone Googles me for more than two seconds, you can pretty much figure out my political views. But when I step into a journalistic community, I know what is expected of me to behave or how I’m supposed to behave, and I know that the person, even if we are diametrically opposed, is being held to the same expectations. That will be enforced, and the person who has to do that [the moderator] has the tools to do that without harming themselves. And we can present and interact with, at our own will and desire, the sections of our community that best represent that.

Too often, when people talk about the Internet or making a better Internet, they talk about making a “nicer” or “more civil” Internet. I think it’s a good position to have. There are some amazing civil comments doing some really good work with comments.

But I always feel that, for certain spaces, it’s not about whether or not we are civil to each other, or [whether we] necessarily agree on everything. It’s that we know what we expect and can control our experiences.

“A person has a right to be racist. A person has a right to be awful. They do not have a right to make me listen to it.”

The problems with harassment, when it tips over, is that I can no longer control my experiences. I don’t want to talk to this person, but your platform won’t let me not talk to this person. I don’t want this person to see me, but you’re preventing me from saying that. I don’t think this is a real person, I think this is a bot, and I’m doing more work in finding that out than you are.

A person has a right to be racist. A person has a right to be awful. They do not have a right to make me listen to it. And a platform has a right to be racist. They have the right to be for one group only. But they have to be clear about that, and they have to be direct with that. No person should go into a platform expecting one thing and being promised one thing, and getting something completely different, often to the terms of abuse, and not have a way to address that and not have the platform stand by that.

So if I tell you, “we’re not going to have this language,” even if it’s just a social contract and not legally binding, I’ve made you a promise. I should keep that promise. And if I don’t keep that promise, there should be a way for us to talk about why it didn’t happen. That, I think is community — less than “we all have the same filter bubbles.”

Filter bubbles come from the place where we stop trying to develop ways to talk to each other at all. Nobody has figured out who’s supposed to step into that void of “we’re gonna have to talk to each other at some point or we have to at least come to agree on basic facts.”

The Coral Project’s end goal users, as outlined on its website. (Credit: Coral Project)

I have also received pretty bad and scarring comments, so I appreciate the Coral Project’s aims.

I’m a Twitter veteran and some of my harassment has made it into national and international news. It’s really trippy and it’s not fun. I think it’s a thing that we could do better at protecting against. I’m a very large free speech advocate.

I don’t like it when people are banned for speaking what they believe, or saying what they say. I will spend the rest of my life at the top of my lungs, and possibly throw hands if necessary, to fight them about it — but they have a right.

Too often, the idea of “we’re going to push it onto Facebook” or “we’re going to push it onto social” is less about protecting or developing good spaces for conversations, but more about being “I’m not the one responsible for this one. Good luck.”

We are responsible. We are the people who have said: “This is what we want to do: we want to tell the world about itself.” So we have to tell the world about itself truthfully. But we can do that without causing random [access] harm to everyone, and usually to the most marginalised. I believe we can. But I also am very famous for being overly hopeful.

“We are the people who have said: ‘We want to tell the world about itself.'”

What is the journalist’s role in online communities? Are they community members, are they also contributing, are they asking the questions? What’s their role?

Journalists are all sorts of things. People are using our tools, which is very exciting. In one of our tools, created with Bocoup, you can choose an emoticon and one thing you want the president-elect to concentrate on. And they have been getting good with that.

We [Coral Project] have a community online, we make newsletters, we go to conferences and we talk to people. We also counsel. In trying to build around a community, we also hope to form a community of people who are like, “you know what, we want to talk about this. Usually people don’t think it’s important but we’re going to think it’s important.”

There are some journalists who look at us and say, “you are very sweet, I’m never going to use this.” And that’s fine. That is OK. But we want that to be a discussion we’re having, and not just a quiet thing where we’re going, “comments are terrible, we’re not going to do anything about it” or “comments are terrible, we’re not going to talk about how they got that way.”

Journalism is so important, now more than ever. We want to work toward bringing people back to interacting and trusting journalism, because they know that this is part of their lives.

The Coral Project outlines its goals and needs on its public Trello board. (Credit: Coral Project/Trello)

Do you think journalists are losing trust in their readers?

I think readers are losing trust in us [journalists]. We have numbers on it. They don’t trust us. And part of that is because they don’t know us, or they don’t know what we do.

There are ways for us to be connected to our readers and inform our readers and do the intelligent and vital work of journalism without being so disconnected. People are like, “once I give people what they want, I’ll have to give them simple, bad journalism.” I don’t think that’s true. I think you can give people a connection so that they trust the journalism you give without having to dumb it down.

This interview has been edited and condensed for clarity.

TOP IMAGE: The Coral Project distributed this sticker at their workshop at MozFest 2016.

Have a very data Christmas

This is Interhacktives’ latest attempt to persuade you that data journalism can be relatable and human, and this time we’ve teamed up with a powerful ally: Christmas.

Christmas is a time of year for turkey, mince pies, stuffing, stockings, trees, treats, presents, and… data? On this week’s Data Day, James Somper and Luke Barratt look through the news to find some of their favourite examples of data-driven Christmas journalism.

Luke made his mince pie joke again, but this time you don’t have to wait until the end to hear it.

Follow James on Twitter @jsoomper, Luke @lukewbarratt, and Interhacktives @Interhacktives.

You can read Kate Hughes’ article on the true cost of Christmas here. The Money Editor of the Independent counts up our rising festive spending, and comes up with some eye-popping numbers.

The financial services company PNC has done what it’s been doing every year for the past 30 years, and calculated how much the presents in the ‘Twelve Days of Christmas’ would actually cost you. Partridges are cheaper this year, but what about pear trees?

Finally, Anjana Ahuja has a rather more serious story for the Financial Times about the mounting evidence that the vast quantities of alcohol consumed every Christmas are having a very serious effect on our physical health.

Thanks to Podington Bear for our theme song, ‘Am-Trans’.

Interview with Louise Crow of WhatDoTheyKnow

Sending Freedom of Information (FOI) requests can be a daunting job for UK journalists. From the early stages of drafting your request, the risk of not getting the information you need can feel like a minefield.

However, thanks to WhatDoTheyKnow (WDTK), this task has been made a little easier. WDTK, an open platform website powered by MySociety, a charity aimed at promoting online democracy, helps users with requests and makes past requests publicly available.

In this interview, Interhacktives’ Ella Wilks-Harper speaks with Louise Crow, MySociety’s senior developer, about the future of FOIs, government secrecy and open data. Crow opens up about the drawbacks of open data and how governments are releasing large quantities of information that are not of high quality. However, with the help of WhatDoTheyKnow, technology is playing a growing role in helping ordinary people become more data literate; a positive step in holding organisations to account.

How did you get involved in MySociety?

About seven years ago, I came back from living and working in the States, and was looking for something useful to do. MySociety was advertising for people with coding and a certain amount of time, and I had both of them.

louise-500x500
Louise Crow courtesy of MySociety

Do you think there are limitations of FOIs when people do not have the skills to analyse the returned information or data?

FOIs are used by lots of different kinds of people. I think one of the benefits of having the [government’s] responses [to FOI requests] responses in the open is that anyone can come and interpret the information that has been released. A lot more information is going out into the open.

David Cameron had a vision that your everyday person without any coding skills —  otherwise known as an “armchair auditor” — could hold the public sector to account. Do you think they exist?

Almost anything exists. I guess a more important question is whether they exist in significant amounts to hold power to account. I think that’s always going to be difficult, and any solution that is based on data only is going to be a naïve solution. You have to have a look at the nature of power to try and put checks on power, if anyone is to have a chance at holding power to account.

Technology can help. I think what we do at MySociety tries to help the little shift in that balance by making the technology available to ordinary people very good. But I don’t think it can be the only solution.

Journalists like yourself, and having a strong and free press, are a strong opposition to government. All of that stuff has to work together.

What are the other solutions?

I guess it is the scope of civil societies. Journalists like yourself, and having a strong and free press, are a strong opposition to government. All of that stuff has to work together.

Have you noticed any chances in the handling of FOIs and open data in the last five years?

It is hard to say as I have only be working on this project since 2012. Within the last five years, I think the difficulty with open data at the moment or one difficulty with open data at the moment is that it is not driven by demand.

If you look at FOI and open data together, open data is what authorities think people want to know — or, in a more cynical interpretation, what they want to release.  So that maybe they [public bodies] don’t like to release data, so it’s very old or they like to release success stories so it’s a specific subset.

Technology can help. I think what we do at MySociety tries to help the little shift in that balance by making the technology available to ordinary people very good.

Do you think FOIs is a stepping-stone for greater transparency?

I think FOIs are definitely one positive step, and the right to access laws* is a really heartening step and to the extent to which technology can help. I don’t kid myself that it’s the only part, but I think that’s worthwhile in terms of transparency.

*[This right of subject access means that you can make a request under the Data Protection Act to any organisation processing your personal data.]

Do you think there is a movement in helping people become more data literate and be more engaged with the vast amounts of data out there?

I think that is one of the big challenges of the modern age: information overload and information provenance. We have seen in political happenings in the last few months this question of [how] you can get lots of information but whether it is of high quality is hard to find out. Google and big platforms have a big role to play there.

Nick Mark Ian Louise and Matthew over laptops(1)
Louise Crow courtesy of MySociety

At MozFest you spoke about being neutral in your position working in MySociety. Can you expand?

I think a very practical answer to that question is MySociety is a charity. As a charity, we are legally obliged to be politically neutral. That’s kind of what right to information is all about. It is about something that applies for everyone, whether you agree with them or not.

And that has to be a principle across all the rights [right to know, right to information and press freedom] , because life changes and lots of people are going to be using them. Like freedom of speech, it can’t just be something when you say something that I like to hear.

And finally, do you send out FOIs yourself?

You know, I have never sent out one. To be honest, there has never been something yet where I’ve felt like I really need to know this. [But] I have certainly benefited from reading other information that people have requested.