Have you ever wondered what goes into making an election forecast model? This week, we talked to Tom Saunders (@tomjs) about how he built his 2020 US Election forecast website, his top data tips, and how to profit from Nate Silver being immature online. Tom is a Financial Journalism postgraduate student at City University with an impressive sideline in data specific projects.
Hi Tom, thanks for talking to us! We loved your election forecast and wanted to dig into the process behind building it. First off, why did you decide to create this project?
It was during lockdown, and I really didn’t have anything else to do. I was interested in doing the same thing in 2016. But the difference is, after 2016, there were a few people who released the code for their models. And this year, The Economist did the same thing. So in 2016, I literally wouldn’t have known where to start, but with the code, and there were quite a few academic papers, I thought that there was a decent possibility that I could actually do it.
When you were going into building it, what were the aims; what did you want to get out of it?
I knew that I wasn’t going to be able to make something which sort of competed with Nate Silver or The Economist. The Economist model is part made by a guy who’s statistics Professor at Columbia, Nate Silver has been doing it since, you know, 2008. So it was more just exploring how these things work than competing, and I did some evaluation about how they performed afterwards and mine was the worst. It was mainly a learning exercise.
We’re still impressed. So once you had decided to build it, how did you then plan and approach it?
In my mind, there are basically four components to it, which I built one after the other. The first one was getting the poll data. The second is wrangling that into a, well, into data which you can input into a model. The third part, for me the hardest part, was making the model. Then after that, getting the results to a model, wrangling them back into something that is readable and then visualising. So I guess that’s maybe five. But it was a sequential one after the other, trying to work out how to do every single thing.
On the site, you mentioned that you had to get to grips with probabilities and the theory behind forecasting. What was that like?
The forecasting I did find quite difficult because, well, I read up a lot about Bayesian statistics, and there’s some really good books you can get online. But then the stuff they were using, these time series, are complicated and it’s quite hard to find an explanation for it which is simple. I did want to try and, rather than just copy someone’s work, to be able to make my own version and really understand what was going on. A lot of the academic papers would explain it, but then they wouldn’t have source code. You don’t usually put that in the paper, which is understandable. Or you have the source code, but you don’t actually understand what you’re seeing. Especially with the models, which are built in Stan with MCMC language (Markov chain Monte Carlo), there’s just not that much out there on how to use it and how it works. So it was a lot of learning piece by piece. It was literally like 80 lines of code so it shouldn’t have taken very long, but it took me forever because I really just didn’t know what I was doing.
What data did you decide to use in the end? And where were you getting it from?
So the polling data, I was quite lucky, I found The Economist has this Google Sheets which they use to add all the polling data to. So I got the link for that and I would just read that into my programs, I didn’t have to worry about scraping when you’ve had to find polling data. But I think with the US polling data there are websites like RealClearPolitics that do that for you. Apart from that, the other data was economic information, which is used to build a fundamentals forecast. You basically say “the economy’s like this, Trump’s the incumbent president”, etc. So that’s one part of the forecast. The only other thing was results from 2016 because in states which don’t have any polls you use the 2016 numbers more or less.
Did you have a live update as more polling information came in, and was that something that you could automate into the model?
I had a bash script which ran the R code, which would take the polling data, wrangle it, load it into a model, and then take the stuff out of the model, format it. And then everything was on GitHub. So what it would do is it would then upload the output data to GitHub, which will then automatically be visualising the site. Because the site was D3.js, it was basically reading the data directly off this site. So when data changed on the site, the website would change automatically. I had it so that the bash script would run, I think it was it was every sort of three or four days, and then in the run-up to the election it was every single day.
So then talking about the data visualisations that it was feeding into, how specifically did you create those? What decisions did you make around that?
It all uses D3.js, really because I’d never used it before and I just wanted to use it. There’s a lot more code to do something which in other libraries maybe you could do in a lot less, but you have complete control over what you do with it. I knew that I wanted a visual map with the states and how I saw them. I tried with each state, instead of giving out, Texas has, I don’t know, 55% of being Republican, to reduce to fractions. If I say I think Texas is a 55% chance of going Republican, the way of saying that makes it sound quite certain. But if you say there’s a one in two chance, then it’s the better way of saying it. And then for the individual states, I knew that I wanted to have the vote share with the confidence intervals and the polls over the top. Because I’d seen it in a paper before, but I’d also not seen many other models do a similar thing.
We like confidence intervals! Now for some quick fire questions on data journalism in general. How did you become aware of data journalism, and what attracts you to it?
I became aware at university because I was sort of into journalism, but I was into programming and numbers. So I was like, this just makes sense, because it’s both of those things. I think it’s quite interesting because it’s a new type of journalism, and it’s quite important. I think that data is something which is, I don’t know, I feel like most journalists don’t do it very well. And it’s also quite a big hole, in most newspapers.
Do you have any top Twitter follows to recommend?
There’s, I mean if you’re into modelling or anything around polling, then Nate Cohn, Dave Wasserman, G. Elliott Morris, and Nate Silver. And the thing with those ones, especially G. Elliott Morris – he’s the guy who runs The Economist forecast, and Nate Silver is FiveThirtyEight. They argue all the time on Twitter, like really very unprofessional. And I think actually Nate Silver ended up blocking G. Elliott Morris on Twitter. So it’s really childish, but, as a third party, they go into such in detail arguments that you actually learn so much about what they’re doing, and the different issues with each other’s model, which you wouldn’t understand if they were more sort of grown-up.
That is good of them! Anything else that you’ve read/watched/listened to, that you think is good on those subjects?
I have these books by Edward Tufte, they’re really good for data visualisation. But I would say that I personally think the best… Well, at least for me, I just like being part of The New York Times and New Statesman, which have these amazing data journalism teams, and just seeing what they’re doing. Once you can see what they’re doing, then you can really think about what else is possible.
Are there any underutilised data sources or sites that you think journalists could make more of?
I guess staying on the thing I’ve been looking at at the moment which is polls, is in the UK something journalists are awful at. I mean, there are websites which have polling numbers with great data for predictions. But every journalist will pick up the latest YouGov poll when it comes out every week and be like, “This YouGov poll shows Labour ahead by one point, so labour is winning”. When, actually, polls need to be an average and need to be contextualized. So I think that’s something which is underutilised by most people. You might have two polls come out in the same week, and they have different numbers. And most polls have a margin of error of say, plus or minus 2%. English journalists are a bit behind US journalists in the way we understand polls.
Any general resources or advice for getting into data journalism?
On the programming and data side, there are so many good resources out there to learn from. I used to use W3 schools. Datacamp is really good. I know that a lot of professional data journalists use R. But if I was going to give some advice, I’d probably say learn Python and then R. Because R is really, really good, but it’s not like a proper programming language. But if you learn Python, and then you want to go and learn D3.js and JavaScript, it’s the same basic fundamentals. It’s a lot easier to do. But yeah, there are so many, and Stack Overflow, which is this website where programmers ask questions, is so good.