Interhacktives sat down with Jeremy Tarling, senior data architect for the BBC, to find out what the world’s premier news organisation is doing in digital innovation.
What actually is a data architect?
To be frank, I think it’s a job title the BBC created. It’s a role that’s part information architect, so my job is firmly based in the online part of the BBC’s output, looking at how we best structure our news website to communicate with the audience. It’s also about looking at the underlying data model, and as we move away from the flat, article-based approach that BBC news has used for the last 17 years towards something a bit more modern, and dynamic and data driven. The role is all about how we make those connections, those links, between our content. How are different pieces of content related to the same storyline, or related to the same person, place, organisation, those kinds of things. Those are the two main areas: part IA, part data geek. It is looking at, not so much data journalism, but journalism itself as data. It’s looking at the sort of data that’s contained within the content we produce.
Do you mine for data in any way, or provide it?
Well, we have a project called ‘The Juicer’ which is a piece of software that we wrote that basically reads the news website, it scrapes it for content. We’re currently working on extending that to be able to process audio streams as well. What The Juicer does is it juices the news for people, places and organisations. Here’s how that works: It uses natural language processing to parse the text of a piece of content, an article on the BBC news website, and it will look out for, and will do some parts of speech recognition which is fairly standard. It’s a set of software called GATE. So the software parses the text and then uses something called dbpedia, which is a machine friendly version of Wikipedia, as it’s a dictionary of people, places, organisations. It looks for matches in any news content, and if it finds any then it annotates or tags that piece of news content with that person, that place, etc. What this lets us do is go to a database and say, “Give me the 20 most recent pieces of news that have been automatically tagged with Syria, or Hommes, or President Assad.” It’s analogous to what we do manually where we create an index around a particular topic or storyline, except this is being done by a machine so it’s massively scaled. We can pretty much generate an index on anything that’s been in the news and suggest to users as a topic they might be interested in. That’s probably the extent of our data mining – it’s looking at our own content. We have also started to apply that same approach to other news sources, so we’re also mining the Guardian news website and some local news sites, and looking for topics that are recurring locally, nationally, internationally in those news sources and seeing if we can build up any useful user experiences on top of that.
So it’s sort of like super-tagging?
It’s machine tagging, which presents its own problems. The key difference between machine tagging and human tagging is quality. There’s a very basic level where you can help a machine to understand the difference between Turkey the country and Turkey the bird. You can do that okay, you can create some sort of disambiguation rules. So, if we’re talking about Syria, Damascus and Istanbul, it’s probably not the bird. If the article mentions Bill Gates, Steve Jobs and Apple, it’s probably not the fruit. That’s the edge of how you can help a machine how to determine what kind of thing you’re talking about, but it doesn’t get you that far. If you take people with the same name, or places with the same name – in the UK there are 12 places called Newport – it can be quite a challenge for a machine whereas a journalist will know exactly which one the article is talking about. Hopefully we can provide journalists with tools to make that tagging as painless as possible, so if they were tagging a place they would see it on a map. Ideally what you want is a bit of both, the machine providing educated suggestions and the journalist simply saying ‘yes’ or ‘no.’
Is it a big project?
It’s run out of a project called BBC News Labs, which is part of Future Media. It’s a small operation, it’s a guy called Matt Shearer and myself part time and a couple of folks. We explore these sorts of technologies: text mining, journalism as data, something called newshacking. There’s a newsHACK event coming up soon, in fact, in which we work with universities and other news organisations to see what future formats news might turn into given the massive amounts of data that are becoming available. I was reading something this morning by Jeff Jarvis, and he talks about news as a service and the fact that the whole industry model is flawed when it comes to the internet. We can no longer assume 100% of users are seeing 100% of advertisements, instead we need to start thinking about what’s important to people on the internet: convenience, and speed. That’s exactly the kind of thing News Labs looks into.
News Labs is part of BBC R&D, and the idea of it being a separate operation from the main cut and thrust of daily news production is so that we can do these kinds of investigations without getting to bogged down in supporting the website. With any big organisation, there’s a tendency for everyone just trying keep the ship afloat and not do any R&D.
Will this tagging system boost SEO or is it just for intra-BBC purposes?
It should boost SEO but I’m wary of talking about that. It’s something of a black art, and when you’re spending pubic money you have to be careful with the black arts. I expect that if we start producing indexes around topics, and a good internal link structure that reflects that fact, that search engines will follow those links and will see that. If you take local news, we currently provide 58 news indexes – Birmingham and Black Country being one that covers a massive area of the country, the second most populous of the country after London. What we’ll have with this tagging approach is, instead of having this one giant area that covers 10 or 15 million people, we’ll tag our content with places that fall within that area – Wolverhampton, Dudley, etc. Each of those will be available as an index, a much more granular approach to news. You can say that you’re interested in following these places as topics.
Provided the BBC can continue to push the big stuff – the breaking news and BBC trending as well – then I think this could be really good service, as well as a boost to SEO. The BBC always does well anyway, partly because of Google’s secret sauce – I think they think of us as an authority – and partly because of inbound links.
The other thing that we’re looking at to help with SEO is embedded semantic markup: it’s a way of adding some code to the underlying source code of a web page. So, Google and a few other search engines use a site called schema.org and what that does is describe a vocabulary of metadata that if you embed in your page then it will help search engines to understand what your webpage is about. Search engines are great at pattern matching, but they’re not so great with meaning – it’s the Turkey problem again.
How far along is this Juicer project?
Well, it started about 12 months ago, and has now got 100,000 pieces of news content in it. Hopefully, we’re going to try and make some public APIs so developers and journalists can play around with it and build their own software on top of it. That would be a really nice thing to do.