Traditionally, parsing data in threat intelligence has involved the use of a team of experts that, according to Kodama, “have a very specific focus on your industry, or your geography or your technology”.

“You would have firms that specialised in a very specific thing,” he says. “They might specialise in a particular type of fraud scheme, for example if they had expertise in a particular way of laundering money that primarily would appeal to law enforcement, but it might have application also for the banks that work closely with that.”

Each team a company would consult with would have what was termed a ‘problem set’, essentially an area of expertise, and it would be up to companies to build a network of such experts with problem sets relevant to their business area.

However, this rather old-school approach has been turned on its head by companies such as Recorded Future.

“We decided to take a different approach to this, sort of technology-first instead of personnel-first, and say let's collect information very broadly,” explains Kodama. “Let's focus on the ability to take information, raw data collect, in lot of different formats, bring it in with automated collection, structure it and align the data models so that you have an ability to look either with automation or with human mind across different data sets and make connections.”

This approach allows the jargon to move from problem set to analytics, allowing companies to access tailored data without large numbers of humans being involved in the process.

“Basically roll-ups, outliers, trends, exceptional data points that are meaningful for the specific application, and use that to say let's express what matters to me,” he says. “It's essentially a set of queries, and so then automation can find those data points for me instead of my man in Havana.”

And with the level of available data seemingly forever on the rise, this approach is proving far better suited to the modern realities of cybersecurity.

“If all of this is going to continue to scale at the rate of how fast can I acquire highly trained professionals who know this particular knowledge work, then it's not going to scale very well,” says Kodama. “So we wanted to get things on a different curve, and essentially that's we've been working on for the last seven, eight years.”

When Recorded Future started, its focus was on offering raw data to analysts, and a part of the company’s business still remains in this area.

“The customers who've been with us for the longest are analysts,” says Kodama. “They don't trust anything until they've seen the data and validated for themselves, in a lot of cases as well they should, and so we've started there and built from at that point. “

However, over the last decade the market has changed, with growing cybersecurity threats forcing businesses to change the way they respond.

“Back then you would talk to people at the corporate level or at the state and local level in the US, and they would say this isn't really a problem for us, is the Feds deal with this, and they've got their specialised teams and assets and so forth, and then when something goes wrong they call me or they send me a report, but it's not something that I worry about,” he explains.

“But as you're well aware, the risks that companies and smaller organisations are exposed to, they've changed dramatically in the past 10 years. It's not that anybody I think really wanted to do this: they had to.”

As the market has changed, new types of users have appeared, prompting Recorded Future to grow its offerings in response to the point where it now offers a gradient of products, from fine, granular data to summary advisory information.

“There's another level up where people look at it as really at the level of not data but information. They tend to be in more traditional operational security roles,” Kodama says, adding that such users tend to utilise the product to get additional information about security incidents.

“In that context they're not really trying to do a deep investigation and find green data, they're just saying 'I've got about four or five things that I'm looking at to make a verdict on this incident’.

“And the highest level is where they're really not looking for data or information, they're really looking for advisory.”

This can take the form of regular reports about relevant changes in the threat landscape or custom tailored projects for particular businesses, and it is this type of expertise where Recorded Future is seeing the biggest growth in demand.

“The growth is fastest at the higher levels, because for everyone who is already working with intelligence to a degree, most of the market still is on the other side of that,” he says. “They're watching it emerge and develop and trying to understand what's the cost-effective way for me to bring this into my organisation, because hiring a team and providing them the capabilities that they're going to need to do the work, it's a very expensive proposition.“

This changing market has also prompted Recorded Future to launch Fusion, a software-as-a-service product that the company claims is the first universal threat intelligence solution. This effectively takes technology-driven threat intelligence to its natural conclusion, where companies manage their threat intelligence through a single platform, assisted by automated processes.

“The Fusion capability basically lets [companies] centralise a lot of different security and threat intel data in the system and then take information like that from their internal systems, distill it down to the stuff that they're comfortable sharing with us – because that's always a concern – but then automatically put it into Recorded Future and then get customised outputs back,” explains Kodama. “This takes some of the things that they're doing at smaller scale, as a human process, and ramps them up to an automated process.“

This approach allows companies to start with the areas they are looking at, but which they may not have the time to analyse to their fullest, and extract deeper insight than would otherwise be possible.

Kodama gives DNS names flagged in incident response as an example of such data.

“I may not have taken the time to manually look them up, but if you've got information about them then forensically that might tell me that something that I thought was a run-of-the-mill infection on some end-point computer actually was part of a larger campaign of intrusions,” he says. “And whoever was dealing with that might not have taken the time to know to take the investigation there, because that was one of 60 things they had to deal with that day.”