THE DATA CRUNCH

Much of this data is what’s called unstructured data – data that’s not neatly packaged up like rows and columns in a table. The challenge with unstructured data can be summarised by the following question: ‘How do you find what you are looking for when you don’t know where to look?’

Nowhere is this challenge more acute than in maintaining our national security.

National security agencies face the most imminent and complex Big Data challenges, as one of the most powerful weapons of today’s criminal is their ability to hide in an unstructured online environment.

This ability means that national security agencies often need to find the needle in the data haystack, but the haystack is growing at a phenomenal rate and the needle is often deliberately obscured.

Supporting national security agencies to combat crime and terrorism in the online world is the focus of a new national research hub based in Adelaide. Established in July 2014, the Data to Decisions Cooperative Research Centre (D2D CRC) is becoming one of Australia’s leading providers of Big Data capability, resulting in a safer and more secure nation. 

As well as helping national security agencies develop the tools needed to effectively operate in a new law enforcement frontier, the D2D CRC is also creating advanced Big Data tools to better analyse intelligence, surveillance and reconnaissance data so that military decision-makers have access to the information they need to complete their missions.

Researchers from the D2D CRC work in a world where virtual and physical reality collide. One of the D2D CRC’s projects, being led by UniSA’s Professor Bruce Thomas, aims to establish a new interactive visualisation environment for intelligence analysts to work in. In this ‘immersive intelligence pod’, analysts will be presented with – and can interact with – all relevant data and information for their immediate task. Once in the realm of science fiction, working in a virtual environment with Big Data from ‘the cloud’ is now a possibility.

While Big Data presents a number of complex challenges, it also provides new opportunities to strengthen national security. Research undertaken at the D2D CRC will revolutionise data analytics, storage and management, and in the process boost our national security agencies’ capacity to fight terrorism and crime in a Big Data world. 

Dr Sanjay Mazumdar is the CEO of the Data to Decisions Cooperative Research Centre, a collaboration between government, industry and research, established to tackle the Big Data challenges of defence and national security.

Find out more at the Data to Decisions CRC website.
Connect with D2D CRC and Prof Mazumdar: @D2DCRC / @SanjayZum

Researchers, organisations and governments alike are drowning in a sea of information, and the ability to harness that data has become a priority for many industries. But it is the area of law enforcement where Big Data will play a critical role in fighting crime and protecting citizens, writes Dr Raymond Choo.

The Australian Attorney-General Department’s National Plan to Combat Cybercrime acknowledges “the need for a more accurate intelligence picture to assist in identifying emerging trends and better direct resources to areas which would have the most substantial impact on the activities of cyber criminals”.

One way this can be achieved is by gathering and analysing intelligence from electronic evidence held or seized by law enforcement agencies undertaking investigations. But right now, the world is facing a ‘big forensic data’ problem due to the volume, variety, and velocity of electronic evidence. The sheer volume of data causes issues in relation to collection, storage, analysis, archiving, and particularly, extracting and using intelligence from seized data. 

Given the vast amount of structured and unstructured data seized, there are currently significant backlogs of evidence awaiting analysis, often lying unassessed for months or even years, with even the largest digital forensic laboratories struggling to keep up.

UniSA PhD student Darren Quick and I have been investigating ways to address this big forensic data problem. There is currently very little published work regarding methods or frameworks to apply data mining techniques or other approaches to reduce or analyse the large volume of real-world data. In addition, the value of extracting or using intelligence from digital forensic data has had minimal discussion in the academic sphere.

But in order to address the Big Data challenges affecting law enforcement investigations, it is crucial that we undertake research into data reduction techniques, data mining, intelligence analysis, and the use of open and closed source information.

In a recent publication, we have outlined a data reduction and data mining framework for digital forensic evidence, which incorporates a process that reduces data volume by focusing on a subset of information.

This framework allows a law enforcement investigation to review items of importance, reduces data storage requirements for archival and retrieval purposes, and provides a capability to undertake intelligence analysis of digital forensic data.

We have also proposed a method of Data Reduction by Selective Imaging (DRbSI), using the data reduction technique of feature selection. Applying this method to recent test cases and sample data from the South Australian Police Electronic Crime Section has already demonstrated a significant reduction in original media volume, and therefore processing time.

Big Data also means big research opportunities. We are now working towards validating and refining the data reduction and data mining framework and DRbSI methodology to ensure our research will deliver outcomes that improve law enforcement and combat the big forensic data challenges of our time.

Dr Raymond Choo is a researcher and a senior lecturer at UniSA. He regularly presents around the world on the areas of cyber security, Big Data and cloud computing.