Saving a Day’s Worth of Work Using Magnet AUTOMATE
In this guest post, Aaron Sparling, Officer, Investigations Branch, Digital Forensics Unit at the Portland Police Bureau, talks about how he worked with Magnet Forensics and ATOLA Technology on implementing Magnet AUTOMATE in his lab and how it benefited him.
Have you ever heard the phrase: “find the needle in the haystack”? The simple solution to this request would be to burn the haystack and expose the needle. However, if you work in digital forensics, then I am sure you have heard the modified version of this which translates to ‘find the needle in the needle stack.’ Unfortunately, we cannot simply burn the stack down to expose the needle. Not to mention that every day that needle stack is growing is size. I am sure you may have figured by now that the needle stack represents the copious amounts of digital evidence or data which forensics analysts are faced with in modern times. Depending on your workspace, the number of devices and the volume of data you may be tasked with examining on a per case or incident basis can exceed 30-50 terabytes in size.
I am very familiar with this scenario. I work in a lab which services a very large agency and population and receives hundreds of devices and multiple terabytes of data annually. To add an additional layer to the problem, as an analyst we don’t always have in-depth insight into the investigations. What this ultimately means is that the forensics analyst could waste countless hours of high-value analysis time hunting for those valuable crumbs to only find they were of little to no value to the investigation. So, what do you do when you have been handed 20-30 devices to analyze and you have little to no context in regard to the investigation? Simple, get the data into the hands of the investigator as fast as possible so that they can search, bookmark and pivot on that data which supports their investigation. This way, you, the analyst, can then do the much-needed forensic analysis on those artifacts which are relevant to the case.
An Example of a Typical Workflow
Even when using creative workflow methods, the imaging and processing of multiple devices can take weeks(yes, weeks!). Let’s use a real data set to walk through this logically. Imagine you have been given seven PCs and eight external hard drives totaling 46.5 TB to be imaged and analyzed. Understanding that not all forensics labs are equipped equally as personnel, budgets, software and hardware vary. The imaging and processing workflow may look something like this:
- Load the first hard drive into physical write blocker and start imaging and verification process. Wait 4-6 hours for imaging and verification to complete and repeat the process until all 15 hard drives are imaged.
- Load the forensics images into your analysis software and process the data. Barring no issues and depending on what you have configured the software processing to conduct this can and will take days before it is completed.
- Once processing is completed and verified you may build timelines or run artifact connections against your case database. Again, this will take numerous hours to complete.
- Remembering that the goal is to get the data back to the investigator as quickly as possible and if your analysis software has this feature you would now consider generating a portable case for review. Using the 46.5 TB data set as our baseline for this scenario, it could take numerous hours to generate the portable case.
The workflow mentioned above is what I would classify as liner or simple 1:1 logical progression. There is nothing wrong with this methodology or workflow. However, just by the nature of the reliance of human interaction to monitor and complete simple tasks such as swapping hard drives for imaging, there are time gaps that add to the overall time to completion.
Illustrating the Time Gaps in This Workflow
Imagine you begin your workday (9AM to 5PM) by loading the drive into the write blocker in starting the imaging process not knowing how long it will take. If we use six hours as an agreeable base line for this task, then the image is completed mid-afternoon. However, you do not spend the six hours watching the progress bar as the drive is imaged, instead you dive into the many numerous other tasks you must do in your day. Let’s agree that the imaging completed at 3PM, however you are already deep into another task and cannot break away until 4PM, that means the drive sat idle for 60 minutes. That is one example of a time gap in the workflow.
Let’s add another layer to this. It’s 4PM and you manage to load the image file into your analysis software and start the case processing prior to leaving the office at 5PM and the processing runs and completes sometime around 4AM. This means that the case would sit idle for at least an additional five hours or until the analyst returns to the lab and begins generating the portable case. If we ran this scenario on a Friday it could be more than two full days that the case sat idle over the weekend.
Imagine if we were to add multiple devices to this scenario and what the time gaps would look like and the amount of idle time the case would encompass?
What if we could remove the human interaction from the tasks which do not require the analyst direct interaction. We can, and this is accomplished through automation.
Automation comes in many different forms, such as simple for loops in bash scripts to complex chaining of tasks which can remove some of the mundane tasks from the analyst workflow. I was fortunate to work with Jessica Hyde and Gavin Hornsey from Magnet Forensics along with the entire Magnet AUTOMATE team and the staff from ATOLA Technology on a proof-of-concept project in my lab. Let me walk you through a case and show you how it worked for me.
Real Case Timeline Comparison
The case involved three (3) computers in which one of the PC towers had multiple hard drives installed, four (4) loose 3.5” hard drives and two (2) USB flash drives. The data needed to be imaged and processed totaled just under 5TB (4.906 TB). The case involved SIM swapping and Dark Web market marketplace criminal activities. There was a time sensitive nature to the case as it involved multiple parties working the investigation. The agreed upon workflow would be to image the evidence, process the images with Magnet AXIOM and generate a portable case for investigative review.
The first step was to triage the evidence and scope the relevance or potential of the device(s) at the request of the investigator. Placing the hard drives into a physical write blocker and running Magnet OUTRIDER, I was able to quickly (under an hour) triage all drives and reduce the total number of drives to be imaged and processed down to just two. Next was to process the evidence using the traditional or standard workflow which was currently deployed in the lab, knowing that there would be time gaps due to the number of devices/media needed to be analyzed and the required human interaction to do so. This workflow took a total of 58 hours and 5 minutes to go from imaging to a final generated portable case.
Now that the case imaging, processing and portable case creation is complete I AUTOMATE’d the workflow to see just how much time could be saved as well as to illustrate the lower impact orchestration and automation can have on the reduction of human interaction to reduce machine downtime, thus allowing the analyst (me) more time to focus on tasks involving forensic analysis.
I connected the two (2) computer hard drives to the ATOLA Task Force Image station which was networked to the Magnet AUTOMATE cluster, filled in my relevant case information and launched a basic workflow template from within the AUTOMATE dashboard. The workflow imaged both hard drives, copied the images to a separate RAID array, processed both hard drive images with Magnet AXIOM, combined the case processing into a single case and generated a portable case in 40 hours and 48 minutes. All of these tasks were completed without me having to interact with the workflow at any point, leaving me free of interruptions to work on other cases and lab projects while AUTOMATE worked in the background.
The automation process saved 9 hours and 43 minutes of over all time, which is more than a standard 8-hour workday. The time savings has a number value associated with it which can be validated and verified, however, I would like to highlight that the automation process allowed me to work on other projects and analysis without the interruptions of having to manually work through the imaging and processing workflow of two separate hard drives. It is the lack of interruptions one gains from the automation process which I believe to be just as valuable if not more valuable than the overall case imaging and processing time savings.
This is just one simple example of how utilizing some already existing hardware in our workspace, we were able to simply and quickly AUTOMATE an already existing workflow. It should be noted that the AUTOMATE application can be used with any forensics tool that has a CLI interface, python frameworks such as Volatility, and scripts can all be implemented and written into your custom AUTOMATE workflows. This means that you can mature and tailor your workflows to support the needs and demands of your lab and or consumers. There are so many possibilities to what we as forensic analysts and examiners can do in regard to automating our workflows thus allowing us more time to do those time-consuming deep-dive forensic analysis of the relevant artifacts. I am extremely excited to see what the automation projects and workflow the community develops as we navigate the ever-growing sea of data and devices.
To learn more about Magnet AUTOMATE and request a free consultation and personalized demo, visit https://www.magnetforensics.com/products/magnet-automate/ and fill out the form to contact us.