Being Forensically Curious: The Process of Scripting

Our in-depth three-step forensic research methodology series, which started with “The Process of Discovery,” led to “The Process of Testing” and “The Process of Finding and Parsing,” concludes today with our final blog, “The Process of Scripting.”

Our previous blog posts relied on research by Magnet Forensics’ Jessica Hyde and Basis Technology’s Cesar Quezada, as well as commentary by forensic research experts Cheeky4n6Monkey and Heather Mahalik. We turn now to Preston Miller and Chapin Bryce, authors of the newly released book The Python Digital Forensics Cookbook. Having developed a script for use with Magnet AXIOM, Preston and Chapin graciously agreed to answer our questions about their process.

Magnet Forensics: How did you both first get into scripting? How long did it take you to learn, to the point where you felt comfortable using and sharing scripts on the job? 

Preston: A summer internship with a cyber investigations firm was the impetus behind me learning to program. I came from a physical science background and lacked the computer science skills most of my colleagues had. I spent a lot of that summer getting acquainted with the ins and outs of Python as it became apparent what a desirable and useful skill it was. Within half a year, I became comfortable enough to start working on open source projects and sharing my code. Not long after that, I started teaching other students about Python and its application to forensic investigations at my graduate program.

Chapin: I started with scripting through a project while in college, where we used Python in digital forensic workflow automation. Over the next few months, I became more comfortable with writing and developing my own code. About a year into developing with Python, I became comfortable sharing my developments with the community and introducing more scripts into my investigations. Later that year I hosted presentations for fellow students to learn about using Python in forensics and presented a Python-based project with my development team at several industry conferences.

Magnet Forensics: What’s easy about learning a new scripting language? What’s difficult, and how can learners compensate?

Preston and Chapin: Learning a new language is difficult. There are certain factors which can help speed up the process, including previous programming experience, but prepare to dedicate time and expect to run into frustrating problems early and often. That said, a language like Python has a very straight-forward syntax, unlike other programming languages, and the development life cycle is very rapid, comparatively. These features make it a great language for a field that requires quick thinking and even quicker solutions.

Magnet Forensics: Do you recommend setting goals and timeframes for learning how to script? If so, can you provide examples?

Preston and Chapin: Repetition is key. When first learning any language (computer or otherwise), set aside time every day and focus on learning new content or cementing your understanding of previously learned concepts. Even if you can only spare thirty minutes, daily repetition helps strengthen those programming synapses and will more quickly improve your programming capability.

Sometimes the hardest part in developing a script is determining where to start and what to automate. To make things easier, take a task you regularly perform in your casework and automate one step of it. Continue to add functionality to your script until you can automate the entire task (if possible). By doing this, you will learn both how to use the language and when to use it.

A problem we often see in fledgling forensic Python developers is an over-reliance and use of the language in scenarios where other tools exist or other steps can be taken to accomplish a given task. In these scenarios, you would not want to reinvent the wheel and spend time developing code unnecessarily.

Additionally, know that automation cannot be applied to everything as if it were a panacea. There are certain tasks that automation can help and others that are more trouble than they are worth. Keep these points in mind as you consider what repetitive processes you regularly perform in an investigation.

Magnet Forensics: Besides goal-setting, what are some other ways that new scripters can make time for research and writing scripts? What kinds of daily, weekly, and/or monthly activities go into learning a new language?

Preston and Chapin: Beyond time, and much like learning a language, you have to immerse yourself in the language to obtain maximal results. There are a number of online and print resources, including our first book Learning Python for Forensics, which can be used to jumpstart a deeper understanding of the language and its capabilities within the cyber investigation space.

Magnet Forensics: How important are collaborations to the research and scripting process? How can examiners go about building the kinds of professional relationships that lead to collaborations?

Preston and Chapin: Very important. Working with others is a great way to learn new tricks, confirm your understanding of core programming concepts, and keep yourself motivated throughout the learning process. After working at a summer internship together, and upon returning to our respective alma maters, we began an open-source project designed to process Android artifacts and create automated reports. This gave us the opportunity to learn from each other and explore the development cycle jointly.

Collaborating with peers within your field is a great means of establishing and building these types of relationships. Additionally, there are a plethora of open-source projects one can join to start developing professional relationships (and scripts). Many of these projects have contribution guides and are happy to bring people of all skill levels onboard.

Magnet Forensics: Tell us more about your book overall, and what drew you to want to work with Magnet to develop the macOS daily.out log parser? What went into developing that script, and what else can readers expect from the book?

Preston and Chapin: Python Digital Forensics Cookbook was written to showcase the many ways in which Python can be used in the context of any Cyber investigation. Programming has been a great tool for us; we wanted to share our knowledge with the community and assist other examiners looking to add to their toolbox. The book is comprised of 60-plus recipes tackling a whole gamut of forensic processes. Our succinct recipes take a “no frills” approach to solving common challenges faced in investigations and covers a wide range of artifacts and data sources. The examples we cover will improve the accuracy and efficiency of any examiner’s analysis capabilities – no matter the situation.

We are very excited to share our daily.out log parser script in the book and on the Magnet Artifact Exchange for use against macOS systems. Our collaboration with Magnet showed us how simple it is to build upon an already powerful forensic analysis suite and enhance it further with support for custom artifacts using Python. In under 50 lines of code, we were able to transform a stand-alone script into a plugin that leverages AXIOM’s processing, reviewing, and reporting engine.

Magnet Forensics: What advice would you give examiners who want to code, but aren’t sure where to start?

Preston and Chapin: Get involved with the community. With the advent of Github and the proliferation of open-source projects, there is an abundance of opportunities for those enterprising individuals that want to learn how to develop code. You will find, as was our experience, that showing enthusiasm and a capability to contribute in some way is all you need to get started on most open-source projects. Developers are excited to find like-minded people willing to invest time and energy contributing to their projects.

Thanks to Preston and Chapin for talking with us about their process, their book, and for involving AXIOM in their research!