Being Forensically Curious: The Process of Testing

In our last blog about being forensically curious, we used research by Magnet Forensics’ Jessica Hyde and Basis Technology’s Cesar Quezada, plus some commentary from Cheeky4n6Monkey, to show how to discover important information about new and/or unsupported apps—or simply, those you want to validate to ensure your forensic tool is acquiring and parsing all the data it should.

Once you’ve discovered all that you can about a new or unsupported app, it’s time to test the app—and your hypotheses about it. This allows you to verify what you think is happening to ensure you understand the way the app stores data, as well as how your forensic tool or parser decodes, parses, and presents the data.

To test an app properly takes some effort, especially up front, but it’s worth the return when you document it well – it’s methodology that will give you ironclad testimony skills in court and solidify your credibility as an expert witness, because it takes you out of the realm of assumption-making and firmly into the realm of provable facts.

Here’s how to do it:

Step 1: Create Known Data

Creating known data helps later on with decoding. You get a sense for how data is stored and how the device interprets it. For example, when you take a screen capture of a text message with a date and time stamp, it can help you understand the device’s date/time format as a reference point.

Before you begin actually creating the data:

1. Read about the apps so that you know which features to test, based on the app’s description.
2. Ensure you are testing the same version of the app, either on the device or via an emulator like Genymotion or BlueStacks. (Note: emulators are easier to work with for Android than they are for iOS, which only offers simulators that aren’t as useful for data population during testing.) You may have to roll back the app to an earlier version to accomplish this; this is likewise easier with Android than it is with iOS. Keep in mind, too, that newer apps don’t allow you to use older versions of themselves to create data.
3. Create a data population plan. How many profiles will you need? (Jessica says that generally, 2-4 are sufficient.) What pieces of data will you need to test? Keep in mind that your fake profiles should last some time, helping you test additional apps and devices, so make them as complete and believable as possible.

Creating known data can actually be a fun part of the process. It can take time, so it may be worth your while to approach one or more buddies to create it. This has multiple benefits:

• You avoid using your own or someone else’s personal data.
• You can include “distractor” data that has nothing to do with a “case” your fake profiles are the subjects of, such as contacts the “suspect” never actually communicated with.
• It will be easier to test the next app—including if you end up lending or borrowing someone else’s test device, you can make testing easier for them in the future, too.
• You’ll have built relationships with other examiners for future research.

This last point is valuable no matter at what point in your career you are. If you’re a student, you can collaborate with other students or find a mentor this way. If you’re a professional, you can mentor students or junior forensic examiners. You can learn more about the other person’s methods and add to your overall level of expertise, all by planning for the data you want to test.

Once you have the population plan, the correct app and operating system versions, and you know what you want to test on each app, start recording profile data. Says Cheeky4n6Monkey: “Perform testing as realistically as possible (e.g. send/receive messages) using a test reference device of the same make/model/OS version/app version as the exhibit.”

Step 2: Test the Known Data

After you’ve recorded the fake profiles, it’s time to test the data you’ve created. If you can, test the app across platforms; its functionality can be different. If you’re communicating with someone on the other platform, you need to be able to get any data they sent, even if it didn’t present correctly.

This means to test the data in Android, iOS, Windows, macOS, etc. “An app such as Skype will have functionality across all these platforms,” says Cesar, “but the way the data is saved will likely be different across them all. When you get the data from the other platforms, you can ensure that the data they sent was properly received by your test device. Say the iOS version of an app has the ability to drop a GPS pin on a map, but the Android device you’re testing doesn’t have that ability yet. We’d like to know that the pin was sent and see how the test device handled that piece of data.”

Heather Mahalik, a senior instructor with the SANS Institute, offers another caveat: it’s possible for evidentiary data not to parse out the same way test data does, even with the same app/OS/versions/models. “This is common when the acquisition method differs or if you test on a jailbroken or rooted device,” she explains.

“If a test device is rooted or jailbroken, you will get access to more files. When you try to pull the same information from a device that isn’t rooted or jailbroken, the data you extract may be limited. Something else I have seen is where app versions vary, and this can change everything. The entire database may be different.”

“Skype, for example, is using a secondary database to store chat messages in current versions. Most are aware of the traditional main.db, but now we have to dig into another database that may not be parsed by the tools. Always,” Heather concludes, “go right to the directory and examine the files manually to make sure nothing is missed!”

To test the data, pull the app’s APK from the physical image you made of the device. In the user data partition, you’ll find the system folder; there, you’ll find the app folder. Extract the APK file from the image, then sideload it onto your test device or emulator.

It should go without saying that at this point of the process, you need to start documenting what you’re doing. Screenshot the fake profile page once you’ve populated its data; then, screenshot test data, and have a form or other way to record types of data you populate from there (e.g. whether you’re sending an SMS, MMS, or chat message).

Documenting your work doesn’t only help you replicate your results in future tests. It’s also useful when you’re collaborating with other examiners, or you want to share your results in a blog, a presentation, or when you’re stuck on something and need assistance. It can also help you justify your interpretation of results when it comes time to testify in court.

This in-depth three-step methodology started with our previous blog, “The Process of Discovery,” and leads to the next steps in forensic research: finding and parsing. Subscribe to our blog to get the latest updates on this and other topics!