The AXIOM Performance Journey: How Much Faster is it Now?

By Jad Saliba, Founder & CTO of Magnet Forensics

It’s almost been two years since we released Magnet AXIOM v1.0, and it’s been quite the ride since that release day!

AXIOM was created primarily because we were looking to add more functionality to IEF — but IEF was based on a platform that was reaching its limits. We wanted to build a new platform that would bring us into a new era of digital forensics and still deliver traditional features — like imaging and file system access – all while allowing you to complete more of your case in one tool and provide next-level analysis features.

To accomplish this, we needed to reimagine and redesign the back-end case storage along with the front-end user interface. Delivering these new capabilities would require the data to be stored differently and we needed to display the data in a more intuitive way.

Unfortunately, on larger cases this resulted in slower performance from AXIOM on process and on the review interface (called AXIOM Examine) than on IEF and the IEF Report Viewer. AXIOM was working a lot harder — doing more, storing more — but we were determined to have that same level of performance with speeds at least at IEF level.

It turns out that if you give a challenge to engineers, they won’t rest at just being “as good”, they have to be better, to beat the previous “score” — especially when you have competitive engineers like we do. ?

We were able to get AXIOM processing to a place where it was consistently 20-30% faster than IEF across a wide variety of images, and have made some recent improvements on the review side to speed up things like case loading, switching views, searching/filtering, and more. We still have more improvements scheduled since this isn’t really an area where you’re ever “done.”

It’s also interesting how perception of performance versus the reality of performance can skew things. It’s been said “perception is reality” and there’s definitely some truth to that. There are some parts of IEF that still may seem “faster” than AXIOM while they actually aren’t, and we’re working on that too.

But as they say “show, don’t tell” — so let me show you. I took an image from a hard drive that contained real world data on it (i.e. not a drive we put test data on or simulated regular usage on) and ran side-by-side tests using the latest versions of IEF and AXIOM. Below are the details of the hardware used to run the tests and the hard drive used as the data source:

Testing Machine

• Dell M3800 Laptop
• i7-4702HQ 2GHz CPU (4 cores, 8 threads)
• 16GB RAM
• 256GB SSD + 1TB HDD (case stored on SSD, image stored on HDD)
• Windows 7 Pro

Performance tip: Storing your case/image on an SSD is the ideal scenario for best performance with AXIOM and IEF – if you store your cases on a network-based storage location, you will see degraded performance due to the database reads across the network. This is something we are working on, but nothing beats fast, local storage.

Also note: I purposely did not use a big “forensic workstation” as I wanted to show how AXIOM can be used on consumer hardware that is decently spec’d out.

The Dataset

• 500GB hard drive
• 89.9GB allocated
• NTFS filesystem
• Windows 7 Home Premium OS installed on drive – in use approximately 4 years
• E01 image created – size of image is 82.2GB (medium compression setting used)
• IEF case size: 43.2 GB
• AXIOM case size: 39.9GB
• Total records in case: 675, 574 (with the usual “hot spots”: pictures, Windows event logs, web history)

The Tests!

First, let’s look at processing time. I used the same settings in IEF and AXIOM:

• All artifacts selected
• “Full Search” on all partitions
• No additional processing options selected

Processing Time

IEF (v6.12.4.9640)

Search Duration: 07:59:50
Indexing Duration: n/a

AXIOM (v1.2.5.8637)

Search Duration: 06:12:52
Indexing Duration: 00:04:04

Comments

AXIOM completed the same search about 1 hour and 43 minutes faster, including indexing time. That’s about 21% faster! The new database layout and indexing will come in handy later as well, as you’ll see.

Review / Analysis

Okay, we’ve got our cases. Let’s run through some common operations in both review tools. First, we need to open the cases:

Case Load Time

IEF: 00:00:17
AXIOM: 00:00:18

Pretty close. We’ve made some improvements on the AXIOM side recently since we know that in the past, large cases could take a long time to load.

Now, let’s load an artifact that has a good number of records and sort on one of the columns.

Load “LNK Files” and sort on column “Last Accessed Date/Time” (65,618 records)

AXIOM: 00:00:07
IEF: 00:01:25

We can see that AXIOM gets you into the data much quicker than IEF and then sorts the data faster too. The other thing to note is that if this was a bigger dataset (over 100,000 records) or you had the “max records” setting in IEF to a number less than the total number of records, IEF is only sorting the visible/loaded records – not the entire dataset for that artifact.

This was a pain point in IEF and something we made sure to resolve in AXIOM.

IEF has better scrolling speed, which allows you to quickly scan records – but we’re working on making this just as good in AXIOM, if not better.

Next, let’s run through 3 single keyword searches and then a keyword list.

Single keyword “guest” (50,931 results)

AXIOM: 00:00:50
IEF: 00:10:17

Whoa! That’s a pretty big difference. AXIOM completed the search in approximately 90% less time, or 10 times faster. As you can see in the AXIOM video, we’re doing some caching so that subsequent searches completed even faster – the 2nd search I ran for the above recording completes in just 10 seconds.

Single keyword “chris” (346,769 results)

AXIOM: 00:01:06
IEF: 00:20:04

That’s a lot of results. As you can see, the large number of hits only affected AXIOM slightly versus the last keyword, but IEF approximately doubled in time due to the additional hits.

Single keyword “craigslist” (1,182 results)

AXIOM: 00:00:54
IEF: 00:08:27

Less results this time, AXIOM went back down to the ~50 second mark, and IEF also dropped compared to the last two keywords.

Next let’s try a keyword list. We’ll import a text file containing these keywords:

angel
car
rental
yahoo
facebook
phoenix
insurance

Keyword list – 7 keywords (24,294 results)

AXIOM: 00:01:52 (plus 3 seconds to display after selecting the keyword list in the filter bar menu – see below screenshot)
IEF: 00:08:49

AXIOM’s time increases a bit compared to the single keyword searches, but it’s doing something additional – data is saved in the case database so that future searches of those keywords will be near instantaneous. See the below screenshot for how you would access those keywords – you can also see the hit count next to each keyword:

IEF stayed in the lower range due to the hit counts not being high, but still takes much longer than AXIOM and future searches of the same keywords will require the same amount of time each time they are run.

Okay, let’s do some filtering now. Let’s say you want to see all activity in the year 2010 for this case.

Filter all dates/times in the year 2010 (119,099 results)

AXIOM: 00:00:09
IEF: 00:20:53 (using “Global Date/Time Filter”)

Wow…as you can see, AXIOM leverages the new back-end changes to really accelerate filtering. Filtering in AXIOM also gives you additional features like selecting days of the week, filtering time independent of date and vice versa, and a cool anchoring feature that allows you to see all activity X number of minutes before/after a specific point in time.

Let’s try filtering on a single artifact, and on a single column.

Filter all Windows event logs for event ID 4624 (“an account was successfully logged on”) – 14,478 results

AXIOM: 00:00:08
IEF: 00:01:17

Again, AXIOM finishes much faster than IEF – not quite as dramatic as the last filter but still pretty good!

Lastly, let’s try loading up the Timeline view to see all artifacts with one or more timestamps in a visual representation.

Load Timeline View (480,292 records)

AXIOM: 00:00:38
IEF: 00:09:43

That’s a lot of records, but AXIOM gets through it in short order. IEF does its best with a large number of records but falls far behind AXIOM.

Conclusion

As you can see, we’ve made some big strides on the performance side in AXIOM over the past year and a half.

We started with the processing side and more recently have made some great improvements on the analysis/review side. We’ve got more great performance improvements coming in the near future too, so stay tuned!

If you’ve been using IEF for a long time, you might try AXIOM and feel like it is slower than IEF, due to the different/new interface. But if we add up the numbers above, you can see that AXIOM is significantly faster than IEF…and the numbers don’t lie:

IEF (processing and review time): 09:15:00
AXIOM (processing and review time): 06:22:00

That’s a reduction in time to process and review of 31%, almost 1/3 less time total, and approximately 3 hours less time overall! That’s a big time savings no matter how you look at it, and great from a percentage perspective as well.

Well, there you have it! Here at Magnet Forensics, we are committed to continuous improvement and producing the best tools possible. Please reach out to us if you are running into issues, see areas where we can improve, or have ideas on new things we should do – we’re not happy unless you’re having a good experience with our products.

Thanks for being on this journey with us – I’ve said it many times before but it really is our honour to support you in the important work you do. We appreciate your support over the years as well and look forward to continuing to work together with you to impact this world in a positive way.

– Jad & the Magnet Forensics team