Tuesday, May 31, 2011

Musings on Structured Analysis

Starting a new investigation is always exciting for me.  What goodies will this image contain?  After popping the drive in a write-blocked bay, my fingers itch to start running as many processes as my poor little machine can handle:  NetAnalysis; RegRipper; PhotoRec (my new favorite carver); and enough encripts to make any CPU gnash its teeth and long for the day it can retire to the far less challenging world of high-performance computer gaming.  Like many other techies, I have had to fight my inherent nature to start mucking about immediately without a care for a structured process.  The problem is, once an investigation gets underway, we analysts can get so caught up in the minutiae of the exam that things we have done many times as a matter of course may slip the mind.  Take the following (completely made-up) scenario between Frustrated Examiner (FE) and Calm Reason (CR):

FE:  Why is xyz not in the registry?! 
CR:  Well, did you check the restore points?
FE:  *crickets*  Um... one moment.  *Furious mouse clicks*  Uh... never mind.

The problem here, as many would point out, could be solved simply by following a list that includes a bullet-point along the lines of "Restore Point Analysis Performed."  Check.  There is a lot to be said for the checklist method of analysis.  It can focus even the most hex-centric mind.  Checklists save the day.  And there is much rejoicing.  So why is it that my own experience with checklists has been checkered with resentment that occasionally leads to drawing unflattering facial hair on its typeface? 

Where comprehensive checklists go wrong is in being, well, comprehensive.  With skyrocketing data storage and efforts to mitigate costs in litigation, it is very rare that I am asked to perform a complete examination of any media larger than a thumb drive.  Usually, a DF investigation is performed with a specific question in mind that the analysis is supposed to answer.  And the steps taken to answer one question, while perfectly valid in that instance, may be completely extraneous in the attempt to uncover answers to a different question. 

While in Las Vegas at the ADUC, I attended a lecture by Jesse Kornblum with the intimidating title "Statistical Validation and Data Analytics in eDiscovery."  It is a testament to his devotion to his work that a presentation with that title was one of the most fun and entertaining lectures of the conference.  And to prove that I did indeed have some cognitive functions on my third day in Vegas, I have decided to create my very own decision tree for targeted checklists.


A flow chart, while pretty and impressive to upper management, is great but it still does not solve all investigative ills.  Checklists are great tools - but like any other tool in DFIR, they are just a tool.  Sticking rigidly to a list of procedures takes the investigator out of the investigation.  While I wouldn't argue that analysts should dump the binary content of a drive out of the box and just pick up random bits without any sort of structure, neither would I recommend blinders.  Follow the white rabbit!  Just remember to take notes along the way.

19 comments:

  1. I've got a checklist for malware detection examinations that lists tools to use (it's a "living document", so it can be changed), and includes space for findings, notes, etc. However, the cool thing about it is that I can NOT do something on the checklist, and include a reason/justification for skipping the step.

    FE: Did you check for alternate data streams?
    CR: Nope.
    FE: OMG!! It's on the checklist! Why not???
    CR: Because it's phat...and by that, I mean the file system is FAT.

    The real benefit of having a checklist is that it's documentation, and serves very well as a teaching aid for new(er) examiners. Of course, one of the best questions to ask is, "why?", as in "why do you do this?", or "why don't you do this?"

    ReplyDelete
  2. @Keydet89 this is why I am an HC groupie (but, you know, in a professional, notatallweird sort of way).

    NTFS - you're still FAT with a "P"

    ReplyDelete
  3. My case notes (you keep case notes...right???) tend to reflect some saunters off of the beaten path and toward minutiae...albeit with an effort to keep things somewhat on track, and keep with the structured analysis.

    However, that doesn't mean that the structured analysis is something set in stone. The best place to start with improving a process is with your last engagement...what could I have done better? What can I take away from my last analysis and add to my process?

    If you didn't document it, it didn't happen.

    If you can't explain what you did, to the degree that another analyst can do the same things with the same tools and same data...did it really happen?

    ReplyDelete
  4. Yes, I definitely keep case notes! I think your process is precisely the perfect mesh that I was trying (albeit not very well) to advocate. I like the idea of having a checklist specific to malware. In fact, I think I should invest the time in creating multiple checklists (and keeping them "living") to include other variations like spoliation and fraud.

    Granted, there are certain things that would pretty much be included in any examination. Registry analysis for starters (well... if it's a Windows system). Looking for data movement? Check the registry. Deletion of documents? Check the registry. Internet history? Ya...

    While I don't like the idea of a "comprehensive" checklist with no room for additional notes, I think that a documented strategy going in is absolutely necessary, as long as the examiner can recognize when the battle plan should change as more information comes to light, and have a method for explaining such beyond a "Complete" or "Not Complete" checkbox.

    ReplyDelete
  5. Looking for data movement? Check the registry.

    While I like the way you think (i.e., "...check the Registry"...), I'm curious as what you'd look for in the Registry with respect to data movement. That sounds very interesting.

    ...a documented strategy going in is absolutely necessary

    Agreed, 100%.

    One of the biggest benefits I've found of having a structured, documented analysis process...even if I have to sort of pull it together on my way to get the data...is this...

    What happens when many analysts receive a drive or an image and are asked to find malware? Or, how about if the customer/owner says that they suspect that the system was infected with Zeus or some other "banking Trojan"? From many of the reports I seen and analysts I've spoken to, the "process" consists of either mounting the image as a volume or booting it in a virtual environment and "checking it". There may be an attempt to scan it with AV.

    I've received drives and been asked to "find malware", with no indication as to why malware was suspected. In some cases, I haven't found any malware to speak of...and the process I have used is (I believe) VERY comprehensive, even going so far as to checking the contents of the image for indications of MBR infectors. Having that documented process allows me to be far more complete and comprehensive that simply scanning the image with an AV scanner, and to perform the tasks in a focused manner, taking less time and saving the customer $$$.

    ReplyDelete
  6. I probably should have used a term other than "data movement." In this case, I was thinking in terms of IP theft. A scenario I've encountered many times is "We have reason to believe Former Employee may have copied certain documents to a thumb drive before leaving the company. Is there any evidence to support this?" While the registry isn't the only place to look in this kind of investigation, it would be one of my first areas of interest for things like mounted drives, etc.

    ReplyDelete
  7. "We have reason to believe Former Employee may have copied certain documents to a thumb drive before leaving the company. Is there any evidence to support this?"

    Yes, I've seen this question posted several times in various forums (forii??) and have seen folks recommend "checking the Registry", but I don't often see anything specific regarding _what_ they would check that would indicate that files were moved from the local hard drive to any other location.

    ReplyDelete
  8. In IP theft cases, I've found there can be a lot of corroborating evidence. I've found copying utilities installed, recently opened files with names of interest in external drive paths, not to mention the coup de gras of actually also having the USB drive in question containing the files and then verifying that it was indeed mounted to that particular computer.

    ReplyDelete
  9. I like the blog and your take on things. Keep up the good work…..

    I don’t use a checklist but a documented procedure and the examination portion has a baseline of steps to complete for the different investigations I face (policy violations, investigative support, or security incidents). Going into an examination I tend to reflect on the goals of the examination and I think about what baseline examination steps I need to complete. For example, one of my baseline examination steps is to examine the volatile data but if the case is for a financial investigation that occurred months ago then I may skip the step since my initial interest is months in the past. If the case is for a malware infection then things would be different and the examination of volatile data (if available) would be completed. I use this reflection to help map out my examinations and to keep my actions focused on the data I need.

    I continuously update my procedure including the baseline examination steps. As you mentioned the steps aren’t set in stone and I couldn’t agree more. Flexibility is needed since everything can’t be accounted for but the steps do help in establishing a repeatable process.

    ReplyDelete
  10. @Corey I will most definitely keep blogging - especially since it has been an amazing learning opportunity for me. Plus I get uber l33t folks like yourself and @keydet89 who are willing to share their knowledge! I love your blog, too. My experience has been primarily in post-mortem investigations, but I have a feeling that incident response and live/memory analysis will start to be on the docket more as time goes on.

    ReplyDelete
  11. > My experience has been primarily in post-mortem investigations, but I have a feeling that incident response and live/memory analysis will start to be on the docket

    I was in a similiar boat. Pretty much all of my work invovled post-mortem investigations (non-IR on top of that) but I had an interest in IR and breach investigations.

    To update my skillz and knowledge, I started researching and testing the different methods, techniques, and tools to examine these types of cases. I even updated my procedure to support these types of cases. I'm glad
    I took the first step instead of waiting til something showed up. It helped me be to ready for the first IR examination that came down.

    ReplyDelete
  12. @Corey Excellent point and a shining example of how it should be done. I think we tend to be reactive rather than proactive in this business (although some of that comes with the territory - new things come out every day).

    So far my premonitions on the matter have led me to do a lot of research and reading, but I have not moved on to the next logical step of moving beyond the theoretical realm and into testing. I'm going to use this as the impetus needed to push me on to that next step.

    ReplyDelete
  13. ...USB drive in question containing the files and then verifying that it was indeed mounted to that particular computer.

    You know, Cory Altheide and I did some of the first published research on this topic several years ago, and even given all of the "press" it's gotten, from Rob Lee/SANS and even in my own books and blog, this still remains one of the most misunderstood and misquoted areas of Registry analysis...

    ReplyDelete
  14. @Keydet89 Very much aware of your role! (Both editions of my WFA books are covered in highlights, notes, and random bits of paper... especially in that area) Would you mind elaborating? I hope I haven't completely misunderstood.

    ReplyDelete
  15. Hi we don't have checklist we follow exacly but I do agree we should. My method I follow is to study the background of each case a compile a checklist specific to my case that I am busy with even then I found that the lists are something ever expanding as you learn more and a case grows. But I even tend to fall of the path as something challenging catched my eye and then I get excited and run in the opposite direction however our mentor drills it in our heads to stick to the scope give them what they want, but giving the I/O what they want is in my opinion not always the same as doing a non thorough examination compromises my integrity and standards. I need advice on how to change the view of I/O that its not CSI where you push a button and in seconds receive the evidence you need.

    ReplyDelete
  16. @Anonymous I don't know if we will ever truly win the war against the CSI inspired can't-you-just-push-a-button mentality. One thing that I like to explain to my clients is that where an analyst really proves their worth is in providing analysis. Automated functions can be a great help in an investigation, but where is the context?

    For people who work in the eDiscovery field, things are a little more cut and dried: does a document exist or doesn't it? In forensics we deal with even more ephemeral data (if any data can be considered more ephemeral than others... hmm) like artifacts. And we have to take things a bit further from "this contains (or doesn't contain) this information" to the next step of "what does that mean?" and "how" and "why."

    ReplyDelete
  17. Great idea to start posting and this topic resonated with me very well. Its refreshing to see that other examiners are looking at the registry hives with tools like RegRipper. I have recently worked with others who did not. But once I gave them their first look at RegRipper, the look of amazement would make them forget about using EnCase or FTK going forward. (Frankly, I don't think they would ever look back.)

    Its nice to know that there are other examiners working in S. California. Keep up the good work.

    ReplyDelete
  18. @Thomas I don't know how I missed your post... but just found it. Definitely exciting to meet (?) another forensicator in SoCal. If you ever want to get a group together, let me know! I'd love something along the lines of @Keydet89's NOVA group, but I'm sure it would have to be a bit more humble.

    ReplyDelete
  19. Not to say anything negative about RegRipper, but you can't trash EnCase and FTK because that tool exists either. Both EnCase and FTK have similar tools built into them. Perhaps the folks that have been using those tools didn't even know that functionality existed...

    ReplyDelete