Investigate

 

Introduction

This document details how an analyst should conduct investigations and triage in the normal duties of their job. It will describe concepts and logic that enable analysts to look at data, ask investigative questions, evaluate those questions and arrive at assertions about what they have found.



An Alert

For the most part, analysts begin their investigative work when an alert is generated. Alerts are fantastic because they are a statement that something has happened and something needs to be done about it. Typically alerts are filled with information that an analyst can then use in their investigation. Alerts are generated by detections and these detections can come in many shapes and sizes.

Some just look for a particular action like someone unlocking their car door, others might have access to more context like someone who isn’t the owner of the car unlocking the door. Going even further some detections might look at how the car door was opened or at what time of day it happened. Further, some detections monitor all these little data points and paint a picture of what it would expect someone getting into their car to look like (These are called baselines). Ultimately a detection does not have to be an alert because they can be abstracted together over time or when a certain condition is met and then turned into an alert.

It’s really important to understand what type of alert your looking at (not what the alert is, we aren’t there yet) as how it was generated decides what your investigative questions look like and the distance between performing analysis and taking action on the alert (very scary alert = very quick response). Particularly you also need to understand what your alert tooling can and can’t give you because the less capability you have the shorter the distance to you making a decision.

Picking what's important

Imagine a conveyor belt and you have a shopping basket and its your job to fill that basket with things that look scary or interesting, on that conveyor belt comes along 4 failed user authentication request. well ok that might interesting but we wouldn’t want that in our basket just yet because we see those on the conveyor belt all the time, but you pick it up and the label says its this really important person you know say maybe the head of finance, that’s a bit different to the usual failed authentication request and causes us some more worry so we put it in our basket.

Next on the conveyor belt is an application connecting to the internet, so you take a look and recognise its an application your company always gets on the conveyor belt and with a quick check of the address for the network connection it all matches up so you don’t bother adding it to your basket. Then comes along a word document that invoked a script interpreter, now we know sometimes this can happen especially in teams and departments that do a lot of data processing because it can speed things up but we also know script interpreters and productivity apps are a really common adversary technique so we make sure to put that in our basket for deeper analysis later.

With this process in mind, it is clear there are a few things any analyst needs to be an expert in, first and foremost what adversary procedures exist and how their techniques manifest in our systems (https://www.goblinloot.net/2023/02/writing-detections-when-stuck-with-edr.html). This is an area of study you can never stop as you are literally competing against other humans who are trying to make sure they don’t land in your basket. The MITRE Attack Framework which captures how adversaries are going to be compromising our clients https://attack.mitre.org/groups/ is a great introduction but you must also be keeping up with threat reports and news. You can use the following list to achieve this https://github.com/QueenSquishy/Zombie/blob/main/Reading/Daily. Luckily in our team, we also have the ability to recreate actual adversarial behaviour and perceive how it would appear in our telemetry and tooling. An interesting observation is that the volume and diversity in which ways adversaries will do this grows quite slowly because they simply take the path of least resistance. Additionally, a more recent trend is adversaries decreasing the sophistication of their attacks as relying on human nature lends to much more gain for much less effort.

For an analyst, this decrease in sophistication means we need to spend a lot more time baselining behaviour and capturing data points related to identities rather than reverse engineering malware or writing signatures, it’s now a shared perception in all teams that identity is the new endpoint and the criticality we treat our detection pipelines and investigations for identity-related alerts must also rise accordingly.

Next, an analyst needs to understand what is their capability to ask questions:

  • How do you establish if the file is unique?
  • How do you see what other behaviour the user was doing?
  • How far back can we look?
  • What asset is involved?
  • Do we need to look at other assets?
  • Are there other detections that can help build a better picture
  • How would the behaviour look if it was an adversary?
  • How would the behaviour look if it was a legit administrator?

There are countless questions like this and as analysts, you need to know how to find the answers, but importantly not just find the answer but confirm what you found hasn’t been manipulated or subject to bias. If there are questions you don’t know how to answer you need to discover how.

On the topic of bias, we are going to cover a few. First and foremost humans love shortcuts if you give them an idea (which an alert will always do) they will be drawn to chase and confirm that idea is true ignoring variables and other legitimate data that might outright disprove or lower the quality of that idea. Being actively conscious of this is really important because going back to the very beginning we need to always consider the distance between analysis and action.

So how can we take control of this bias, well we stick with what we know for certain first, if we triage a brute force alert we know:

  • We are looking for large volumes of events
  • Authentication methods may differ
  • The source of the authentication request may be new

Forcefully covering bases we know will validate an idea will then give us the foundation to be more explorative and ask those riskier questions like:

  • Is the source already compromised?
  • Could it be a novel technique?
  • Is it a misconfiguration issue

The next bias we are going to cover is spotlight bias, this is the concept of what you can or most easily see is the only place you look. This leads to incomplete investigations and can be defeated with a creative mindset. As an example if you are trying to identify when a user first accessed a system and you check for EventID 4624 and it is not present does that mean no one accessed the system? Of course not so let’s widen our spotlight and look at things like

  • Can we see file event activity suggesting the user profile is being built?
  • are there any scheduled tasks set to execute on logon that ran recently?
  • Users browse the internet almost always, can we see browser activity?

Importantly what makes spotlight bias so dangerous is sometimes you cant control where the spotlight lands and it can be difficult to move but as an analyst, you should have both an expansive skill and tool set to help mitigate this.

Next is novelty bias, this one is really simple because humans love fancy shiny new things, this new threat or tool will almost always hold zero relevance to you no matter how cool it is to look at. Of course, you need to make your own assessment on this but do not expend energy on something simply because it is new. Brilliance in the basics always comes first.

Finally, we are going to cover familiarity bias as its one that will strike every analyst and its a primary component to the alert fatigue we all know and love. This bias manifests because as detections were described earlier security technology doesn’t know when something is evil just when it could be so your effectively going to be looking at a lot of “could be” all the time. There are a few things you will always need to be forcefully conscious of to tackle this

  • Are you investigating less the more you see the alert?
  • Do your notes usually look the same across the same alert type?
  • Adversary techniques change, have your investigative questions?
  • Your technology changes, has your investigation expanded appropriately too?

Never ever is there an alert that does not warrant a complete investigation, however, there are some important stipulations.

  • It’s the analysts responsibility to escalate and advocate for issues with the detection
  • Analysts work the alerts all day, they must be actively driving the investigative AND triage process quality
  • A communication channel between analysts and engineers must exist
  • Engineers must reduce the investigative load on analysts as much as possible
  • If a computer CAN do it then a computer SHOULD do it. Keep toil as close to zero as possible
  • Engineers are the innovators for analysts, they must be building new tools, visualisations and case management improvements.
  • Detection pipelines should balance recall and precision, not analysts’ brains.

Summary

The depth of this subject is incredible and if you’re not constantly chasing it, if you’re not constantly looking to improve you will fail. Investigating and analysis is an entire field that very few people ever really study, keep learning and keep stopping adversaries.

Popular

Endpoint on Adrenaline : One

Image
Introduction   This blog series will capture how to maximise the protection of an endpoint using the various technologies in the Defender suite. The controls outlined in the series will each need to have their own considerations taken into account for a given environment but hopefully at a minimum it will be clear just how powerful of an offering Microsoft has.  Unfortunately due to the highly modular and distinct diversity in controls Microsoft offers things are often overlooked and misconfigured when tests are conducted. With this in mind, the shortest possible route to a solution for this is utilizing the recommendations, CSPM and Secure Score technology contained within Defender 365 and Defender for Cloud as it will automatically conduct posture assessments and inform you of things you have missed and may want to turn on (IE dont take these recommendations lightly) Lastly, I would recommend reading the book “Defender for Endpoint in depth” from packt as it showcases the complexitie
Read more

Brilliance in the Basics

Image
Introduction Tired of watching you and your friends get compromised, do exactly what's in this blog and start beating adversaries. Avoiding the memes adversaries win because of simple mistakes and neglect and we all already know what they are so I'm going to list them for you. Its all for free too. Enumerate To put it simply, there shouldn't be anything you don't know about your environment, you should know who all your users are, where all your electronic devices are and what they do, what applications you have and what versions they are. Enumeration is the jet fuel for making good defensive decisions. I wrote about how to enumerate your environment here  Securing your estate: The First Step (goblinloot.net) . Follow these steps and become the arbitrator of your own environment  Your perimeter is a bridge, not a wall Monitor your perimeter as best as you can but always assume it has already been defeated. Monitor endpoint system and process telemetry and southwest traf
Read more

Endpoint on Adrenaline 3

Image
Introduction Now that I have covered the advanced features obtained from Defender for Cloud and the complexities of Defender for Endpoint with the objective of delivering as much protection as possible to the endpoint, I will now explore how to expand on this greatly and open up even more use cases. Microsoft Sentinel This technology will allow us to unify and correlate data generated by other controls. I have decided to deploy Microsoft Sentinel into 4 primary categories : System Administrator - Asset Uptime, Application Inventory, Device configuration and Troubleshooting Network Engineers - Netflow and session traffic, violations and alerts for potential issues Threat Detection - Analytics and playbooks Threat hunting - Workbooks & Machine learning Using Microsoft Sentinel we can achieve a plethora of incredibly powerful use cases with Defender 365 endpoint data. Below is an illustration of how my SIEM works to enable analysts to prevent and detect threats Data Sources In order t
Read more

Writing detections when stuck with EDR

Image
Introduction Making the leap to purchasing and maintaining an EDR solution can be huge for organisations so huge in fact that they never really progress from there when it comes to visibility on assets. This is an important consideration because organisations should aim to achieve some level of detection engineering but will likely never get to explore the associated complexities. So how can organisations effectively and easily write detections that actually help to protect them when they are only able to leverage an EDR tool. Understanding your tools limitations To most people EDR tools are incredibly verbose in the telemetry they capture from any given asset but unfortunately relative to overall telemetry available they actually only capture what the vendors think is the most pertinent. This is for obvious reasons because ingesting telemetry at the scale vendors do comes at a significant cost so they implement "cost-saving measures" like limiting the amount of any one given
Read more

Endpoint on Adrenaline Two

Image
Introduction Continuing on from my last post that captured using Defender for Cloud to gain powerful additional features on top of defender for endpoint to protect your endpoints we are going to take a closer look at Defender for Endpoint itself and extend into some endpoint-specific use cases for Defender for Identity that tie directly back into Defender for Endpoint. Firstly a really important consideration when configuring Defender for Endpoint is that alot of the heavy lifting from a prevention standpoint actually happens in Defender Anti virus. This means that while im going to briefly cover some settings to make sure Defender Anti Virus isnt hindering Defender for Endpoint im going to reserve further Defender Anti Virus controls to the next part in this series. This also of course means that when your doing your own testing just installing the MSSense agent (What you get from the scripts, intune etc) is actually not Defender for Endpoint in its totality so you simply can not m
Read more

Investigate Two

Image
Introductions How you as an analyst handle true positives is life and death in the eyes of potential victims. Traditionally the industry elected to prioritize overzealousness and sending more than not to cover their failings, This however is no different to guesses and it is possible to arrive at strong data-backed decisions on events that could be the compromise of an estate True Positive True positives exist at the heart of all our jobs, they are what we wake up for in the morning and why we endlessly pursue understanding how to protect our clients. The strictest possible definition of true positive is where an outcome of a prediction or model is returned true. This broad definition is important because the assumption carried with a predication or model can change. As an example many security technologies aim to identify when an application often unwanted by an organization is present within a system, these are commonly dubbed PUPs. (Potentially Unwanted Programs). The general cons
Read more

Securing your estate: The First Step

Introduction A commonly forgotten fact within the cybersecurity industry is that most organizations are not equipped nor have started to form any sort of security program. This is the case for many reasons because most organizations are SMBs so they don't and likely will never have funds to purchase people and technology and the fact that the cybersecurity industry is incredibly toxic for low expertise and or low capability teams as the well of knowledge we all rely on has been poisoned by vendors and 'thought leaders'. With this in mind, I have created a small resource that will guide you through what you need to do to make a start. This resource won't reference vendors and I will try to avoid specific jargon. Enumerate The term 'You can't secure what you don't know' stands strong even to this day and should be your primary focus before you do anything else. Understanding what assets, you have both physically (computer hardware) and logically (applicati
Read more

Standardized Note Taking Format For Analysts

Image
  Introduction This post outlines a format for note-taking designed to aid analysts and ensure the knowledge they acquire over time is kept so that it can easily be consumed later. This format is designed to be technology agnostic so it can be applied to a note-taking tool or platform of the analyst’s choice. I designed this format because analysts often received what you could call unplanned information whereby at any given moment during their day they could learn something highly impactful to their role via any medium. This really exasperates the need for good note-taking practices for analysts. Illustrated above is the wireframe that analysts can implement into their note-taking platforms. Each section contains sub-sections that allow for different knowledge to remain segregated and more searchable. Meeting Notes Within any given week analysts can have countless meetings and so it is important to note down items from meetings for future review. Not every meeting needs to be capt
Read more

Attack Simulations for your SOC

Image
 Introduction SOC analysts spend 99 percent of their time looking at the same data and patterns over and over again. They develop muscle memory that helps them use their platforms without much effort but also in grains in them potentially poor practices. Most analysts are new to the industry and are taking certifications like CompTIA Cysa+ and Azure SC200 but none of these certifications teach them what genuine malicious actions look like. So how do you train a team of people how to detect malicious activity and respond in the ever-morphing threat landscape? You throw them through the ring of fire. You make them triage real (as close as possible) incidents and appropriately monitor their behavior and progress to tune the processes perpetually. Analysts are arguably the single most important role within a SOC, they carry the burden of triaging alerts and correctly identifying whether malicious activity is occurring. They are the first, middle and last line of defense. However, despite t
Read more