Goblin Diary #2 - AI Tools for Analysts 🐯

Dont Use AI 

Analyst work is built on the human capacity for creativity, memory recall and information gathering and using so called 'AI Tools' will actively diminish you in these areas. Your ability to form useful thoughts is built on the continued labour of your mind. Each aspect of the labour you endeavour to partake in creates its own unique quality in your own cognition. Your drive connects you to this labour but does not assure you of any benefits. If you ask AI tools to format documents you will get worse at building narratives and if you ask AI tools to process registry modification events you will get worse at pattern recognition. 

After slowly reducing the individual minute qualities of your cognition you will one day be asked to solve a difficult problem. You will fail. You will resign the problem as not possible or unfair blinded by the damage you have already done to yourself. People who lack the capacity to understand even turn to frustration and anger but it will be in vain. Set in you will be behaviours that have overwritten what took perhaps 18 years to develop.

Our world is full of complexity and challenge. Our drive pushes us to explore mysteries and grow. These things build you with incredible depth. Inserting AI into the middle means you are not being built by your experiences you are merely speeding through them. The experiences you can derive from using AI are as shallow as the puddle at the end of every street. Do not trade the ocean for a puddle just because its easier to swim in. Love the labour, Become Better.



Introduction

AI tools are being gradually integrated into Security Operations Centres to either increase the quality of investigations or expand a given analysts capacity to handle workload. This blog post walks through what you should consider when evaluating AI tools for these purposes. In particular the focus of this document is to enable people to review the text outputs and establish whether what the AI tool is generating is valuable.

In the future I will publish further content on how to generate simulations and measure triage responses and case management functions of any given AI tool.

Understanding AI Tool triage efficacy 

When assessing the quality of an AI tool for SOC triage workflows consideration must be taken for each individual component of the process. These components can be articulated as the following:


Investigation 

Investigation describes a process whereby an analyst processes a provided alert, generates potential investigative avenues, weighs those avenues for appropriateness and finally translates selected investigative avenues into actions or tasks that collect evidence. By assessing an AI tools outputs through each abstraction an assessment to how well it will perform during a live incident can be arrived too. 

  • Alert comprehension 
  • Investigative Avenues 
  • Collection of evidence
  • Analysis of evidence 
  • Conclusions 

Alert Comprehension 

In most instances an analyst engages with an investigation due to the triggering of an alert. Alerts capture behaviours that have been observed within telemetry and provide a high level description of what the behaviour is. Analysts apply this information against their own understanding of the relevant systems and adversary behaviour to create a list of potential explanations for the behaviour and also a list of additional behaviours that would further suggest that the alert is a true positive. 

This can be measured through the following products: 
  • Is the AI tools description of the behaviour functionally similar to the alert 
  • If the AI tool has identified a relationship between the behaviour and adversary playbooks 
    • Is the alluded to association known to be correct 
    • Does the AI tool stipulate a confidence rating for the association 
    • Has the AI tool identified other possibilities 
      • items that increase likelihood of a true positive 
      • items that increase likelihood of a false positive

Investigative Avenues 

Once an alerts basic content has been captured a list of unanswered questions is typically generated by an analyst. These questions will either enable an absolute conclusion to the alerts nature or enrich the analyst during further reasoning. Investigative avenues are usually weighed for likely effort expenditure and closeness to previously successful investigations. Where avenues that would take very little effort to explore and are known to be of been useful in past investigations are selected first. 

This can be measured through the following products: 
  • Do the avenues selected by the AI tool reflect those found to be successful in past incidents? 
  • Do the avenues appear relevant to the alert?
  • Are the investigative avenues sorted or ordered by priority? 
    • are justifications provided for sorting? 
  • How broad are the investigative avenues? 
    • do the avenues utilise multiple data sources 
    • do the avenues utilise all the data sources available
  • Did the investigative avenues return results that forwarded the investigative towards a confident conclusion 
    • Is a false positive justification provided 
    • Is a true positive justification provided

Collection of evidence 

Collecting evidence can be considered a demonstration of an analysts forensic ability. Translating investigative avenues into evidence collection requires broad domain knowledge of computer systems and available tools and the rigor to ensure that items collected are easy to process and store.

This can be measured through the following products: 
  • Did the AI tool collect evidence from all the relevant sources 
  • Was the evidence collected easily searchable

Analysis of evidence 

Once evidence has been gathered it must be associated to the drafted investigative avenues and used to provide either an absolute conclusion to what is being explored or justification for further evidence collection. Where an analyst is aiming to understand a behaviour in more detail the evidence should contain data not found in triggering alert and where questions to existence of additional behaviours are being explored the evidence should be from a surface or data source known to be interacted with by adversaries. 

Using the gathered evidence analysts assess whether its properties are suggestive of malicious intent and compare what is being analysed to known good examples. In some instances this is done cognitively particularly for evidence such as authentication attempts from Russia but in others where an evidences properties are highly variable this is achieved through direct comparisons of different data sets. This direct comparison is used to arrive to assertations on how unlike the properties are compared to baseline. Where certain variables are deemed highly unlike a known good baseline these are considered positively contributing to the investigation. For example volume of data downloaded from a SharePoint point site over time and distribution of file types in the download.

This can be measured through the following products:

  • Has the AI tool used the evidence to generate verifiable facts
  • Has analysis of the evidence generated confidence statements
    • likelihood of true positive
    • likelihood of false positive
  • Are references provided for the AI tools reasoning
  • How much of the evidence was used by the AI tool
    • Facts derived
    • Associated to adversary behaviour
  • If minimal or no context is derived from a piece of evidence
    • Does the AI tool prompt for more evidence collection
    • Is reasoning provided for the absence of information

Conclusions

Investigative work must end with conclusions. These conclusions can either be a statement to uncertainty and what work is required to alleviate the uncertainty or a statement to the nature of the behaviours captured in the alert. To arrive to a conclusion analysts fairly evaluate each component of their investigation and associate their findings to their own understanding of how adversary behaviour manifests in the products or systems. Often conclusions contain some uncertainty due to the general nature of computer system openness however this uncertainty should be justified and surrounded by context that describes specific circumstances the behaviour would be a false positive.   

This can be measured through the following products:

  • Did the AI tool conclude a true or false positive
    • Was it correct
  • Is the provided conclusion correlated or linked to evidence
    •  Does the evidence support the conclusion
  • Is the conclusion clear and without ambiguity
  • Does the conclusion provide insight into causality
    • Is evidence used to arrive to causality
  • Is the degree of confidence in the conclusion stated

Assessment of language

Beyond measuring an AI tools outputs for investigative efficacy its important to understand how well the tools write with regards to syntax and semantics. AI tools deployed in the SOC are typically multi modal large language models that tokenise content and arrange it based on probability. This probability is primarily shaped by pre deployment training where a model vendor will control inputs and programmatically adjust parameters and conditions until the model generates outputs that are shaped in a desired way.

These shaped outputs are what analysts will consume when using AI tools to aid in investigations and alert triage so its important to measure whether the model vendor is designing the models to communicate in a way that analysts can apply into their work.

The below general framework has been created to help measure the models outputs when using the measurements described in the ‘Investigations’ section. This section is slightly subjective as the shape and style of communications preferred by an individual analyst is dictated by what they have been exposed to in the past. 

When making an assessment of the AI tools output each item below should be easily extractable from the text:

Point

Where an AI tool is creating an output the point must be clear. What is being said should be easily identified amongst facts or statistics and it should be associated to surrounding context. The point acts as the idea the analyst wields when they read the literature and review any findings.

Reason

A reason is a collection of details on why the point is being made. It is usually the most comprehensive component and reveals to the analyst surrounding supporting context, processes of thought and any assumptions that are being made.

Evidence

Evidence consists of artefacts either forensic data points or facts that any reasonable person would arrive to based on the given information. Often forensic data and facts about the data are paired together to create a piece of evidence. Evidence must have integrity in way that makes it easy to challenge as not hallucinated (references) and be relevant to the point.

Poor output

Considering the three aspects of the outputs above you can identify poor content either through the absence of one or more of the items or where continuity between each aspect is missing. Continuity throughout the text is important to ensure analysts are able to weigh the total set of circumstances where if poor continuity is present it is likely analysts will not understand relationships between different pieces of content nor how to attribute severity or causality due to the fragmented information.

Beyond the three aspects to writing you should also consider whether the AI tool is overly repeating points or highlighting facts or information with significant bias. Analysts using AI tools must be presented with content fairly and while using a structure or shape to the text that makes it clear what is the most critical information can be useful it must not be in a form that obscures other pieces of information. If an AI tool repeats a single point for different distinct purposes in the structure of the text then it is likely acting without sufficient evidence.


Ending

AI tools are still struggling to integrate well with Security Operations Centres particularly due to the Cyber Security industry's tendencies to act without rigor or deliberateness. Ensure that you establish small use cases and aim to perfectly achieve those first before attempting to use AI tools for all analyst work. 



Popular

Brilliance in the Basics

Image
Introduction Tired of watching you and your friends get compromised, do exactly what's in this blog and start beating adversaries. Avoiding the memes adversaries win because of simple mistakes and neglect and we all already know what they are so I'm going to list them for you. Its all for free too. Enumerate To put it simply, there shouldn't be anything you don't know about your environment, you should know who all your users are, where all your electronic devices are and what they do, what applications you have and what versions they are. Enumeration is the jet fuel for making good defensive decisions. I wrote about how to enumerate your environment here  Securing your estate: The First Step (goblinloot.net) . Follow these steps and become the arbitrator of your own environment  Your perimeter is a bridge, not a wall Monitor your perimeter as best as you can but always assume it has already been defeated. Monitor endpoint system and process telemetry and southwest traf...
Read more

Investigate

Image
  Introduction This document details how an analyst should conduct investigations and triage in the normal duties of their job. It will describe concepts and logic that enable analysts to look at data, ask investigative questions, evaluate those questions and arrive at assertions about what they have found. An Alert For the most part, analysts begin their investigative work when an alert is generated. Alerts are fantastic because they are a statement that something has happened and something needs to be done about it. Typically alerts are filled with information that an analyst can then use in their investigation. Alerts are generated by detections and these detections can come in many shapes and sizes. Some just look for a particular action like someone unlocking their car door, others might have access to more context like someone who isn’t the owner of the car unlocking the door. Going even further some detections might look at how the car door was opened or at w...
Read more

Endpoint on Adrenaline : One

Image
Introduction   This blog series will capture how to maximise the protection of an endpoint using the various technologies in the Defender suite. The controls outlined in the series will each need to have their own considerations taken into account for a given environment but hopefully at a minimum it will be clear just how powerful of an offering Microsoft has.  Unfortunately due to the highly modular and distinct diversity in controls Microsoft offers things are often overlooked and misconfigured when tests are conducted. With this in mind, the shortest possible route to a solution for this is utilizing the recommendations, CSPM and Secure Score technology contained within Defender 365 and Defender for Cloud as it will automatically conduct posture assessments and inform you of things you have missed and may want to turn on (IE dont take these recommendations lightly) Lastly, I would recommend reading the book “Defender for Endpoint in depth” from packt as it showcases the co...
Read more

Endpoint on Adrenaline 3

Image
Introduction Now that I have covered the advanced features obtained from Defender for Cloud and the complexities of Defender for Endpoint with the objective of delivering as much protection as possible to the endpoint, I will now explore how to expand on this greatly and open up even more use cases. Microsoft Sentinel This technology will allow us to unify and correlate data generated by other controls. I have decided to deploy Microsoft Sentinel into 4 primary categories : System Administrator - Asset Uptime, Application Inventory, Device configuration and Troubleshooting Network Engineers - Netflow and session traffic, violations and alerts for potential issues Threat Detection - Analytics and playbooks Threat hunting - Workbooks & Machine learning Using Microsoft Sentinel we can achieve a plethora of incredibly powerful use cases with Defender 365 endpoint data. Below is an illustration of how my SIEM works to enable analysts to prevent and detect threats Data Sources In order t...
Read more

Endpoint on Adrenaline Two

Image
Introduction Continuing on from my last post that captured using Defender for Cloud to gain powerful additional features on top of defender for endpoint to protect your endpoints we are going to take a closer look at Defender for Endpoint itself and extend into some endpoint-specific use cases for Defender for Identity that tie directly back into Defender for Endpoint. Firstly a really important consideration when configuring Defender for Endpoint is that alot of the heavy lifting from a prevention standpoint actually happens in Defender Anti virus. This means that while im going to briefly cover some settings to make sure Defender Anti Virus isnt hindering Defender for Endpoint im going to reserve further Defender Anti Virus controls to the next part in this series. This also of course means that when your doing your own testing just installing the MSSense agent (What you get from the scripts, intune etc) is actually not Defender for Endpoint in its totality so you simply can not m...
Read more

Investigate Three

Image
 Analysis  In this post, I explain analysis and the associated techniques to mean at the lowest possible level a human’s ability to consume external stimulus of its near-infinite complexity and produce thoughtful and data-backed decisions. Within our industry, we are plagued with information twisted and corrupted by entities wishing to profit, As such this is where your analysis starts as a human. At every moment at every crossroad, you must take a data point and allow it to flower into thousands of other thoughts, then you must evaluate each subsequent thought and draw a line back to “what is already” known, The further away the thought the thinner the line back to known but the greater room the flower to bloom. That is to say don’t discount radical ideas. When presented with an alert security vendors expend significant energy into putting “options” available to the individual reviewing the alert. In general, there will never be an option you should not explore what is key ...
Read more

Investigate Two

Image
Introductions How you as an analyst handle true positives is life and death in the eyes of potential victims. Traditionally the industry elected to prioritize overzealousness and sending more than not to cover their failings, This however is no different to guesses and it is possible to arrive at strong data-backed decisions on events that could be the compromise of an estate True Positive True positives exist at the heart of all our jobs, they are what we wake up for in the morning and why we endlessly pursue understanding how to protect our clients. The strictest possible definition of true positive is where an outcome of a prediction or model is returned true. This broad definition is important because the assumption carried with a predication or model can change. As an example many security technologies aim to identify when an application often unwanted by an organization is present within a system, these are commonly dubbed PUPs. (Potentially Unwanted Programs). The general cons...
Read more

Writing detections when stuck with EDR

Image
Introduction Making the leap to purchasing and maintaining an EDR solution can be huge for organisations so huge in fact that they never really progress from there when it comes to visibility on assets. This is an important consideration because organisations should aim to achieve some level of detection engineering but will likely never get to explore the associated complexities. So how can organisations effectively and easily write detections that actually help to protect them when they are only able to leverage an EDR tool. Understanding your tools limitations To most people EDR tools are incredibly verbose in the telemetry they capture from any given asset but unfortunately relative to overall telemetry available they actually only capture what the vendors think is the most pertinent. This is for obvious reasons because ingesting telemetry at the scale vendors do comes at a significant cost so they implement "cost-saving measures" like limiting the amount of any one given...
Read more

Standardized Note Taking Format For Analysts

Image
  Introduction This post outlines a format for note-taking designed to aid analysts and ensure the knowledge they acquire over time is kept so that it can easily be consumed later. This format is designed to be technology agnostic so it can be applied to a note-taking tool or platform of the analyst’s choice. I designed this format because analysts often received what you could call unplanned information whereby at any given moment during their day they could learn something highly impactful to their role via any medium. This really exasperates the need for good note-taking practices for analysts. Illustrated above is the wireframe that analysts can implement into their note-taking platforms. Each section contains sub-sections that allow for different knowledge to remain segregated and more searchable. Meeting Notes Within any given week analysts can have countless meetings and so it is important to note down items from meetings for future review. Not every meeting needs to be ...
Read more