“Cyber Terrain”: A Model for Increased Understanding of Cyber Activity

In my first Science of Security post, I recommended that organizations consider hiring cyber security scientists to help organizations in developing a strong, rigorous scientific foundation to cyber security while providing structure and organization to a broad-based body of knowledge on the domain. In my second Science of Security post I provided an example of applying the scientific method to cyber security operations. In that second post I mentioned that every cyber attack has 4 basic components: Threat Actors, TTPs, Cyber Terrain, and Defenders. In this post I’ll introduce a multilayered Cyber Terrain model and describe how this can be used to help systematically organize the broad-based body of cyber security knowledge to enable increased understanding amongst cyber participants.

Traditional “terrain” maps show physical features to help users better understand the terrain and how best to navigate the terrain. To apply this concept to cyberspace we will leverage a multilayered cyber terrain model that allows us to conceptually model, organize, and understand the features (laws, policy, security technology, etc.) and activity (cybercrime, APT, hacktivism, etc.) that takes place in the cyber terrain.

Cyber terrain can be used to model the Defender’s cyber terrain, the Threat Actor’s cyber terrain, and just about everything that makes up the Internet of Things (IoT). Most of the time when I mention cyber terrain to people they tend to visualize routers, switches, and other physical hardware. While the hardware is part of the cyber terrain it only represents 1 of the 15 layers that I’ll introduce in today’s post.

A large body of research indicates that visual cues help us to better retrieve and remember information. The research outcomes on visual learning makes complete sense when you consider that our brain is mainly an image processor since much of our sensory cortex is devoted to vision. The cyber terrain model allows us to visualize the cyber terrain to help accelerate learning and promote a shared understanding.

Cyber terrain is a concept developed by the U.S. Department of Defense as an updated defense in depth model. The original defense in depth multilayered model was introduced a long time ago and was focused primarily on addressing the path of data in and out of the network (OSI layers 2, 3, & 4). The cyber terrain model is designed to address both the path of the data in and out of the network as well as what happens after the data arrives. The cyber terrain provides a more comprehensive approach since one can use it as a lens to view activity from both the defensive aspect, but also from threat actor perspective, which can reveal critical information needed to better defend against an attack.

Defense in depth is still a good defensive strategy, but it is limited since its focus was solely defense and only focused on the network layers. It also doesn’t account for key aspects of threats such as geolocation, persona, etc.

The cyber terrain model I’m introducing in this post builds off the efforts of the U.S. DoD and is designed to represent the full triangle of sustainment or the three pillars of cybersecurity: People – Organizations & Processes – Technology. To achieve this we need a model that represents both physical, real-world layers as well as the virtual layers of cyberspace. A model capable of representing the data, technology, people, processes, activity across all the traditional security engineering, security operations, and security intelligence areas on the defender side as well as the threat actors, TTPs, and infrastructure used by the bad guys. The cyber terrain model can also enable a shared understanding by engineers, operators, analysts, executives, and board members. This lead to the creation of the 15 layer cyber terrain model shown below. 

Layer 0 – Geographic Layer

The geographic layer represents the geographic area where real-world devices, people, organization buildings, and other physical items resides. The geographic location of physical items helps to give context to applicable cyber laws, policies, etc that apply to items in specific geographic areas as represented in the government layer. For example, a company with physical offices in both the United States and the United Kingdom would have different government level cyber laws that apply to that organizations people and technology depending on where the people and technology are located geographically. 

This layer can also represent geographic location attack vectors such as leaving a few BadUSB infected USB thumb drives in the parking lot outside the office of a targeted organization. The layer can also represent risk from natural threats that affect an organization’s people and technology in specific geographic areas such as earthquakes or flooding.

·         CAPEC-ID:406 – Social Information Gathering via Dumpster Diving https://capec.mitre.org/data/definitions/406.html

·         CAPEC-ID:407 – Social Information Gathering via Pretexting https://capec.mitre.org/data/definitions/407.html

o   CAPEC-ID:413 – Pretexting via Tech Support https://capec.mitre.org/data/definitions/413.html

o   CAPEC-ID:414 – Pretexting via Delivery Person https://capec.mitre.org/data/definitions/414.html

·         CAPEC-ID:507 – Physical Theft  http://capec.mitre.org/data/definitions/507.html

·         CAPEC-ID:391 – Bypassing Physical Locks http://capec.mitre.org/data/definitions/391.html

Layer 1 – Physical Layer

The physical layer represents the physical layer of the OSI model and includes all the hardware, cables, etc. This layer includes physical security and controlled access spaces such as locked server rooms. It’s important to keep in mind that items in the physical layer actually exist and therefore have a location, this means there is a strong link between the geographic layer and the physical layer. Here are a few examples of common attack patterns at the physical layer:

·         CAPEC-ID:507 – Physical Theft  http://capec.mitre.org/data/definitions/507.html

·         CAPEC-ID:391 – Bypassing Physical Locks http://capec.mitre.org/data/definitions/391.html

·         CAPEC-ID:397 – Cloning Magnetic Strip Cards http://capec.mitre.org/data/definitions/397.html

·         CAPEC-ID:547 – Physical Destruction of Device or Component http://capec.mitre.org/data/definitions/547.html

·         CAPEC-ID:453 – Malicious Logic Insertion via Counterfeit Hardware https://capec.mitre.org/data/definitions/453.html

·         CAPEC-ID:455 – Malicious Logic Insertion via Inclusion of Counterfeit Hardware Components https://capec.mitre.org/data/definitions/455.html

Layers 2-7 – Logical Layers (Communications Ports and Protocols)

The logical layers represent the upper 6 layers of the OSI model which enables us to model the communications ports and protocols of the cyber terrain. A defender might share that an observable indicator for beaconing activity by a threat actor’s TTP is a specific pattern observed in packets at the network layer, or a pattern in an http GET request at the application layer. Here are a few examples of common attack patterns in the logical layers:

·         CAPEC-ID:383 – Harvesting Usernames or UserIDs via Application API Event Monitoring (Application Layer) https://capec.mitre.org/data/definitions/383.html

·         CAPEC-ID:293 – Traceroute Route Enumeration (Network Layer & Transport Layer) https://capec.mitre.org/data/definitions/293.html

·         CAPEC-ID:309 – Network Topology Mapping (Network Layer, Transport Layer, & Application Layer) https://capec.mitre.org/data/definitions/309.html

·         CAPEC-ID:311 – OS Fingerprinting (Network Layer, Transport Layer, & Application Layer) https://capec.mitre.org/data/definitions/311.html

·         CAPEC-ID:316 – ICMP Fingerprinting Probes (Network Layer) https://capec.mitre.org/data/definitions/316.html

·         CAPEC-ID:310 – Scanning for Vulnerable Software (Network Layer, Transport Layer, & Application Layer) https://capec.mitre.org/data/definitions/310.html

·         CAPEC-ID:315 – TCP/IP Fingerprinting Probes (Network Layer, Transport Layer, & Application Layer) https://capec.mitre.org/data/definitions/315.html

·         CAPEC-ID:312 – Active OS Fingerprinting (Network Layer) https://capec.mitre.org/data/definitions/312.html

·         CAPEC-ID:291 – DNS Zone Transfers (Application Layer) https://capec.mitre.org/data/definitions/291.html

·         CAPEC-ID:307 – TCP RPC Scan (Transport Layer) https://capec.mitre.org/data/definitions/307.html

The different layers that make up the cyber terrain allows us to breakdown activity by layer and to consider countermeasures or security controls that might apply to the different layers. They also help to increase understanding when sharing information with other organizations or describing observed activity to other defenders.

DDoS attacks have made the headlines several times over the past decade but breaking it down by layers helps to provide more actionable information and increased understanding. Consider the below image from National Cybersecurity and Communications Integration Center at the U.S. Department of Homeland Security which shows DDoS attack possibilities by OSI Layer. 

                                     Source: https://www.us-cert.gov/sites/default/files/publications/DDoS%20Quick%20Guide.pdf

Layer 8 – Machine Language 

The machine language layer is used to represent data such as binary executables, class files, shared libraries (e.g., DLLs), or other machines code. The machine language layer also includes items such as embedded system such as those used in SCADA systems, BIOS, and firmware on various devices such as video cards and storage devices.

·         CAPEC-ID:37 – Lifting Data Embedded in Client Distributions https://capec.mitre.org/data/definitions/37.html

·         CAPEC-ID:190 – Reverse Engineer an Executable to Expose Assumed Hidden Functionality or Content https://capec.mitre.org/data/definitions/190.html

·         CAPEC-ID:205 – Lifting Credential Key Material Embedded in Client Distributions https://capec.mitre.org/data/definitions/205.html

Layer 9 – Operating System

The operating system layer is used to represent the operating systems used by the defender or the threat actor to include operation system weaknesses, vulnerabilities, security configuration issues, and attack patterns. Here are a few common attack patterns for the operating system level:

·         CAPEC-ID:9 – Buffer Overflow in Local Command-Line Utilities http://capec.mitre.org/data/definitions/9.html

·         CAPEC-ID:45 – Buffer Overflow via Symbolic Links http://capec.mitre.org/data/definitions/45.html

·         CAPEC-ID:8 – Buffer Overflow in an API Call http://capec.mitre.org/data/definitions/8.html

·         CAPEC-ID:14 – Client-side Injection-induced Buffer Overflow http://capec.mitre.org/data/definitions/14.html

·         CAPEC-ID:118 – Gather Information http://capec.mitre.org/data/definitions/118.html

·         CAPEC-IDS:268 – Audit Log Manipulation https://capec.mitre.org/data/definitions/268.html

·         CAPEC-ID:270 – Modification of Registry Run Keys https://capec.mitre.org/data/definitions/270.html

·         CAPEC-ID:17 – Accessing, Modifying or Executing Executable Files http://capec.mitre.org/data/definitions/17.html

Layer 10 – Software Application

The software application layer is used to represent software applications installed across the different operating systems. Not only does this include the application code itself, but also the necessary application and service infrastructure used to support the application execution, such as web servers, .Net framework, OSGi, etc.  These execution containers may also reveal critical information that could be used by adversaries to better understand an attack surface or leak information about the organization due to insecure configuration.

This layer is also used to represent secure coding, software application configuration issues, vulnerabilities, weaknesses, and attack patterns. This is also where languages that are compiled to bytecode, such as Java and .Net reside.  In recent days, languages utilizing bytecodes have become a popular target by attackers. Here are some examples attack patterns that include machine code.

This is one of the most popular layers for attacks when you consider software applications such as browsers (Internet Explorer, Firefox, Safari, Chrome, etc) and office applications (MS Office, Adobe, etc). Here are just a few examples of the types of attack patterns we might see at this level.

·         CAPEC-ID:69 – Target Programs with Elevated Privileges http://capec.mitre.org/data/definitions/69.html

·         CAPEC-ID:118 – Gather Information http://capec.mitre.org/data/definitions/118.html

·         CAPEC-ID:76 – Manipulating Input to File System Calls https://capec.mitre.org/data/definitions/76.html

·         CAPEC-ID:35 – Leverage Executable Code in Non-Executable Files http://capec.mitre.org/data/definitions/35.html

·         CAPEC-ID:472 – Browser Fingerprinting http://capec.mitre.org/data/definitions/472.html

·         CAPEC-ID:13 – Subverting Environmental Values http://capec.mitre.org/data/definitions/13.html

·         CAPEC-ID:46 – Overflow Variables and Tags http://capec.mitre.org/data/definitions/46.html

Layer 11 – Persona

The personal layer is used to represent the various ways in which people are represented in cyberspace such as user accounts, userIDs, email addresses, phone numbers, etc. This can include full credentials that allow access to information. A single person can have multiple persona identifies in cyberspace, a common tactic used by threat actors to better hide themselves.

Persona accounts are normally the first level of technical attribution as defenders discover threat actor persona accounts that are tied to specific TTPs (Phising, malicious domain registrations, carding, etc). Since persona details represent humans in cyberspace, they could reveal attributes that could potentially lead to a specific person or organization.

This information could be gathered through open source intelligence, taken as part of an attack on an information system, obtained from the domain registrations, or perhaps gathered through monitoring of threat actors in underground forums and black market sites. 

Here are just a couple simple examples of attacks patterns dealing with persona information:

·         CAPEC-ID:404 – Social Information Gathering Attacks https://capec.mitre.org/data/definitions/404.html

·         CAPEC-ID:383 – Harvesting Usernames or UserIDs via Application API Event Monitoring https://capec.mitre.org/data/definitions/383.html

·         CAPEC-ID:156 – Deceptive Interactions https://capec.mitre.org/data/definitions/156.html

·         CAPEC-ID:151 – Identity Spoofing https://capec.mitre.org/data/definitions/151.html

·         CAPEC-ID:98 – Phishing https://capec.mitre.org/data/definitions/98.html

·         CAPEC-ID:163 – Spear Phishing https://capec.mitre.org/data/definitions/163.html

·         CAPEC-ID:164 – Mobile Phishing  (aka MobPhishing) https://capec.mitre.org/data/definitions/164.html

Layer 12 – People / Supervisory / Temporal

Unlike the persona layer which focuses on the various forms of identify that a human has in cyberspace, the People, Supervisory, and Temporal layer is used to represent the real-world people (the actual individual) such as defenders and threat actors, supervisory functions such as starting, stopping, modifying, or redirecting a cyber operation, and the temporal data surrounding activity in the cyber terrain. All operations in cyberspace begin with a human being and this is the layer in which actual human beings are represented. This could be money mules, carders, APT actors, botnet operators, defenders, etc. Ideally, defenders want to identify who the actual human person is behind the activity for the purpose of prosecution.

Below are a few high-level attack patterns aimed at the humans in the loop.

·         CAPEC-ID:404 – Social Information Gathering Attacks https://capec.mitre.org/data/definitions/404.html

·         CAPEC-ID:410 – Information Elicitation via Social Engineering https://capec.mitre.org/data/definitions/410.html

·         CAPEC-ID:416 – Target Influence via Social Engineering https://capec.mitre.org/data/definitions/416.html

·         CAPEC-ID:527 – Manipulate System Users https://capec.mitre.org/data/definitions/527.html

·         CAPEC-ID:156 – Deceptive Interactions https://capec.mitre.org/data/definitions/156.html

·         CAPEC-ID:98 – Phishing https://capec.mitre.org/data/definitions/98.html

·         CAPEC-ID:163 – Spear Phishing https://capec.mitre.org/data/definitions/163.html

·         CAPEC-ID:164 – Mobile Phishing  (aka MobPhishing) https://capec.mitre.org/data/definitions/164.html

Layer 13 – Organization

The organization layer allows us to represent organization policies, processes, and procedures that apply to the defender’s organization. These could be the organization’s own items or those of another organization. An example might be security benchmarks from the Center for Internet Security or standards from the International Organization for Standardization (ISO). This could also be a threat actor’s organization such as the Hacktivist organization Anonymous, an underground carding organization, or a foreign competitor’s organization.

Much like persona accounts represent real people in cyberspace, organizations have their own identities in cyberspace. Some cyber activity might only be attributed to an organizational level identity such as SEA / Syrian Electronic Army. It’s important to try to link threat actor persona accounts with the organization they belong to for better overall attribution.  

Layer 14 – Government

The government layer allows us to represent government items such as cyber laws, regulation, frameworks, and data. For example, in the United States there are more than 50 statutes that address various aspects of cybersecurity either directly or indirectly to include things such as the Privacy Act of 1974, the Counterfeit Access Device and Computer Fraud and Abuse Act of 1984. The NIST Cybersecurity Framework. Vulnerability and Security Configuration information from the National Vulnerability Database. This layer can also represent alleged government associations of threat actors such as the Mandiant APT1 campaign association with the Chinese military / government, the alleged ties to the Iranian government behind the DDoS attacks on financial institutions, or the alleged connection to the Russian government behind the recent JPMorgan breach.

Cyber Terrain Analysis

Now that we have covered the different layers of the cyber terrain we can consider cyber terrain analysis. I won’t go into much detail on cyber terrain analysis in this post but a quick overview of key points should help get people thinking in the right direction.

The U.S DoD uses a process called OCOKA for traditional terrain analysis. OCOKA is an acronym for Observation, Cover and Concealment, Obstacles, Key Terrain, and Avenues of Approach. These all directly map to the cyber terrain. Let’s look at each of these steps below:

·         Observation – What can be seen and where? Where are the various sensors in the different layers of cyberspace within the defender’s cyber terrain? What can those sensors see?

·         Cover and Concealment – What can I hide from threat actor observation? Consider all the information exposed about operating systems and version numbers to resources outside the defender’s cyber terrain. A good example to show leaked information is http://www.shodanhq.com/

·         Obstacles – How can I make it harder to attack? This could be technology or process driven mitigations and countermeasures for each attack pattern applicable to the defender’s cyber terrain in order to limit movement within the network. This is generally called Risk Remediation Analysis in the cybersecurity community.

·         Key Terrain – Key assets, accounts, data, etc. Within the cybersecurity world this is generally known as Crown Jewels analysis. Losing these to a threat actor would be a significant defeat for the defender.

·         Avenues of Approach – This is the various paths that can be taken to exploit a target. Consider both the exact paths into and out of your network along with what specific attack patterns apply based on the specific assets (software applications, operating systems, etc) inside the defender’s cyber terrain. This is generally referred to as Threat Susceptibility Analysis in the cybersecurity community.

o   Exploit Target – When considering the avenues of approach it helps to analyze the exploit target of each attack pattern based on the assets present in the defender’s cyber terrain. This include:

§  Security Configuration Issues – example CCEs

§  Software Vulnerabilities ­– example CVEs

§  Software Weaknesses – example CWEs

§  People – example social engineering

When modeling adversaries with the cyber terrain, layers 11-14 are almost like filters to be applied to the layers below when correlated with attack patterns as they can be used as means to better understand the kinds of attacks that an enterprise might see from the different types of threat actors. The attack patterns help to identify the weaknesses, vulnerabilities, and configuration issues that threat actors would typically look to exploit.

By looking at the critical assets (key terrain) within the defender’s cyber terrain, through techniques such as crown jewels analysis, and then determining who would want those assets and why, defenders can better understand the kinds of threat actors that they are likely to face, the patterns of attack associated with those types of threat actors, and ultimately the weaknesses, vulnerabilities, and configuration issues that are likely to be exploited in an attack. With this data, mechanisms for better detection and prevention could be put into place across different cyber terrain layers.

Summary

In this post I presented readers with a new 15 layer cyber terrain model that enables organizations to organize a broad based body of cybersecurity knowledge and visualize the physical and logical parts of the cyber terrain. There are strong relations between the layers and activity can be observed across different layers.

By breaking down the cyber terrain to individual layers we are presented with a new way to analyze and understand the complexities of modern cyber operations layer by layer where we can consider both technical and policy based mitigations and countermeasures for each layer of the defenders cyber terrain.

A fundamental aspect of intelligence preparation of the operational environment (IPOE) is detailed terrain analysis to include the threat actors, their TTPs, attack patterns, and use of the cyber terrain as well the defender’s TTPs, cyber terrain, and key terrain. Modern threat intelligence can include actionable information for each of the 15 cyber terrain layers. Using a multilayered cyber terrain model can help us to organize this knowledge to support increase understanding and accelerate learning while advancing an organization’s intelligence driven security program.

Getting Inside the Threat Actor’s OODA Loop to Stop Undetectable TTPs

In my last Science of Security post I shared information on the Science of Security discipline and the core themes being researched in the discipline. I recommended in that post that organizations should consider hiring cyber security scientists to help organizations in developing a strong, rigorous scientific foundation to cyber security while providing structure and organization to a broad-based body of knowledge in the domain.

In this post I’ll walk us through a 2009 experiment I conducted in the Science of Security core theme of Attack Analysis. While this experiment is a good 5 years old and cyber tradecraft has moved forward since then, it provides a good example of applying the scientific method to the hard cyber security problems that many organizations face when dealing with the escalating threats in cyberspace. As we progress through the different steps of the scientific method I’ll align each step in the example to other common processes familiar in both cyber and business operations such as the OODA Loop, PDCA cycle (ISO-9001), Intelligence cycle, and Intelligence Feedback loop to help foster greater understanding. I've also created a graphic to help visual learners see how the various cycles align.

What is the scientific method?

The scientific method is the process by which science is carried out. Science builds on previous knowledge, and this can lead to improvements and refinements over time. The process starts with a question. Scientists then formulate a general theory about what the answer is, conduct research to form a more specific hypothesis with predictions of what we think will happen, develop procedures to test the hypothesis, conduct an experiment to prove / disprove the hypothesis, measure the results (observations vs predictions), and complete the process by publishing the results to formally capture the knowledge.

Question – What’s the hard problem?

Most of the organizations I’ve worked with identify hard problems or challenges that the organization is facing and where the organization would like help in finding a solution. The question for our 2009 example was asked by a senior executive in response to “What keeps you up at night?”

How do I defend against threat actors who are conducting active cyber operations using TTPs that are currently undetected by the commercial cyber security solutions providing defense in depth in my enterprise? How can I increase detection and prevent more of these attacks from being successful?

I’ve heard this question stated a few different ways by a few different leaders over the years and it’s really a challenge we all face. As a scientist it’s my job to build and expand the knowledge in the domain. I’m going to develop a general theory based on what is known and then conduct research to make more specific predictions with a hypothesis that can be tested by experiment. The results of the scientific method can be shared with other scientists for them to validate the results or shared with engineers who can leverage the new knowledge to develop new solutions. This helps organizations understand where they should invest their limited time and resources to get the best return on investment. As the old saying goes, knowledge is power.

Theory

When developing a theory, you have to really examine and dissect the question. A theory has been extensively tested and is generally accepted, while a hypothesis is a speculative guess that has yet to be tested. My theory for this hard problem was the following:

To defend against threat actors who are conducting active cyber operations using TTPs that are currently undetected by commercial cyber security solutions the defender has to get inside the threat actors attack cycle or OODA loop (between when the threat actor starts an attack and the time they target the defender’s organization) in order to identify and develop countermeasures that will render the attack unsuccessful and give the defender an advantage.

Whether you spent time in military operations or business operations, it’s widely accepted that if you’re spinning your operations tempo faster than your competitor or adversary you’ll come out ahead. This is the basis of the decision cycle developed by USAF Colonel John Boyd called the Observe-Orientate-Decide-Act (OODA) loop.

The OODA loop focuses on strategic requirements and works well with the Plan-Do-Check-Act (PDCA) cycle which focuses on the operational or tactical requirements which together enables organizations to perform adaptive (cyber defense) cycles . The OODA loop also serves to explain the nature of surprise and shaping operations in a way that unifies Gestalt psychology, cognitive science, and game theory. We want to shape the threat actor’s active operation to the defenders advantage. The threat actor will be surprised the TTP they worked so hard to make undetectable was not successful against the defending organization which could in turn generate doubt and confusion for the threat actor.

Research

The research phase of the scientific method is where scientists identify sources of raw data and information needed to form an explanatory hypothesis with predictions, identify resource requirements, and research what procedures need to be followed to test the hypothesis.

The theory here is that the defending organization needs to observe that the unknown threat actor has started an attack cycle, orientate on the threat actor’s undetectable TTPs, decide which observable patterns are indicators for the TTP, and to take action to defend against the TTP. Research needs to figure out how to take this general theory and form a more specific and testable hypothesis.

The scientist has to also consider the human elements at play in the experiment such as the level of maturity of the defending organization and the level of maturity of the threat actors.

The defender had implemented defense in depth, conducted user training, carried out routine cyber hygiene, and had analysts focused on cybercrime and advanced persistent threats (APT) in addition to traditional security operations. In this case the defending organization was in the upper half of the maturity model.

The threat actors are unknown so the level of maturity will be assessed based on the leveraged TTP and level of TTP detection by cyber security solutions. For this hypothesis and experiment we focused only on threat actor TTPs that had little or no detection (< 10%) by security solutions. Threat actors were assessed to be in the mid to upper half of the maturity model if they were able to render their TTP to have little or no detection during active cyber operations.

The scientist has to understand what tactics, techniques, and procedures (TTPs) of the threat actor is observable. In other words, TTP data that can the defender see with their eyes or via technology based sensors in the cyber terrain. TTPs really fall into 2 general categories. Attack Patterns and Malware.

Attack patterns are blue prints for the process or method used by the threat actor when conducting social engineering attacks, supply chain attacks, communications attacks, software attacks, physical security attacks, or hardware attacks. CAPEC is an example of a free, publically available, community developed list of common attack patterns. Threat intelligence analysts identify which attack patterns are present when analyzing threat actor’s full attack cycles.

https://capec.mitre.org/

Mature defending organizations will also leverage attack patterns and common attack pattern ID numbers in security operations and security engineering when conducting penetration testing and secure code review as a standardized way to capture knowledge about how specific parts of an attack are designed and executed, while providing the attacker’s perspective on the problem and the solution, and also offers guidance on ways to mitigate the attack’s effectiveness. Attack patterns focus on the human threat actor’s TTPs whereas malware is the threat actor’s configured and deployed technological TTP.

Malware is the type of TTP we’ll focus on in this example because malware analysis data contains the observable information the defender needs to orient on in order to decide on the best action to take to defend against the TTP. Remember, we don’t know who the threat actors are, how they are delivering the malware TTP to their targeted victims, or which specific organizations are being targeted by the threat actors.

One of the predictions we’ll test is whether or not there actually are unknown attacks by unknown threat actors that the defender can discover and whether or not they can develop countermeasures before their organization gets hit. Since we don’t know who the threat actor(s) is, how they are delivering the TTP or who the targeted victims are, the defending organization has no direct indications and warnings that an attack is even happening against other organizations or if their organization will be targeted. The scientist has to look at all the components of the attack during the research phase to discover possible sources of information that can be used.

Every cyber attack has 4 basic components: Threat Actors, TTPs, Cyber Terrain, and Defenders. Cyber security scientists have to analyze each of these items to include getting inside the heads of both the threat actor and the defender. We’ve identified that we don’t know who the threat actor(s) is, what TTP they are using, or what cyber terrain the threat actor is using to deliver the TTP or which defender organizations are being targeted. An assumption we are testing is that threat actors have successfully delivered the TTP to unknown defender organizations so we need to focus on possible indications and warnings we could discover based on the typical courses of action carried out by the defending organization.

We need to get into the mind of the defender. When a defender discovers malware or a suspicious file inside their organization through triage, an alert user reporting a suspicious email, or during incident response, the defender wants to analyze the suspicious file or malware to understand the observable behaviors and actions it performs. If the defender’s organization doesn’t have internal malware analysis tools the defender will normally leverage community tools in the public domain such as VirusTotal, ThreatExpert, or other similar online solutions. This presented an opportunity for us to possibly discover active attacks that we were previously unaware of based on what others are seeing prior to the attack targeting our organization.

Remember, this experiment example was from 2009, threat intelligence and threat sharing didn’t really emerge in the mainstream cyber security community until around 2011-2012. While there was limited sharing of very basic threat intelligence, it was primarily ad-hoc silos in industry verticals and these sources of information weren’t considered during this research nor was commercially available threat data like black lists. That data was already being used by the defending organization so the indicators produced during this research experiment would supplement any existing threat sharing and commercial available threat data.

For this research experiment we would focus on malware analysis reports from the public domain that were produced during the last 24 hours that 1) had little (< 10%) or no antivirus detection and 2) contained observable command and control communications. The malware analysis information meeting these conditions represents the “undetectable TTPs” leveraged by unknown threat actors who have targeted unknown defending organizations.

Inside the malware analysis data, command and control communications were selected to be the observable indicators for the threat actor’s TTP (the malware). This data was selected because this is the phase of the attack cycle that is observable within the defending organization’s cyber terrain just after the TTP was delivered and successfully installed on an internal asset as it attempts to notify the threat actor it successfully breached the targeted organization. If we can stop the TTP at this last line of defense, the threat actor will not know where specifically the threat was stopped in the attack cycle after it was delivered to the defender’s organization.

Now that we’ve identified the sources of information needed and the type of information available to us, we can solidify our hypothesis, develop our procedures, and resource requirements. While the research phase only required the cyber security scientist, conducting affordable experiments in real world cyber operations requires leadership buy in and approval of additional resources. For this effort, the following resources were identified and approved:

3 hours per day for a Threat Intel Analyst to analyze the malware analysis data from the previous 24 hours.

1 hour per day for planning intelligence driven courses of action (COA)

3 hours per day for implementing intelligence driven courses of action for mitigation and countermeasures.

1 hour per day for the lead scientist to oversee and monitor the ongoing experiment and overall research project.

Total human resources needed was equal to 1 full time employee divided into different roles for the 6 months of the experiment. The lead scientist was also covered full time during the research, measurement, and conclusion phases of the scientific method that occurred before and after the 6 month experiment.

No special hardware or software was needed to carry out the experiment.

The research phase of the scientific process aligns to the collection and processing phase of the intelligence cycle. The scientific process needed to develop sources of information to collect and process to help answer the question just as the intelligence cycle develops sources of information to collect and process to respond to a request for information (RFI) or ongoing information needs. This also aligns to the observation phase of the OODA loop where the defender must observe the information coming from these sources in order to analyze or orientate on the observations.

Hypothesis

A hypothesis is an educated guess about how things work. Most of the time a hypothesis is written like this: "If _____[I do this] _____, then _____[this]_____ will happen." The hypothesis should be something we can actually test where we can measure if the predictions match the real world observables. This phase of the scientific process is aligned to the intelligence cycle analysis phase and the orientation phase of the OODA loop. Scientists and intelligence analysts want to predict or forecast what will happen based on their analysis.

For this research experiment we can hypothesis the following:

If we monitor online malware analysis sites then we can identify TTPs containing command and control communications where the TTP had little (< 10%) or no antivirus detections that had been submitted and analyzed in the past 24 hours.

If we can identify TTPs containing command and control communications with little (< 10%) or no antivirus detection that were submitted and analyzed during the previous 24 hours then we can produce a daily threat intelligence product on the TTP with the command and control communications as observable indicators and a recommended course of action the defending organization should take to mitigate the threat actor’s TTP.

If we can produce and disseminate the daily threat intelligence report to security operations then we can plan and implement courses of actions based on the intelligence.

If we can implement the recommended course of action using the observable indicators provided from the threat intelligence to block the TTP’s command and control communications then we can prevent the attack from being successful since the threat actor doesn’t know where in the attack cycle the defender stopped the TTP.

If we can detect the blocked command and control observable indicators for the TTP within the defender’s cyber terrain then we can identify which internal asset the command and control information is coming from to contain the incident, and through the internal investigation and incident response process we can build out a complete attack cycle for the threat actor’s active cyber operation from delivery (when it first enters the defender’s cyber terrain) to command and control (when it exits the defender’s cyber terrain).

If we can build out the entire attack cycle from delivery through command and control we can then pivot on the entire set of observables directly associated with the threat actor’s use of the external cyber terrain and operationally configured TTP to determine if we have seen attacks by this threat actor before and if the attack is part of a larger campaign.

If we can build out the entire attack cycle from delivery through command and control then we can develop observable indicators for each phase of the attack to enable mitigations and countermeasures to be developed to stop the attack earlier in the attack cycle as it enters and moves through the defender’s cyber terrain.

Procedures

The procedure phase of the scientific method in this example covered the specific procedure that needed to be carried out during the experiment. In this case procedures for the threat intelligence analyst to analyze, produce, and disseminate the daily threat intelligence report as well as the procedures to be followed for planning and implementing the intelligence driven courses of action. The follow on actions for investigations and incident response were not included since the organization already had robust procedures in place for these activities and the primary focus this experiment was on the discovery and mitigate of unknown attacks by unknown threat actors.

The scientific method process here is aligned to the production and dissemination phases of the intelligence cycle, the decide and act phases of the OODA loop, the plan and do phases of the PDCA cycle, the planning and implementing courses of action in the intelligence feedback cycle.

Experiment

The 2009 experiment was approved for 6 months. Experiments need to run long enough to prove or disprove the hypothesis. The procedures developed for the experiment are carried out during the time frame of the experiment.

Data Analysis – Results of Experiment (Did observations validate predictions?)

At the end of the experiment we enter the data analysis phase of the scientific process to determine if our hypothesis predictions were correct or incorrect based on the real world observations of the experiment. This phase of the scientific process is aligned to the measuring phase of the intelligence feedback cycle and the check phase of the PDCA cycle.

Conclusion

The 2009 experiment I lead and used in this example validated the predictions of the hypothesis. We were able to validate there was unknown attacks taking place against other unknown organizations based on submissions of malware samples to online malware analysis sites. We were able to validate that we could discover TTPs with little (< 10%) or no antivirus detection and through analysis generate a daily threat intelligence report. We were able to validate that we could plan and implement courses of action to mitigate the threat based on the threat intelligence reports. We were able to validate that we got inside the threat actors attack cycle and ahead of threat when data analysis revealed 72% of the total number of detections at the command and control phase of the attack cycle were attributed to the indicators produced as part of the 6 month experiment. (The remaining 28% of detections at this stage of the attack cycle were attributed to commercially available block lists / black lists the customer was paying for and threat intelligence shared by partners.) We were also able to validate the attack cycle intelligence gain for each of the detections through building out the complete attack cycle and follow on analysis as part of the incident response process.

We recognized in the conclusion that if the defending organization was the only target or was the first X number of victims to be targeted by the threat actor that this effort would have little to no impact since there wasn’t an opportunity to get ahead of the threat actor’s attack cycle.

We also recognized and recommended that organizations could benefit from establishing more robust threat information sharing to allow greater opportunities to get inside the threat actor’s attack cycles and to increase the discovery of unknown threat actors and campaigns.

The conclusion phase of the scientific method is where we present the results of our experiment to include lessons learned, recommendations for follow up experiments, new standard operating procedures, and possible technology based solutions development. This step of the scientific method closely aligns to the feedback part of the intelligence feedback cycle and the act part of the PDCA cycle. An example of this is providing feedback on the information quality of the threat intelligence reports produced during the experiment. Was the threat intelligence Accurate? Timely? Usable? Complete? Precise? Reliable? Relevant? Predictive? Or Tailored? This helps push quality improvement and justification for continued investment.

I hope by sharing this example of applying the scientific method to a hard problem in real-world cyber operations that it helps others understand the role of cyber security scientists and how they might fit within your organization.