Threat Modeling 101

In today’s digital era, technology has become pivotal for businesses, governments, and individuals globally. It facilitates trillions of dollars in commerce, harbors intellectual property, and stores vast amounts of personal data. These digital assets are invaluable and, consequently, prime targets for well-funded adversaries.

To safeguard these assets, robust security engineering is not just an option—it’s a necessity. Security must be integrated into products from inception and maintained throughout their operational life. This proactive approach is essential in defending against the sophisticated underground economy that thrives on trading stolen credentials and hacking tools, akin to commodities in legitimate markets.

Adversaries today wield immense computing power, often commandeering botnets with millions of machines or exploiting cloud infrastructures. This vast computational arsenal enables them to launch extensive attacks, including password cracking and denial-of-service assaults.

Smartphones have become repositories of personal and professional data, making them attractive targets. Despite evolving permission models in mobile operating systems, users often overlook the implications of granting app permissions. For instance, seemingly innocuous apps like free flashlight applications requesting access to contacts or network connections should raise red flags.

App Stores, optimized for publishing large volumes of apps, often lack rigorous security vetting. The dynamic nature of modern apps, which update code without user intervention, further complicates security monitoring. This has led to instances where even terrorist organizations have managed to infiltrate App Stores with malicious applications.

Apps like the Google/Microsoft Authenticator are particularly appealing to attackers seeking to circumvent two-factor authentication. Moreover, the landscape of cyber warfare is evolving, with nation-states engaging in espionage and sabotage, as evidenced by incidents like the Office of Personnel Management hack and the malware attack on an Iranian nuclear facility.

As production system defenses strengthen, attackers are shifting tactics, targeting development teams instead. They exploit trusted repositories, such as npm, to distribute compromised packages, hoping to breach developer machines and leapfrog to production systems. Consider your internal systems, development environments, build pipelines, and CI/CD processes. These critical components are prime targets for attacks. Acknowledging this reality is essential—it’s a scary place out there.

Development teams primarily focus on delivering functionality to users. While this is commendable, we must also prioritize security. Thinking like an attacker allows us to identify potential weaknesses. Stepping outside our usual personas, we ask: How could an adversary abuse our software? What defenses can we implement?

Threat modeling shifts our mental focus. When done collaboratively, it keeps the entire team vigilant. We constantly consider the bad actors’ tactics and strategize how to thwart them.

To communicate effectively about threat modeling, let’s establish a common language. Understanding these terms ensures precision in our discussions.

Key Terms in Threat Modeling

Assets

An asset represents something of value within a system that we aim to protect. These assets can take various forms:

1. Data: Protecting sensitive information is crucial. Whether it’s personal data, intellectual property, or financial records, maintaining data integrity and confidentiality is essential.
2. Integrity of Data: Ensuring that data remains accurate and unaltered is paramount. For instance, altering flight control systems in airplanes could have catastrophic consequences.
3. Accountability: In financial transactions or critical operations, knowing who initiated an action is vital. Maintaining an audit trail ensures accountability.
4. System Availability: Denial-of-service attacks can disrupt services, affecting users and damaging a company’s reputation.
5. Physical Well-Being: Connected systems (like cars) impact people’s safety. Ensuring their security prevents potential harm.
6. Reputation: Trust is built on safeguarding data. A breach can severely damage a company’s reputation.

Vulnerabilities

A vulnerability is a weakness in a system that could be exploited. The CIA triad—Confidentiality, Integrity, and Availability—helps us assess vulnerabilities. Additionally, consider Auditability for financial systems.

1. Confidentiality: Protect sensitive information from unauthorized access.
2. Integrity: Ensure data remains accurate and unaltered.
3. Availability: Prevent disruptions to services.
4. Auditability: Maintain a clear audit trail for accountability.

Remember that vulnerabilities aren’t always in code; human factors matter too. For example, phishing attacks exploit human behavior.

Threats

A threat is an actor (not always human) that exploits vulnerabilities. They discover and exploit weaknesses, leading to consequences for the assets we protect.

1. Human Threats: These include hackers, insiders, or social engineers.
2. Non-Human Threats: Worms, AI systems, or automated attacks can also pose threats.

Mitigations

Mitigations are countermeasures that reduce the likelihood of threats being successfully exploited. While code-based defenses are common, other approaches are equally valuable:

1. Training:
   • Educating customer service representatives to recognize and respond to security threats.
   • Ensuring employees understand security best practices.
2. Key Rotation:
   • Regularly changing encryption keys, even if the cryptographic algorithm is robust.
   • Assumption: Keys may leak or be compromised, so limiting the window of vulnerability is essential.
3. Processes and Policies:
   • Implementing secure development practices, such as code reviews and secure coding guidelines.
   • Establishing incident response plans for rapid mitigation.

Mitigations don’t always need to be 100% effective. Even if they slow down attackers or make their job harder, they add value. Logging, for instance, isn’t a direct mitigation but aids in forensics, detection, and evidence collection during incidents.

Risk

Risk is the potential loss or damage when a vulnerability is exploited by a threat. Not all threats require immediate mitigation:

• Resource Constraints: Finite resources may lead to prioritization. Low-risk threats might not receive immediate attention.
• Balancing Act: While risk management involves making choices, prioritize thorough mitigation whenever possible.

Sometimes, due to business needs or resource constraints, we might temporarily deviate from strict security practices. In such cases, documenting security exceptions and obtaining appropriate approvals (e.g., from VPs) ensures transparency and informed decision-making. The business must understand the risks associated with these exceptions and allocate resources to address them promptly.

Assume Breach

Layered Defenses

The old model of “inside is safe, outside is dangerous” is flawed. Assume attackers are already inside. Secure internal services as rigorously as external ones. Authentication, encryption, and access controls matter everywhere. Relying solely on external services or platforms without considering their potential failures is risky. Assume that other layers might fail and design your system to be defensible even when those layers break down.

Defensive Programming

Defensive programming involves anticipating failures and handling them proactively. For example, when coding a method, don’t assume that input parameters are always well-formed. Add error checks, validate formats, and implement robust error-handling routines.

Security by Design

Waiting until the end of development to add security features is ineffective. Treat security requirements as functional requirements from the outset. Test security features thoroughly, ideally with automated unit tests or system integration tests.

Key components in threat modeling

Use Cases and Scenarios

Begin by defining the system’s scope and purpose. What are the primary use cases or scenarios it supports? Understanding this context helps shape your threat model.

Data Flow Diagrams (DFDs)

DFDs illustrate how data flows between different components within the system. Pay attention to trust boundaries—where data crosses from one security context to another. Identify entry points into the system—where external input interacts with internal components.

Assets Enumeration

Enumerate the assets you’re protecting. These could be data (e.g., user profiles, financial records), services, or infrastructure components. Knowing what’s valuable helps prioritize security efforts.

Threats and Mitigations

List potential threats that could exploit vulnerabilities in the system. For each threat, propose mitigations—countermeasures to reduce the risk. Sometimes, accepting a calculated risk is necessary, but document it clearly.

Security Assumptions:

Acknowledge any assumptions you’re making about external systems or components. For example, relying on a well-established authentication stack (like the Identity Teams) is a security assumption. Trusting their expertise is reasonable.

Remember, threat modeling is an ongoing process, and it helps build security into the design rather than adding it as an afterthought.

Visual representations used in threat modeling

1. Data Flow Diagrams (DFDs):

• DFDs are commonly used to illustrate system components, data flows, and trust boundaries.
• Trust boundaries are depicted as red dashed lines, separating different security contexts.
• System components (represented as circles, ovals, or boxes) communicate via arrows indicating data flows.
• DFDs provide a high-level visual overview of the system’s architecture.

2. UML Sequence Diagrams for Complex Interactions:

• While DFDs work well for most scenarios, some complex interactions don’t fit neatly into arrows between blobs.
• Consider processes like the SSL handshake, which involve intricate back-and-forth messaging.
• For such cases, UML sequence diagrams are optimized to represent detailed interactions step by step.

3. Lightweight Threat Modeling Notation:

• When collaborating with an engineering team during threat modeling, use a concise notation:
  • T: Represents a threat.
  • M: Indicates a mitigation (a countermeasure to reduce risk).
  • PM: Denotes a potential mitigation (still under consideration or research).

Methodologies

Let’s explore two common methodologies used in threat modeling: DREAD and STRIDE.

DREAD

• Damage Potential: Assess the potential impact of a threat. How severe would the consequences be if it were exploited?
• Reproducibility: Consider how easily an attacker could reproduce the threat scenario.
• Exploitability: Evaluate the likelihood of an attacker successfully exploiting the vulnerability.
• Affected Users: Identify who would be impacted (users, systems, or data).
• Discoverability: Determine how easily an attacker could discover the vulnerability.
• Scoring: Assign scores to each aspect and aggregate them to rank threats. However, this method can be time-consuming and may not always be necessary.

STRIDE

• Spoofing: Focus on impersonation threats. Could an attacker fake their identity?
• Tampering: Assess risks related to unauthorized modification of data or code.
• Repudiation: Consider scenarios where an actor denies their actions.
• Information Disclosure: Evaluate the risk of exposing sensitive information.
• Denial of Service: Identify vulnerabilities that could lead to service disruption.
• Elevation of Privilege: Look for ways an attacker might gain unauthorized access or privileges.
• Use in Threat Identification: STRIDE helps identify classes of threats, especially for those new to threat modeling.

Use of Threat Modeling Tools

• Tools like Threat Dragon or the Threat Modeling Tool can automatically generate threats based on your system diagram.
• While this initial auto-generation is helpful, it can become verbose for complex models.
• After the initial round, relying solely on auto-generated threats may not be practical.
• Many threats may be mitigated by standard security practices (e.g., using HTTPS correctly).
• Don’t treat auto-generation as a substitute for collaborative threat modeling sessions.
• Engage your engineering team in proper threat modeling discussions.
• Focus on understanding what you’re worried about and how attackers might exploit vulnerabilities.
• Capture the results collaboratively to ensure a comprehensive approach.

Practical threat modeling

Having the right tools and a motivated team is essential. Let’s break down the key components:

1. Essential Tools:
   • Whiteboard: Whether virtual or physical, having a space to brainstorm and visualize ideas is crucial.
   • Note-Capturing Tool: Use a tool to capture notes during the session. The goal is to generate actionable work items.
   • Diagramming Tool: Diagrams (such as Data Flow Diagrams) help illustrate system architecture, trust boundaries, and data flows.
   • Work Item Tracking Tool: Capture work items (e.g., mitigations, testing) in tools like Azure DevOps, Jira, or GitHub issues.
   • Whiteboard Photo Tools: Useful for preserving whiteboard content.
2. Time Allocation:
   • Allocate a significant block of time for threat modeling sessions.
   • The first hour often involves discussions about how the system works.
   • The second hour focuses on identifying threats and mitigations.
3. Collaboration and Spectrum:
   • Threat modeling is a design activity. Collaborate with your team—software engineers, product owners, testers, and security experts.
   • Product owners can make decisions on the spot, adjusting schedules or adding security functionality.
   • Invite a spectrum of expertise to ensure diverse perspectives.
4. Preparation and Whiteboarding:
   • Prep work is essential. Have diagrams ready before the session.
   • Ensure you have a whiteboard available during the threat modeling session.
   • Collaborative discussions and visualizations lead to effective threat models.

Additional tips for practical threat modeling

1. Scope and Scenarios:
   • Understand the scope of what you’re threat modeling. What part of the system are you focusing on?
   • Identify relevant scenarios: How could an attacker exploit vulnerabilities in this system?
2. Note-Taking and Collaboration:
   • During threat modeling sessions, designate someone as the note-taker to capture ideas, action items, and follow-ups.
   • Collaboration is essential. Consider dependencies on other teams and create work items accordingly.
3. Avoid Auto-Generated Threats:
   • Instead of auto-generating threats, think like an attacker. Role-play different attack scenarios.
   • Be proactive in identifying threats, even if they’re related to external systems or dependencies.
4. Iterate and Update:
   • Threat modeling isn’t a one-time activity. As your system evolves, revisit the threat model.
   • Significant changes (new functionality, architectural modifications) may require updates to the threat model.
5. Testing and Mitigations:
   • A threat model serves as a test plan. Each identified threat should have a corresponding test for mitigation.
   • Pen testers can benefit from a high-quality threat model to validate mitigations effectively.
6. Negative Testing:
   • Test not only the happy path but also negative cases. Ensure that alerts fire when they should.

Conclusion

1. Changed Perspective:
   • By thinking like an attacker, you gain insights into potential abuse scenarios and vulnerabilities.
   • Historical cases and vulnerability classes become part of your awareness.
2. Preventing Vulnerabilities:
   • Threat modeling identifies security risks early, allowing you to address them before shipping your product.
   • Avoiding incidents or breaches saves time, money, and reputation.
3. Surface Design Flaws Early:
   • Threat modeling surfaces design flaws related to security and development tasks.
   • Implementing mitigations and testing isn’t hard or expensive when done proactively.
4. Test Plan Seed:
   • A threat model serves as a test plan. Each threat should have a corresponding test for mitigation.
   • Pen testers benefit from a high-quality threat model to validate mitigations effectively.
5. Business Awareness:
   • Product owners become aware of threats and security considerations.