SD Elements
Research
Upcoming Events
Resources
What is threat modeling and why it is important?
Threat modeling represents a plethora of different practices to analyze a system from a security perspective. There are many implementations of this practice. Therefore, it may be useful to introduce the definition we have used as a reference while writing this guide.
Threat modeling is a practice of designing and assessing the potential threats to a system in scope, evaluating the eventual risks both from a technical perspective and from the business’s point of view, and identifying what can be done to make those risks acceptable.
The paper focuses on threat modeling from a general perspective without delving into a specific methodology. The considerations and recommendations collected here should, therefore, be applicable to most approaches.
The evolution of threat modeling
In the early days, threat modeling was much simpler and based on systems where threat vectors against the system were well-known. In such cases, creating threat modeling diagrams manually was easier – we had controlled access to the few systems that were available. But in today’s DevSecOps world, things look quite different.
We have highly distributed systems where the emphasis is largely on component aggregation rather than ground-up coding. Execution and control flow are not always predictable through the system. It means we require the expertise of scarce security experts and architects to use the threat model effectively. Another typical factor is represented by the accelerated solution development lifecycle, typical of the Agile methodologies that are so prevalent nowadays: they require even more to focus the energies of the teams on satisfying the functional requirements rather than anything else.
Therefore, aspects like security tend to be considered as a cost, to be best covered by automated tools and processes designed with the main intent of reducing impact on development.
Threat modeling faces the same issues. Therefore, it is crucial to improve the efficiency of this practice if we want it to be relevant within a DevSecOps world.
Phases of threat modeling
Each threat modeling methodology defines its own set of phases. From our point of view, it is possible to identify seven distinct phases that are more or less explicitly present in all the most successful implementations.
Phase 1: Information gathering
At the start of the threat modeling process, we are interested in the context of our threat model. Stakeholders in this phase include developers, product managers, business analysts, security engineers, and security testers. Through a brainstorming exercise, they determine the scope, features, and use cases.
Phase 2: Identification of appropriate threats
The next phase is typically time-boxed and further decomposes the architecture to identify critical assets. This implies that not all threats will be analyzed, just the most important ones based on the experience of individuals and the collective knowledge base. These assets are reviewed against threat identification categories. Attack scenarios are created through real-life examples based on the relevant business context. The severity of the threats is determined, most frequently adopting an approach based on impact and probability. At this phase, there is some assessment of the cost involved to determine whether the identified threats are relevant.
Phase 3: Mitigations of threats
In the third phase, mitigations are proposed against the threats identified previously. They are categorized and prioritized based on all threats. The goal is not to produce a comprehensive list of mitigations for each threat but to cover different types of security controls for achieving defense-in-depth. This will provide different choices for consideration. Each mitigation is associated with a threat to justify and contextualize it. The mitigations must be expressed with unambiguous actionable descriptions and clear, verifiable criteria.
Phase 4: Assess mitigations
In this phase, mitigations are assessed against business and technical risk based on prioritized trade-offs. Recommendations are assessed for impact (both positive and negative) and then defined in a proposed roadmap.
Phase 5: Communicate results of threat model
In this phase, the mitigations are presented to the relevant stakeholders. Trade-off decisions are made based on budget, resource capacity, and organizational risk factors. Mitigations should be enforceable.
Phase 6: Update the threat model
Once approval has been given to undertake the mitigations, the threat model is regularly revised to reflect the latest update to the system in terms of risk and design.
Phase 7: Share knowledge and learn continuously
Learnings are fed back to improve threat modeling. Lessons learned are shared with internal and external communities.
Anti-patterns of threat modeling process
We have seen some good implementations of threat modeling, but many more have been affected by problems. This has allowed the identification of some common anti-patterns.
- Forcing tools to do what they were not intended for. We need to know what characteristics to look for depending on the maturity of the overall solution. Experts use different features than novices.
- Trying to achieve perfection. Analysis tends to focus on completeness rather than knowing how much is “good enough.”
- Using cut and paste rather than thinking about the assumptions that went into a previous threat model.
- Adopting a rigid threat modeling process for all projects without discriminating on scope or relevance.
- Focusing on a diagram-based language. In the end, it is a tool; if it works for you, go for it. Otherwise, use something else. Infrastructure teams, for example, might find tables work better (a list of objects that can relate to properties).
- Think of threat modeling only as a technical activity and ignore risk, use/ misuse cases, and abuse cases.
- Thinking of a threat modeling diagram as the final goal. In fact, it is just a starting point.
- Neglecting a feedback loop to update the threat model. The model should be living and maintained.
- Neglecting to factor in business risk priorities during threat analysis.
- Inability to identify a consistent risk definition to allow for comparisons between different systems.
- Lack of diversity in POVs. By not involving a broad group of stakeholders, there is a reliance on personal bias and assumptions about components or libraries based on previous experience.
- Reliance only on a checklist approach rather than combining with appropriate analysis.
- Blind acceptance of unknown system components without taking the time to conduct an in-depth study of the gaps.
- Focusing on development practices and pitfalls instead of limiting the scope to design.
- Being fully dependent on abstract knowledge bases and knowledge bases. In fact, they do not provide business context and remain at an abstract level.
- Focus on the parts instead of the whole or vice versa.
- Making everything a high priority for fear of having mitigations dismissed as unnecessary.
- Blindly adopting best practice security guidance as mitigations without linking to threats
Gaps with traditional data flow diagrammatic threat modeling
We identified seven key areas where traditional data flow diagram-based threat modeling fails to effectively deliver for today’s DevSecOps world.
1. Speed of updating diagrams
Updating these diagrams is relatively slow since it is largely a manual task. Revisiting a diagram requires re-contextualizing the system, and key participants in the original threat modeling effort may already be on other projects.
2. Lack of consistent Threat Modeling process
We have a lack of consistency because there is no global standard on how to create the right threat modeling diagram. It is left up to individual teams based on what they think is important, and they bring this bias and insight into their discussions. Current standardization efforts are largely based on the diagram language representation rather than true semantic analysis around security.
3. Emphasis on the system rather than a holistic approach
Considering that Threat Modeling is predominantly carried out during the design phases, most of the time it is the Architect and/or Dev lead who will be heavily involved. Other domains like infrastructure, operations, and CI/CD are rarely considered. This leads to an incomplete vision of the system under assessment.
4. Lack of reusable models
There’s a lack of reusability as many teams consider threat modeling to be a one-time activity due to the nuances of a particular system. This is typically managed through the adoption of templates. While templates can help, they typically require all the knowledge for a specific scenario. A byproduct of templates is to theoretically facilitate knowledge retention, but trying to codify this knowledge leads to more complex models. This complexity makes achieving consistency harder, and therefore, it may lead to very different results based on who produces the Threat Model.
5. Focus on subsystems rather than the bigger picture
Many times, Threat Modeling results are not consistent because threat modeling diagramming is largely an art. Each team will produce something that looks different based on what they feel is important. This leads to creating Threat Models from a very narrow perspective of the overall system. As a result, the most important threats impacting the system may be lost. Therefore, the Threat Model leads to wasting a lot of effort in trying to mitigate threats of marginal importance.
6. Lack of measured impact
Many teams do not account for the value accrued from Threat Modeling activities. There is no comparison against other security activities or by not doing it at all. For example, there is no feedback loop on how to effectively prioritize threats against quantitative measurements.
7. Knowledge bases generate a large number of generic threats
Knowledge bases are intended to provide generic mitigation. In the interest of speed, teams sometimes execute this generic advice without taking into account the specifics of the solution and the business point of view, thereby not identifying threats relevant to the business.
Capturing risk
The reason for a threat model is to identify the security risks of a system. The focus of risk should be on the economics and based on meaningful data around frequency and impact. Unfortunately, people don’t do a good job of risk scoring today. We want to use observable quantities to drive objectivity.
Where is threat modeling headed?
Traditional threat modeling relies on security and architectural experts. This excludes a vast number of other stakeholders from the business and technical teams. We believe the next generation of threat modeling will be layered. It will allow both experts and non-experts-experts to contribute to the threat model with relevant functionality limited to their perspective. Threat modeling will account for experience, where non-experts on experts will be intentionally limited in scope, and experts will perform deeper, quantitative risk analysis. The threat model will derive insights developed from other projects and be able to address the full risk lifecycle (identification, mitigation, residual risk), not just at the design phase.
Platform integrations will be a key part of the threat modeling experience. Many useful repositories will be leveraged, such as CWE, CVE, and CAPEC. There will be integrations with other security tools such as SAST, DAST, and risk management. Information viewed through the platform will be relevant to a specific role and meet the context and needs of an organization as needed. The system under analysis will be provided in some codified form, and threats will be represented in a flexible, reusable manner.
Mitigations will be at the core of the threat modeling experience. Different tools in the pipeline will focus on different problems in order to paint a cohesive picture. Threat modeling needs to fit into an Agile workflow. There has to be a process that is flexible enough to address a cross-functional team (developers, project managers, and architects) with different approaches. This means there is no single way to do threat modeling. We need to allow for other approaches. It also implies the ability to go back and change previous analysis. As such, the threat model becomes a living document or model continually being updated based on multiple views on the truth based on each person’s needs.
A new threat modeling practice
The response to the problems identified with traditional Threat Modeling is twofold:
- Drive threat modeling use cases through security requirements. Usually, these are tied to common practices like OWASP Top 10 or from standards groups.
- Create a feedback loop from SecOps back into the Dev teams. This allows for real-time monitoring of anomalies.
Threat Modeling is an important activity, and we need to do it. We just have to evolve our thinking to account for the complexity of our systems today. We need to consider a higher level of abstraction rather than focusing on just lower-level details. Large and complex single template approaches intended to capture all the knowledge tend to be repurposed without due consideration of overall architecture and risk. These templates tend to increase in complexity over time. Instead, we need to create a set of reusable templates that are specialized and focused on a subset of architectural and risk constraints. Multiple templates can then be used to guide and direct DevSecOps teams to initiate basic use cases. Thereafter, the threat model evolves to become more specific to the solution. Finally, complementing this with SecOps creates the collaborative feedback loop that enables unseen threat vectors to be codified and quickly resolved to contain the vulnerabilities.
A maturity model
We recognize that achieving our view for a modern threat modeling practice requires improving over the current one in a number of ways.
This involves actions to increase the Quality of the produced Threat Model to make it more useful as a tool to evaluate the risk and identify relevant Mitigations to be implemented. But it also involves evolving Threat Modeling to be central for Security Risk Management by making it the main part of a LEAN process fully integrated with the various tools and processes adopted by the Organization.
We have identified various categories of tools and processes that are good candidates for being integrated with such Threat Modeling practice, and for each of them, we have identified four different maturity levels:
- The first level corresponds to the experience where there is no integration at all, and even Threat Modeling is performed without a specialized tool. Our reference experience for this maturity level is the whiteboard Threat Modeling practice. At this level, threat modeling is mostly something done by enthusiasts, with no standardization or control over the process.
- The second level sees the introduction of rudimentary Threat Modeling tools. There are various freely available out of there, and most of them are not integrated or extensible. The process tends to be a little more standardized, but it is still in its infancy. The Business point of view is not considered: Threat Modeling is merely done from a technical perspective. Still, this level evidently provides additional value over the fully manual experience.
- The third level starts to see some integration capability and the process. At this level, business is taken more into account, and there is some management of the process, mostly with the intent of standardizing the experience. There is some integration, but it is limited.
- The fourth and last level is obviously where the highest maturity is achieved. At this level, the focus is on full control over the process, Continuous Improvement, and strong integration with processes and tools in use by the Organization.
For each of those levels, we have identified some typical questions that may be asked to understand if the Organization is at that specific level, and typical answers we expect would be obtained. Those questions and answers should be considered indicative and can be integrated based on the specifics of the Organization.
Conclusion
Threat modeling has a lot of potential. We can see an increase in interest in this practice lately. But, threat modeling also has important gaps that need to be addressed. The risk is to get into a Peak of Inflated Expectations, which would be followed very soon by a Trough of Disillusionment.
We are seeing this already: Organizations that have embraced threat modeling a while back are starting to feel the push from business because the adopted process is seen as an expensive exercise with very limited outcomes. In some cases, it is causing projects to be blocked from being delivered to production for weeks or even months. As a result, organizations tend to see threat modeling as a bureaucratic task, a necessary evil with no practical value.
As it is, threat modeling is not empowering business but is blocking it.
The best implementations are already providing a lot of value to customers by helping to identify significant risks and important activities to be done to make those risks acceptable in a timely manner.
The problem is that traditional approaches are no longer scalable or accurate in an agile, cloud-based microservices world. We need to include multiple stakeholders in the process to gain from the diversity of experience offered by both business and technical people. We also need to evolve our approach from a tool perspective into a shared platform perspective where integration with other tools and datasets is possible.
Threat modeling must evolve to fulfill its potential. We need to understand that an integrated experience is essential to make threat modeling useful for the business. We need to make it an integral part of risk management and ensure that the organization implements a virtuous cycle to continuously improve the practice.
Threat modeling is too important to be left in the hands of individual threat modelers.
Next Steps
1. Determine your current state:
Assess prior threat models, understand the business objectives, and assess the level required for the practice. All these will help us determine the maturity level of your practice. Obtaining the maturity level will help us ask the right audit questions and obtain the required outcome.
2. Determine the right future state for you:
Select the right tools and integrations based on cost and feasibility with current tools and optimize effectiveness with current processes. Emphasis should be on optimizing the quality of threat modeling. You don’t need to implement the highest level of maturity. Focus on what makes the most sense.
3. Implement the roadmap:
Prioritize the activities to achieve quick wins and then gradually expand into elements for your future state. Look for additional opportunities to further increase and contextualize your threat modeling practice by onboarding other services surrounding the tools described in this document.