Building a Product Security Practice in a DevOps World

There is much emphasis today on development and delivery. The narrative largely focuses on the speed of software delivery through automated pipelines. Unfortunately, little attention has been paid to product security. However, product security is gaining popularity slowly.

One example is the recent introduction of CWE Hardware by MITRE.

With the integration of product and software lifecycle workflows, managing security across both domains is becoming an essential capability. This capability applies to many industry verticals, including manufacturing (e.g., electronics, semiconductors, industrial manufacturing), IT software, and others. The key aspect is being able to integrate software and product life cycles in a coherent manner so that the final product release is secure. In this whitepaper, we will discuss key product security capabilities and a holistic governance framework to bring these two domains together without slowing down the business.

What differentiates product security from software security?

It is important to understand the key differences between software and product security. While both domains are trying to be secure, their constraints are different:

Building key product security capabilities

The figure below is a snapshot of the different security capabilities that enterprises involved in product development should plan to establish and practice in order to accomplish a better security risk posture throughout the lifecycle of the product. The services of the capabilities established must be leveraged in a phased manner right from the inception stage. To balance between the traditional and DevOps methodology, we have identified four phases for the product development lifecycle:

  • Requirements and Planning
  • Design and Development
  • Deployment and Integration
  • Operation and Monitoring.

Requirements & Planning

A. Quality management: One of the important elements that engineering teams must consider right from the initial phases is building a quality product. A quality product means everything, including higher performance, greater reliability, lower operating cost, and so on. Quality management systems are seen as a driving factor behind many of the verification and validation procedures that are carried out in the world of product security. There are many methodologies, including Lean and Six Sigma, which have the tools and techniques to help build quality products.

B. Safety and security requirements: Designing products, especially in the manufacturing sector, requires consideration for a lot of safety requirements as mandated by varied regulations and standards pertinent to the domains and geography of the consumer using the product. One example is IEC 61508, an international functional safety standard that provides a framework for safety lifecycle activities. The standard covers safety-related systems which incorporate electrical/electronic/programmable electronic devices. Many of the failures in hardware and embedded devices may threaten human life safety, and care must be taken by considering the hazard identification and damage limitation steps. This is in addition to considering security requirements that must be adhered to secure the complete product. A good reference to start with would be the ISA 62443 standard that specifies security capabilities for control system standards.

C. Security training plays a key role and must be done to all stakeholders at every stage of product development. Based on the role, domain, and other aspects, the right training given at the right time will enable teams to build a secure product. Customized training must be rolled out on a mandatory basis to the teams involved, and security professionals must play the role of evangelists, not just gatekeepers, to sign off and perform assessments.

Design & Development

A. Threat model: The importance of Secure Design and Threat Modeling are well known to the professionals building software products. These activities, in fact, carry more weight in the world of product security. Some of the hardware security attributes, like encryption methods and obfuscation techniques, are critical secure design techniques implemented in devices and integrated circuits. There are many commercial Electronic Design Automation (EDA) tools that assist in the design and verification of secure hardware. Architects and security professionals must assess all the scenarios to evaluate the threats in the overall solution for which the product is designed.

B. IAAA protocols: There is a plethora of devices, services, and third-party components, that are integrated for product security. On top of that, many defensive layer security measures have to be planned, and not all of them are in-built, coded, or configured. Organizations use many vendor solutions, secure libraries, and certified tools for establishing Identity, Authentication, Authorization, and Accountability (IAAA Solutions).

C. Secure coding: There are many teams at every layer that convert the security requirements captured into an actionable program by using the respective languages at those layers. For example, the use of VHDL (VHSIC Hardware Description Language) by the hardware teams while specifying the structure of a digital circuit, when using embedded C language for programming microcontroller-based applications, or the use of Java when writing a software application. It is important that secure coding measures are incorporated during those times. This is a good place of reference for secure coding in embedded systems.

Deployment & Integration

A. Secure communications: We have called out this as a separate capability since product development is all about integrating many pieces together, and it is essential to secure the communication of the complete stack. There are many communication channels: device to device, device to machine, machine to machine, within the cloud, on-prem, etc. It is not going to be within the accountability of one team to take care of the end-to-end security. Hence, this requires careful planning by the admins of the Operations or SecOps team.

B. Secure inventory management: Having an inventory of authorized and unauthorized devices is an important security control in an environment operating with a lot of devices. In order to have an effective defense on the network, we need to have an inventory of all the devices that are used. Going by the access control principles, only authorized devices must be allowed access, and unauthorized, unmanaged, and unsupported devices must not be given access inside the network. The information will help further in collecting all the relevant attributes for the devices in the environment.

C. Software composition analysis: Business units that manage a large portfolio of products certainly need SCA tools. There is a lot of open-source code being leveraged by software developers as they focus on rapid time to market. It is estimated that FOSS (Free and Open source) tools make up 60 to 80 percent of an application code base, and managing risk has become a top priority. SCA tools not only help manage the inventory of such open-source tools but also help in mitigating security vulnerabilities.

D. Configuration audits: This is an important activity to perform for products that have a variety of devices built into it. Configuration audits, in general, validate the integrity of product configuration information. There are two critical parameters that should be of interest to product development teams: the Functional Configuration Audit (FCA) and the Physical Configuration Audit (PCA). FCA examines the functional characteristics of the configured product and verifies that the product has met the requirements specified, whereas PCA examines the actual configuration of an item being produced.

E. Privacy audits: Though privacy assessments should be an implicit part of the security activities that are performed throughout the lifecycle of the product development, there is a lot of weightage given to privacy assessments these days by auditors and regulators. In the last few years, the industry has seen many data breaches impacting privacy, especially in the manufacturing industry. These privacy checks will vary based on the domain, geographical presence, and compliance. The International Association of Privacy Professionals owns a lot of resources with regard to privacy and has some good guidelines for professionals on privacy assessments.

Operate & Monitor

A. PSIRT practice: The first half of the last decade saw many security attacks on the transport layer security, BEAST, FREAK, POODLE, and a lot more. Every product team would have mandatorily addressed the security in transit during the pre-production stages, but attackers have found many ways to bypass those security controls during the production environment. SSL-based issues are just an example. Issues like these happen at every layer of the product. This is an ongoing threat, and enterprises must set up a Product Security Incident Response Team (PSIRT) that gets to know about the potential vulnerability in the products that could expose organizations and consumers to security risks. The PSIRT can then coordinate with the engineering team to identify an appropriate course of action.

B. Continuous monitoring: There are many products designed to safeguard human lives that are deployed at coal mines, hospitals, and construction sites. It is important to continuously monitor these systems for any data loss or malware intrusions that disturb the integrity and availability of the system. Regular software security solutions might not help in these circumstances. We need solutions that are designed to accommodate the protocols used in the related domains. For example, a regular firewall used for IT applications may be ineffective in protecting SCADA solutions or automotive products running CAN protocols. The security operations team running DLP and SOC practice must have the required resources and specialists to address these issues.

C. Penetration Testing: This is the last step in the Product Security process but can be done periodically or when there is a need for it. It must be carried out by internal teams and also by trusted and certified vendors. Also, pen testing is a specialized process in product development scenarios. It is about crafting attack vectors that can exploit any layer, say the JTAG interfaces at the hardware level, the domain-specific communication protocols like MODBUS or CAN, the vulnerabilities at the edge layer, or the software operating in the cloud. There are special tools and certified professionals to perform these.

D. Certification and Accreditation: Certification is the technical evaluation of a system or product and its security components. Accreditation is management’s formal approval and acceptance of the security provided by a system. When an enterprise plans to launch a product, there is a clear mandate from the government to follow security standards and achieve the required certifications. Some of the common certifications that product vendors go through involve the common criteria that assess the effectiveness of the security controls built into a system from functional and assurance perspectives or the FIPS 140-2, which is the cryptographic validation standard. There are many service providers that help with the complete certification lifecycle.8

E. Risk Assessment: Risk Assessment is the activity that conveys to the business the technical impact of security in a language that business professionals need. In addition to identifying assets, threats, and vulnerabilities, risk analysis focuses on the impact that the controls have on the cost and benefits to the overall organization. There are many methods that will help in the Risk Analysis process, but it is important to quantify the risk so that the business has insights into Exposure Factors, Annualized Rate of Occurrence, etc. From a product security perspective, the international standard ISO 10377:2013 (Consumer Product SafetyGuidelines for Suppliers) includes guidance for suppliers about product risk assessment and considers safety in design, production, and in the marketplace. This is a critical activity that must be performed for the overall risk management or for activities like Business Continuity Management.

Governance of product security

At a strategic level, governance implies the creation of competencies that are essential to delivering a secure product. Within that context, here are the key competencies:

At the operational level, we need to identify the roles, processes, and controls. Each role has a part to play in the overall value delivery process for software and hardware components. The security controls are there to ensure that risk against an acceptable threshold is being met. This can be, for example, in the form of compliance with software and/or hardware standards or frameworks.

Integrating product development and secure software development life cycles

Employing a governance process that enables your product development lifecycle to go in tandem with a secure software development lifecycle promotes cross-functional collaboration, efficiency, and speed in bringing secure products to market. This entails leveraging the strengths of the secure development lifecycle framework and tools — particularly in managing requirements, secure development guidelines, test cases, and test results, and integrating this with established product innovation and launch frameworks, such as the one shown below.

A secure development process overlaid at various stages of product development ensures security is built from the ground up in all aspects of the product — hardware, software, and firmware — from initial design through to testing, deployment, and ongoing monitoring. Tight integration between these processes becomes increasingly important as most hardware products also include an element of software and/or firmware to manage the device. Hardware products are also increasingly launched with additional features and functionality updates so that they can be more fluid and flexible in nature even after they have been deployed in the field.

Source: Wiley Online Library


Product security is becoming more important as we see the convergence of software and hardware lifecycles. The challenge is trying to integrate the processes and tools in a way that is not disruptive to business speed while remaining within risk thresholds for security and compliance. This implies collaboration across multiple stakeholders, with each stakeholder contributing to the balance between speed and risk.