Software Bills of Materials (SBOMs) have become an increasingly important part of software supply chain security, providing organisations with greater visibility into the components that make up their software.
Over recent years, significant progress has been made in SBOM generation, standardisation and adoption. Established formats such as SPDX and CycloneDX have helped create a common language for describing software components, their relationships and associated metadata.
Regulatory developments, including the European Union’s Cyber Resilience Act (CRA), are placing greater emphasis on software component transparency and vulnerability management. As organisations prepare to meet these requirements, the ability to identify, document and maintain reliable information about the components within their software is becoming increasingly important.
However, as organisations move beyond generating SBOMs and begin relying on them for open source license compliance, vulnerability management and regulatory compliance, an important question emerges:
Can we trust this SBOM?
An SBOM may be technically valid and conform to an established standard, yet still contain incomplete, inaccurate or inconsistent information. Understanding this distinction between validity and reliability is essential if SBOMs are to deliver meaningful value.
Standards Compliance Is Not the Same as SBOM Quality
SPDX and CycloneDX provide structured formats for representing software components and associated information. Their specifications establish rules for how information should be expressed, enabling interoperability between tools, organisations and systems.
Validation against these specifications is an important first step. It helps establish whether an SBOM follows the expected structure and meets the applicable requirements of the chosen format.
However, a successfully validated SBOM does not necessarily provide a complete or accurate representation of the software it describes.
For example, an SBOM may conform to its chosen specification while containing:
- Missing components or dependencies that were not identified during generation
- Unknown or incorrectly reported component versions
- Missing, incomplete or inaccurate license information
- Inconsistent component names and identifiers
- Insufficient information about relationships between components
- Outdated information that no longer reflects the software being distributed
These issues may not prevent an SBOM from passing format validation, but they can significantly reduce its usefulness. When organisations need to respond urgently to a newly disclosed vulnerability or to address a legal claim relating to open source license compliance, the last thing they need is to discover that their SBOM contains incomplete or inaccurate information.
This highlights an important distinction between validating the structure of an SBOM and assessing the quality of the information it contains.
The Challenge of Knowing What’s Actually in Your Software
One of the fundamental challenges in producing a reliable SBOM is accurately identifying the components that make up the software.
Software Composition Analysis (SCA) plays an important role in identifying open source and other third-party components and supporting SBOM generation. Different analysis techniques can provide complementary insights into the software being analysed, helping organisations gain greater visibility into its contents.
For example, dependency analysis can identify components declared through package manager manifests, lockfiles and other dependency information. While this provides valuable visibility into known dependencies, it may not identify third-party code that has been copied, modified or incorporated directly into the source code outside the normal dependency management process.
Source code analysis provides additional visibility by examining the code itself to identify open source components that may not be captured through dependency analysis alone. File-level matching can identify files originating from known open source projects, while snippet-level matching enables the identification of smaller portions of reused code, including code that has been copied, modified or incorporated into proprietary source files. Source code analysis can also support the extraction of license information from source files, providing additional context for assessing open source license compliance. These techniques help uncover otherwise overlooked third-party code and provide valuable information about its potential origins, supporting more accurate component identification.
In our previous article, What You Don’t Know Can Hurt You: Open Source Compliance Surprises from Real Audits, we explored some of the unexpected findings encountered during real-world open source audits, including overlooked components, reused code snippets and conflicting license information.
These examples demonstrate why understanding the actual contents of software can be more challenging than simply examining its declared dependencies. They also highlight the importance of using complementary identification techniques to improve confidence in the resulting component information.
A comprehensive SCA platform should combine dependency analysis with source code analysis, including file-level and snippet-level matching, to provide greater visibility into both declared and undeclared open source software. By drawing on extensive open source intelligence and granular code matching capabilities, such platforms identify components and reused code that might otherwise be overlooked, supporting more comprehensive and reliable SBOMs.
The implications extend directly to SBOM quality. Components that are not identified during analysis would naturally be omitted from the resulting SBOM, rendering it incomplete even when the document conforms to the required format.
The quality of an SBOM ultimately depends, in part, on the quality of the underlying component identification.
What Does SBOM Quality Actually Mean?
SBOM quality cannot be determined by a single validation result. It requires consideration of several characteristics that collectively influence whether an SBOM can be relied upon for its intended purpose.
Completeness
Does the SBOM adequately identify the software components, dependencies and associated information within its declared scope?
An SBOM that omits components, dependencies or important metadata provides an incomplete picture of the software supply chain, potentially leaving security and compliance risks unidentified.
Completeness should be assessed against the intended scope of the SBOM. For example, an SBOM generated from source code may include development dependencies that are not present in the final distributed software. Differences in component coverage do not necessarily indicate missing information, provided the SBOM accurately represents the software and scope it is intended to describe.
Accuracy
Does the information recorded in the SBOM correctly describe the components and their associated metadata?
Incorrect component versions, licenses, identifiers or relationships can result in inaccurate vulnerability assessments, misleading compliance results and unnecessary remediation activities.
Accuracy also requires distinguishing between information reported by an SBOM producer and information that has been independently verified. Where possible, component information should be assessed against reliable intelligence sources or supporting evidence to establish confidence in its correctness.
Consistency
Is component information represented consistently within an SBOM and across related SBOMs?
Inconsistent component names, identifiers, versions or relationships can make it difficult to correlate components with external intelligence sources or combine information from multiple SBOMs.
This becomes particularly important when organisations receive SBOMs from different suppliers or consolidate multiple SBOMs into a product-level view. Identifying and resolving conflicting information helps maintain a consistent representation of the software supply chain.
Traceability
Can the information within an SBOM be traced to the software artefact it represents and the evidence supporting its contents?
Understanding the origin of an SBOM, how and when it was generated, and which software version or artefact it describes is important when evaluating its reliability.
Maintaining provenance and supporting evidence also helps organisations distinguish between information supplied by the SBOM producer and information subsequently verified or enriched, without losing the integrity of the original SBOM.
Currency
Does the SBOM accurately reflect the software version or release it is intended to represent?
Software components and dependencies change throughout the development and maintenance lifecycle. An SBOM generated during development may no longer accurately describe the final distributed software if components have subsequently been added, removed or updated.
Maintaining reliable SBOM information requires organisations to identify relevant changes, associate SBOMs with the correct software versions and ensure that updated information is available when needed.
Together, these characteristics provide a more meaningful foundation for evaluating SBOM quality than format validation alone.
The Challenge of Unknown and Incomplete Information
One of the most significant challenges in SBOM quality is how unknown or incomplete information is represented and interpreted.
Both SPDX and CycloneDX provide mechanisms for expressing certain types of missing or unavailable information, although their approaches differ.
In SPDX, for example, values such as NOASSERTION can be used in supported fields to indicate that no assertion is being made about a particular piece of information.
This can be entirely appropriate. Not every piece of information will always be available, and representing uncertainty explicitly is preferable to introducing inaccurate assumptions.
However, the presence of unknown information raises an important question:
At what point does an SBOM become too incomplete to support its intended use?
Consider an SBOM containing hundreds of components, but with version information missing for a significant proportion of them. The SBOM may still be structurally valid, but identifying which components are affected by a newly disclosed vulnerability could become considerably more difficult.
Similarly, missing license information may limit the ability to perform reliable open source license compliance assessments. The issue is not necessarily that unknown information exists. It is whether organisations understand the implications of those unknowns and have appropriate processes to address them.
An SBOM containing explicitly identified gaps may, in some circumstances, be more useful than one that presents inaccurate information with unwarranted confidence.
Trust does not require pretending that every detail is known. It requires understanding what is known, what remains uncertain and how those uncertainties affect decisions.
Why SBOM Quality Matters Beyond Generation
For many organisations, generating an SBOM has historically been treated as the primary objective. However, producing an SBOM is only one stage in making software supply chain information useful.
As SBOM adoption increases, organisations are beginning to consume SBOMs from internal development teams, suppliers and third-party software providers. These SBOMs may originate from different tools, use different standards or versions, and contain varying levels of detail.
A single software product may also be represented by multiple SBOM documents, describing different components, subsystems or stages of the software lifecycle. This creates additional challenges.
- How should an organisation assess the quality of an SBOM received from a supplier?
- What happens when two SBOMs describe the same component differently?
- How can organisations identify missing information before using an SBOM to make security or compliance decisions? And where multiple SBOMs contribute to a consolidated view of a product, how can organisations ensure that inconsistencies or gaps are not simply carried forward?
These questions demonstrate why SBOM quality needs to be considered throughout the software supply chain, not simply at the point of generation.
Moving Towards More Trustworthy SBOMs
Improving SBOM quality requires organisations to look beyond technical format validation and establish processes for assessing and maintaining the underlying information.
Several practical considerations can help.
1. Establish Clear Quality Requirements
Define what information an SBOM must contain based on its intended purpose, declared scope and applicable requirements.
An SBOM used for vulnerability management may require reliable component versions and identifiers, while license compliance activities may place additional emphasis on accurate licensing information.
Regulatory requirements provide another important consideration. Under the EU Cyber Resilience Act (CRA), manufacturers of products with digital elements are required to identify and document vulnerabilities and software components, including through an SBOM in a commonly used, machine-readable format covering at least top-level dependencies.
However, meeting minimum documentation requirements should not be confused with having sufficient information to support effective vulnerability management. Organisations may need greater component visibility and more detailed information depending on the software, its associated risks and the decisions the SBOM is intended to support
2. Assess Completeness and Accuracy
Evaluate whether the SBOM adequately represents the software within its declared scope and whether the recorded component information is sufficiently complete and accurate.
This involves assessing component coverage, identifiers, versions, relationships and other relevant metadata, as well as identifying missing or inconsistent information.
Where possible, verify component information against reliable intelligence sources, software artefacts and supporting evidence. Different analysis techniques can provide complementary insights, helping identify gaps that may not be apparent from a single source of information.
3. Identify and Manage Unknown Information
Treat missing or uncertain information as something to be understood and assessed rather than simply accepted.
Not every unknown requires immediate remediation, but organisations should be able to determine whether missing information affects their ability to make informed decisions.
Quality assessments should distinguish between information that is unavailable, information that has not been assessed and information that may be inaccurate.
Where appropriate, identified gaps should be addressed through further verification, controlled enrichment or engagement with the original SBOM producer.
4. Preserve Provenance and Supporting Evidence
Maintain information about how and when an SBOM was generated, the software version or artefact it represents and the sources used to establish component information.
Where appropriate, retain supporting evidence and relationships to associated artefacts, including verification results and relevant attestations.
When additional information is verified or enriched, organisations should preserve the original SBOM and maintain traceability between the original information, subsequent assessments and any resulting changes.
This helps consumers understand the origin and reliability of the information provided.
5. Incorporate Quality Assessment into the SBOM Lifecycle
SBOM quality should not be treated as a one-time activity.
As software evolves, new SBOMs are received and additional intelligence becomes available, organisations need mechanisms to reassess the accuracy, completeness and continued relevance of their software component information.
Where multiple SBOMs contribute to a product-level view, organisations should also consider how component information is reconciled, how conflicting information is resolved and how relationships between the original and consolidated SBOMs are maintained.
This requires thinking beyond SBOM generation towards processes for validation, verification, enrichment, consolidation, distribution and ongoing monitoring.
From Validity to Trust
The growing adoption of SBOMs represents an important step towards improving software supply chain transparency.
Standards such as SPDX and CycloneDX have provided the foundations needed to exchange software component information in consistent, machine-readable formats. However, standardisation alone cannot guarantee the quality of the information being exchanged.
As organisations increasingly rely on SBOMs to support open source license compliance, vulnerability management and regulatory compliance, the focus must extend beyond whether an SBOM can be successfully generated and validated.
Organisations must also consider how the underlying information was identified, whether it accurately represents the software within its declared scope, how missing or uncertain information is managed and how its reliability can be maintained throughout the software lifecycle.
This requires a shift in focus from simply generating SBOMs towards assessing, verifying and maintaining the quality of the information they contain.
Ultimately, the objective should not simply be to produce more SBOMs, but to ensure that the information they provide is sufficiently complete, accurate and reliable to support informed decisions.
A valid SBOM is a starting point. A trustworthy SBOM requires confidence in the quality of the information behind it.
