Skip to content

Support for CycloneDX schema version 1.4 #108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 59 commits into from
Jan 13, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
aabb3a1
added draft 1.4 schema documents as of 10-Nov-2021
madpah Nov 10, 2021
a1c3401
skeleton updates to define Schema Version 1.4
madpah Nov 10, 2021
27d1c4f
update to latest draft 1.4 XML XSD
madpah Dec 21, 2021
6d28d98
Merge branch 'main' into feat/67-schema-version-1.4-xml
madpah Dec 21, 2021
c06b273
Merge branch 'main' into feat/67-schema-version-1.4-xml
madpah Dec 21, 2021
2a94a1e
- Added XML Schema validation for XML tests
madpah Dec 21, 2021
09a14a0
- Made Component `version` optional for schema version 1.4
madpah Dec 21, 2021
7e6db9a
- New model objects for `ReleaseNotes` and associated sub-types
madpah Dec 22, 2021
49a6738
added `lxml` as dev dependency
madpah Dec 22, 2021
f0403b7
- Added XML schema validation to all tests in `test_output_xml.py` wi…
madpah Dec 22, 2021
4be4004
added jsonschema validation to unit tests
madpah Dec 23, 2021
4a05fe0
fixed typos
madpah Dec 23, 2021
05e0dde
tests and completing new models for IssueType and ReleaseNote
madpah Dec 23, 2021
9f6ada8
support for `bom.metadata.tools.tool.externalReferences`
madpah Dec 23, 2021
827b01e
altered model to allow `version==""` as the agreed fallback
madpah Dec 23, 2021
5350272
added support for `bom.component.releaseNotes` in Schema Version >= 1…
madpah Dec 23, 2021
d7f3b19
feat: re-worked for vulnerabilities becoming part of core schema (#109)
madpah Jan 5, 2022
d11c687
test: confirmed `bom.metadata.tools[].externalReferences` is populate…
madpah Jan 5, 2022
8862b51
test: added test to prove `Component.version` is now optional for 1.4…
madpah Jan 5, 2022
8a35309
updated to ensure support for `bom.component.releaseNotes` in JSON 1.4
madpah Jan 5, 2022
7281a91
fixed a bunch of typing / flake8 issues
madpah Jan 5, 2022
505fa81
JSON now supporting `bom.vulnerabilities`
madpah Jan 6, 2022
1ef36a9
feat: move documentation to readthedocs.org (#120)
madpah Jan 6, 2022
b7f7a2d
feat: added $schema to JSON BOMs (#121)
madpah Jan 6, 2022
7e6771f
doc: updates
madpah Jan 6, 2022
802a101
doc: updates
madpah Jan 6, 2022
518eca4
chore: attempt to produce manual GitHub action to release a RC version
madpah Jan 7, 2022
b664b6e
chore: attempt to produce manual GitHub action to release a RC version
madpah Jan 7, 2022
938f346
chore: attempt to produce manual GitHub action to release a RC version
madpah Jan 7, 2022
b703ce9
chore: attempt to produce manual GitHub action to release a RC version
madpah Jan 7, 2022
fe1470f
chore: attempt to produce manual GitHub action to release a RC version
madpah Jan 7, 2022
0a18734
chore: attempt to produce manual GitHub action to release a RC version
madpah Jan 7, 2022
d1aa272
chore: add publish to PyPi for manual pre-release pacakge
madpah Jan 7, 2022
c1b0065
chore: add build prior to publish to PyPi for manual pre-release pacakge
madpah Jan 7, 2022
f8e02b8
Merge branch 'main' into feat/schema-version-1.4
madpah Jan 7, 2022
a182d7f
fix: using index when potentially not available
madpah Jan 7, 2022
e701478
chore: missing license header
madpah Jan 7, 2022
1ff3ea9
Merge branch 'feat/schema-version-1.4' of github.com:CycloneDX/cyclon…
madpah Jan 7, 2022
eae6331
addressed some PR comments from @jkowalleck
madpah Jan 7, 2022
ec798d5
addressed PR comments from @sonatype-lift
madpah Jan 7, 2022
c7fb3cd
addressed PR comments from @sonatype-lift
madpah Jan 7, 2022
aef955b
addressed PR comments from @sonatype-lift
madpah Jan 7, 2022
4f5ccaf
addressed PR comments from @sonatype-lift
madpah Jan 7, 2022
b8d7615
doc: cleaned up README in favour of RTD.
madpah Jan 7, 2022
e21a079
doc: tweaked link to RTD
madpah Jan 7, 2022
f3ad678
removed capturing groups from regex as not required
madpah Jan 9, 2022
71c36d4
remove notion that purl will be used for Component bom-ref if bom-ref…
madpah Jan 9, 2022
3cb7745
feat: Relocate Parsers to `cyclonedx-python` (#132)
madpah Jan 10, 2022
cd05f79
doc: added changelog
madpah Jan 10, 2022
2ed766c
doc: moved Python-specific parser docs to `cyclonedx-python`
madpah Jan 10, 2022
285c872
doc: removed python-specific parser docs
madpah Jan 10, 2022
0dac5bf
chore: relocated issue-related models to separate file
madpah Jan 10, 2022
1c04448
doc: removed parser reference
madpah Jan 10, 2022
f95fd3f
updated Component.__repr__
madpah Jan 10, 2022
d0bff8c
fix: addressed how we compare Component objects after feedback;
madpah Jan 10, 2022
9be349c
chore: grabbed latest FINAL 1.4 schema changes
madpah Jan 10, 2022
5990436
cleanup
madpah Jan 10, 2022
eca9cc9
addressed `Component.purl` / PackageURL feedback from @jkowalleck
madpah Jan 10, 2022
f01be8a
doc: describe anaconda release process (#111)
jkowalleck Jan 12, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 0 additions & 34 deletions .github/workflows/docs.yml

This file was deleted.

30 changes: 13 additions & 17 deletions .github/workflows/manual-release-candidate.yml
Original file line number Diff line number Diff line change
@@ -1,22 +1,17 @@
name: Manual Release Candidate
name: Manual Pre Release Publish

on:
workflow_dispatch:
inputs:
branch:
description: 'Branch to produce Release Candidate from'
release_candidate_suffix:
description: 'RC Suffix e.g. rc0, beta1, alpha2. Do not include a leading hyphen.'
required: true
type: string
rc_number:
description: 'RC Number'
default: 0
required: true
type: number

jobs:
release:
release_candidate:
runs-on: ubuntu-latest
concurrency: release
concurrency: release_candidate
steps:
- name: Checkout code
uses: actions/checkout@v2
Expand All @@ -31,12 +26,13 @@ jobs:
poetry config virtualenvs.create false
poetry install
python -m pip install python-semantic-release
- name: Determine RC candidate version
- name: Apply Pre Release Version
run: |
RC_VERSION="$(python-semantic-release --noop --major print-version)rc${{ inputs.rc_number }})"
RC_VERSION="$(semantic-release --noop --major print-version)-${{ github.event.inputs.release_candidate_suffix }}"
echo "RC Version will be: ${RC_VERSION}"
# - name: Python Semantic Release
# uses: relekang/python-semantic-release@master
# with:
# github_token: ${{ secrets.GITHUB_TOKEN }}
# pypi_token: ${{ secrets.PYPI_TOKEN }}
poetry version ${RC_VERSION}
poetry build
- name: Publish Pre Release 📦 to PyPI
uses: pypa/gh-action-pypi-publish@master
with:
password: ${{ secrets.PYPI_TOKEN }}
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,6 @@ html/

# mypy caches
/.mypy_cache

# Exlude built docs
docs/_build
45 changes: 45 additions & 0 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# encoding: utf-8

# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0

# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Set the version of Python and other tools you might need
build:
os: ubuntu-20.04
tools:
python: "3.9"
# You can also specify other tool versions:
# nodejs: "16"
# rust: "1.55"
# golang: "1.17"

# Build documentation in the docs/ directory with Sphinx
sphinx:
configuration: docs/conf.py

# Formats
formats: all

# Optionally declare the Python requirements required to build your docs
python:
install:
- method: pip
path: .
- requirements: docs/requirements.txt
199 changes: 5 additions & 194 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

[![shield_gh-workflow-test]][link_gh-workflow-test]
[![shield_pypi-version]][link_pypi]
[![shield_conda-forge-version]][link_conda-forge]
[![shield_license]][license_file]
[![shield_website]][link_website]
[![shield_slack]][link_slack]
Expand All @@ -22,199 +23,7 @@ Additionally, you can use this module yourself in your application to programmat

CycloneDX is a lightweight BOM specification that is easily created, human-readable, and simple to parse.

## Installation

Install from pypi.org as you would any other Python module:

```shell
pip install cyclonedx-python-lib
```

## Architecture

This module break out into three key areas:

1. **Parser**: Use a parser that suits your needs to automatically gather information about your environment or
application
2. **Model**: Internal models used to unify data from different parsers
3. **Output**: Choose and configure an output which allows you to define output format as well as the CycloneDX schema
version

### Parsing

You can use one of the parsers to obtain information about your project or environment. Available parsers:

| Parser | Class / Import | Description |
| ------- | ------ | ------ |
| CondaListJsonParser | `from cyclonedx.parser.conda import CondaListJsonParser` | Parses input provided as a `str` that is output from `conda list --json` |
| CondaListExplicitParser | `from cyclonedx.parser.conda import CondaListExplicitParser` | Parses input provided as a `str` that is output from `conda list --explicit` or `conda list --explicit --md5` |
| Environment | `from cyclonedx.parser.environment import EnvironmentParser` | Looks at the packaged installed in your current Python environment. |
| PipEnvParser | `from cyclonedx.parser.pipenv import PipEnvParser` | Parses `Pipfile.lock` content passed in as a string. |
| PipEnvFileParser | `from cyclonedx.parser.pipenv import PipEnvFileParser` | Parses the `Pipfile.lock` file at the supplied path. |
| PoetryParser | `from cyclonedx.parser.poetry import PoetryParser` | Parses `poetry.lock` content passed in as a string. |
| PoetryFileParser | `from cyclonedx.parser.poetry import PoetryFileParser` | Parses the `poetry.lock` file at the supplied path. |
| RequirementsParser | `from cyclonedx.parser.requirements import RequirementsParser` | Parses a multiline string that you provide that conforms to the `requirements.txt` [PEP-508] standard. |
| RequirementsFileParser | `from cyclonedx.parser.requirements import RequirementsFileParser` | Parses a file that you provide the path to that conforms to the `requirements.txt` [PEP-508] standard. |

#### Example

```py
from cyclonedx.parser.environment import EnvironmentParser

parser = EnvironmentParser()
```

#### Notes on Requirements parsing

CycloneDX software bill-of-materials require pinned versions of requirements. If your `requirements.txt` does not have
pinned versions, warnings will be recorded and the dependencies without pinned versions will be excluded from the
generated CycloneDX. CycloneDX schemas (from version 1.0+) require a component to have a version when included in a
CycloneDX bill of materials (according to schema).

If you need to use a `requirements.txt` in your project that does not have pinned versions an acceptable workaround
might be to:

```shell
pip install -r requirements.txt
pip freeze > requirements-frozen.txt
```

You can then feed in the frozen requirements from `requirements-frozen.txt` _or_ use the `Environment` parser one you
have `pip install`ed your dependencies.

### Modelling

You can create a BOM Model from either a Parser instance or manually using the methods avaialbel directly on the `Bom` class.

The model also supports definition of vulnerabilities for output using the CycloneDX schema extension for
[Vulnerability Disclosures](https://cyclonedx.org/use-cases/#vulnerability-disclosure) as of version 0.3.0.

**Note:** Known vulnerabilities associated with Components can be sourced from various data sources, but this library
will not source them for you. Perhaps look at [Jake][jake] if you're interested in this.

#### Example from a Parser

```py
from cyclonedx.model.bom import Bom
from cyclonedx.parser.environment import EnvironmentParser

parser = EnvironmentParser()
bom = Bom.from_parser(parser=parser)
```

### Generating Output

Once you have an instance of a `Bom` you can produce output in either `JSON` or `XML` against any of the supporting CycloneDX schema versions as you require.

We provide two helper methods:

* Output to string (for you to do with as you require)
* Output directly to a filename you provide

#### Example as JSON

```py
from cyclonedx.output import get_instance, OutputFormat

outputter = get_instance(bom=bom, output_format=OutputFormat.JSON)
outputter.output_as_string()
```

#### Example as XML

```py
from cyclonedx.output import get_instance, SchemaVersion

outputter = get_instance(bom=bom, schema_version=SchemaVersion.V1_2)
outputter.output_to_file(filename='/tmp/sbom-v1.2.xml')
```

## Library API Documentation

The Library API Documentation is available online at [https://cyclonedx.github.io/cyclonedx-python-lib/](https://cyclonedx.github.io/cyclonedx-python-lib/).

## Schema Support

This library is a work in progress and complete support for all parts of the CycloneDX schema will come in future releases.

Here is a summary of the parts of the schema supported by this library:

_Note: We refer throughout using XPath, but the same is true for both XML and JSON output formats._

<table width="100%">
<thead>
<tr>
<th>XPath</th>
<th>Support v1.3</th>
<th>Support v1.2</th>
<th>Support v1.1</th>
<th>Support v1.0</th>
<th>Notes</th>
</tr>
</thead>
<tbody>
<tr>
<td><code>/bom</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>Y</td>
<td>
This is the root element and is supported with all it's defined attributes.
</td>
</tr>
<tr>
<td><code>/bom/metadata</code></td>
<td>Y</td><td>Y</td><td>N/A</td><td>N/A</td>
<td>
<code>timestamp</code> and <code>tools</code> are currently supported
</td>
</tr>
<tr>
<td><code>/bom/components</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>Y</td>
<td>&nbsp;</td>
</tr>
<tr>
<th colspan="6"><strong><code>/bom/components/component</code></strong></th>
</tr>
<tr>
<td><code>./author</code></td>
<td>Y</td><td>Y</td><td>N/A</td><td>N/A</td>
<td>&nbsp;</td>
</tr>
<tr>
<td><code>./name</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>Y</td>
<td>&nbsp;</td>
</tr>
<tr>
<td><code>./version</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>Y</td>
<td>&nbsp;</td>
</tr>
<tr>
<td><code>./purl</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>Y</td>
<td>&nbsp;</td>
</tr>
<tr>
<td><code>./externalReferences</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>N/A</td>
<td>Not all Parsers have this information. It will be populated where there is information available.</td>
</tr>
<tr>
<td><code>./hashes</code></td>
<td>Y</td><td>Y</td><td>Y</td><td>Y</td>
<td>
These are supported when programmatically creating a <code>Bom</code> - these will not currently be
automatically populated when using a <code>Parser</code>.
</td>
</tr>
</tbody>
</table>

### Notes on Schema Support

* N/A is where the CycloneDX standard does not include this
* If the table above does not refer to an element, it is not currently supported
View our documentation [here](https://cyclonedx-python-library.readthedocs.io/).

## Python Support

Expand Down Expand Up @@ -244,14 +53,16 @@ See the [LICENSE][license_file] file for the full license.
[contributing_file]: https://github.com/CycloneDX/cyclonedx-python-lib/blob/master/CONTRIBUTING.md

[shield_gh-workflow-test]: https://img.shields.io/github/workflow/status/CycloneDX/cyclonedx-python-lib/Python%20CI/main?logo=GitHub&logoColor=white "build"
[shield_pypi-version]: https://img.shields.io/pypi/v/cyclonedx-python-lib?logo=Python&logoColor=white&label=PyPI "PyPI"
[shield_pypi-version]: https://img.shields.io/pypi/v/cyclonedx-python-lib?logo=pypi&logoColor=white&label=PyPI "PyPI"
[shield_conda-forge-version]: https://img.shields.io/conda/vn/conda-forge/cyclonedx-python-lib?logo=anaconda&logoColor=white&label=conda-forge "conda-forge"
[shield_license]: https://img.shields.io/github/license/CycloneDX/cyclonedx-python-lib "license"
[shield_website]: https://img.shields.io/badge/https://-cyclonedx.org-blue.svg "homepage"
[shield_slack]: https://img.shields.io/badge/slack-join-blue?logo=Slack&logoColor=white "slack join"
[shield_groups]: https://img.shields.io/badge/discussion-groups.io-blue.svg "groups discussion"
[shield_twitter-follow]: https://img.shields.io/badge/Twitter-follow-blue?logo=Twitter&logoColor=white "twitter follow"
[link_gh-workflow-test]: https://github.com/CycloneDX/cyclonedx-python-lib/actions/workflows/poetry.yml?query=branch%3Amain
[link_pypi]: https://pypi.org/project/cyclonedx-python-lib/
[link_conda-forge]: https://anaconda.org/conda-forge/cyclonedx-python-lib
[link_website]: https://cyclonedx.org/
[link_slack]: https://cyclonedx.org/slack/invite
[link_discussion]: https://groups.io/g/CycloneDX
Expand Down
Loading