<!--
# SPDX-FileCopyrightText: (c) 2018-2024 Siemens
# SPDX-License-Identifier: MIT
-->
# CaPyCli - Clearing Automation Python Command Line Tool for SW360
[](https://github.com/sw360/capycli/blob/main/License.md)
[](https://pypi.org/project/capycli/)
[](https://www.python.org/doc/versions/)
[](https://github.com/sw360/capycli/actions/workflows/static-checks.yml)
[](https://github.com/sw360/capycli/actions/workflows/unit-tests.yml)
[](https://github.com/sw360/capycli/actions/workflows/unit-tests.yml)
[](https://github.com/tngraf/Tethys.Dgml/blob/master/SBOM/sbom.cyclonedx.xml)
[](https://api.reuse.software/info/git.fsfe.org/reuse/api)
Python 3 scripts to allow license clearing automation using the
[SW360](https://github.com/eclipse/sw360) software catalogue.
## What is SW360?
[SW360](https://github.com/eclipse/sw360) is a software component catalogue application designed to
provide a central place for sharing information about software components used by an organization.
It is designed to neatly integrate into existing infrastructures related to the management of
software artifacts and projects by providing separate backend services for distinct tasks and a set
of portlets to access these services. A complete deployment unit exists (vagrant box or docker
container) that contains a complete configuration of all services and portlets.
Companies like Cariad, Siemens or Toshiba use SW360 to track their use of third party software components.
## Why CaPyCli?
SW360 is for software developers and software developers love to automate tasks. The SW360 user
interface is nice if you want to check a project or search for a single component. But if you have
a project with a JavaScript frontend and hundreds of components, you do not want to add all of them
manually. You want to be able to determine your software bill of materials (SBOM) and you want to
map this SBOM to the information that is already available of SW360.
CaPyCli allows you to
* determine your list of dependencies, your software bill of materials (SBOM)
* determine meta-data for the SBOM items and download source files
* map an SBOM to the data available on SW360
* create all missing components and releases
* create a project that contains all releases of your SBOM
* track the progress on license compliance checks
* show information about the project and its releases
* show information about export control information and security vulnerabilities (if tracked via SW360)
## Basic Syntax
```code
CaPyCli command [sub-command...] [options]
Commands and Sub-Commands
getdependencies dependency detection specific commands
Nuget determine dependencies for a .Net/Nuget project
Python determine dependencies for a Python project
Javascript determine dependencies for a JavaScript project
MavenPom determine dependencies for a Java/Maven project using the pom.xml file
MavenList determine dependencies for a Java/Maven project using a Maven command
bom bill of material (SBOM) specific commands
Show display contents of a SBOM
Convert convert SBOM formats
Filter apply filter file to a SBOM
Check check that all releases in the SBOM exist on target SW360 instance
CheckItemStatus show additional information about SBOM items on SW360
Map map a given SBOM to data on SW360
CreateReleases create new releases for existing components on SW360
CreateComponents create new components and releases on SW360 (use with care!)
DownloadSources download source files from the URL specified in the SBOM
Granularity check a bill of material for potential component granularity issues
Diff compare two bills of material.
Merge merge two bills of material.
Findsources determine the source code for SBOM items.
mapping
ToHtml create a HTML page showing the mapping result
ToXlsx create an Excel sheet showing the mapping result
moverview
ToHtml create a HTML page showing the mapping result overview
ToXlsx create an Excel sheet showing the mapping result overview
project
Find find a project by name
Prerequisites checks whether all prerequisites for a successfull
software clearing are fulfilled
Show show project details
Licenses show licenses of all cleared compponents
Create create or update a project on SW360
Update update an exiting project, preserving linked releases
GetLicenseInfo get license info of all project components
CreateReadme create a Readme_OSS
Vulnerabilities show security vulnerabilities of a project
ECC Show export control status of a project
Options:
command command and subcommand to process
-h, --help show a help message and exit
-i INPUTFILE, --inputfile INPUTFILE input file to read from
-ri RAW_INPUT, --raw-input RAW_INPUT raw data input file to parse repository urls
-o OUTPUTFILE, --outputfile OUTPUTFILE output file to write to
-filterfile FILTERFILE filter file to use
-v VERBOSE be verbose
-t SW360_TOKEN, --token SW360_TOKEN use this token for access to SW360
-oa, --oauth2 this is an oauth2 token
-url SW360_URL use this URL for access to SW360
--nocache NOCACHE do not use component cache
-cf CACHEFILE, --cachefile CACHEFILE cache file name to use
-rc REFRESH_CACHE, --refresh_cache REFRESH_CACHE refresh component cache
-sc, --similar look for components with similar name
-ov CREATE_OVERVIEW, --overview CREATE_OVERVIEW create an mapping overview JSON file
-mr WRITE_MAPRESULT, --mapresult WRITE_MAPRESULT create a JSON file with the mapping details
-name name of the project
-version version of the project
-id ID SW360 id of the project, supersedes name and
version parameters
-ncli NCLI, --no-overwrite-cli NCLI do not overwrite existing CLI files
-nconf NCONF, --no-overwrite-config NCONF do not overwrite an existing configuration file
-dest DESTINATION, --destination DESTINATION the destination folder
-source SOURCE source folder or additional source file
--dbx DBX relaxed handling of debian version numbers
--download enable automatic download of missing sources
--search-meta-data SEARCH_META_DATA search for component meta-data
-old-version OLD_VERSION previous version
-ex show exit code
-rr RESULT_REQUIRED there must be a clearing result available
-xml XML use XML format
-package-source PACKAGE_SOURCE URL of the package manager to use
-all show/use all items
-format FORMAT format to use (text, json, xml)
-fe FORCE_EXIT, --forceexit FORCE_EXIT force a specific exit code
-m MODE, --mode MODE specific mode for some commands
-if INPUTFORMAT Specify input file format
-of OUTPUTFORMAT Specify output file format
-X DEBUG Enable debug output
--forceerror FORCE_ERROR force an error exit code in case of visual errors
```
## Use Cases
Over the time we implemented more and more commands with more and more parameters.
We understand that it is hard for beginners to find the right command for the task
they want to do. Have a look at our [Use Case Overview](UseCaseOverview.md).
## Software Clearing Approaches
From time to time there are questions **why** a command has been implemented in this
specific way or why a command exists at all. Not all organization have the same
approach when doing license compliance. Have a look at our
[Software Clearing Approach Overview](SoftwareClearingApproachOverview.md) to see our
approaches.
## Note about Python Dependency Detection
At the moment there is only support for dependencies defined in a `requirements.txt` file.
Poetry users can create the `requirements.txt` file via
```sh
poetry export --format requirements.txt -o requirements.txt --without-hashes
```
If you are using pipenv, you can create the `requirements.txt` file via
```sh
pipenv lock -r > requirements.txt
```
If your dependencies are defined in `setup.py` you may take a look at
https://dephell.readthedocs.io/cmd-deps-convert.html or
https://github.com/jazzband/pip-tools#example-usage-for-pip-compile to generate
a `requirements.txt` file.
Probably the best solution is if you enhance CaPyCli to support poetry, pipenv or setup.py
directly and open a merge request.
## Examples
### Find project by name
Command:
```sh
capycli project find -name "tr-card"
- or -
python -m capycli project find -name tr-card
```
Result
```sh
CaPyCli - Find a project by name
Searching for projects by name
TR-Card, 1.0 => ID = ff697cd18fe178b26fc601b60e00fcdf
```
More examples and usage notes can be found in [examples.md](examples.md).
## Prerequisites
* Python 3
* A SW360 read (and write) token, see next section.
## API Access
Access to the SW360 REST API requires an access token.
The token can be requested on SW360/Preferences/REST API Token.
The scripts in this repository expect, that a valid token
is stored in the environment variable ``SW360ProductionToken``.
Alternatively you can specify a token using the `-t` option.
For proper access to an SW360 instance the correct url must be own.
The SW360 url can be specified on the commandline with the `-url`
parameter, via the environment variable ``SW360ServerUrl`` or in the
config file (`.capycli.cfg`).
## SBOM Format
The software bill of materials (SBOM) is a crucial information for most operations.
There is no common description what a bill of materials should contain.
There are different formats available, for example the SBOM of CyCloneDX,
nevertheless most tools have their own SBOM format.
We have decided also to have our own flavor of CycloneDX, see [SBOM](Readme_BOM.md),
focused on the information we need to handle components, releases and projects
on SW360. It is a simple JSON format. CaPyCli reads or writes exactly the
information that is needed.
Conversion support from or to our SBOM format is available.
For converting CycloneDX (XML) to JSON or for converting SPDX SBOMs, we like
to refer you to the oepn source tools from [CycloneDX](https://cyclonedx.org/).
## Mapping a SBOM to SW360
SBOM mapping is described in an extra file, see [SBOM Mapping](Readme_Mapping.md).
## Project Management
This is a Python project managed using ```Poetry```.
## Installation
### From PyPi
* using `pip`:
```shell
pip install capycli
```
## Copyright & License
Copyright 2018-2024 Siemens
This program and the accompanying materials are made
available under the terms of the MIT License.
SPDX-License-Identifier: MIT
Raw data
{
"_id": null,
"home_page": "https://github.com/sw360/capycli",
"name": "capycli",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": "sw360, cli, automation, license, compliance, clearing",
"author": "Thomas Graf",
"author_email": "thomas.graf@siemens.com",
"download_url": "https://files.pythonhosted.org/packages/6c/a2/c62fa4321546577fea77ee8463719a9e1ecbe995239407732ca437ff6750/capycli-2.4.0.tar.gz",
"platform": null,
"description": "<!--\n# SPDX-FileCopyrightText: (c) 2018-2024 Siemens\n# SPDX-License-Identifier: MIT\n-->\n\n\n\n# CaPyCli - Clearing Automation Python Command Line Tool for SW360\n\n[](https://github.com/sw360/capycli/blob/main/License.md)\n[](https://pypi.org/project/capycli/)\n[](https://www.python.org/doc/versions/)\n[](https://github.com/sw360/capycli/actions/workflows/static-checks.yml)\n[](https://github.com/sw360/capycli/actions/workflows/unit-tests.yml)\n[](https://github.com/sw360/capycli/actions/workflows/unit-tests.yml)\n[](https://github.com/tngraf/Tethys.Dgml/blob/master/SBOM/sbom.cyclonedx.xml)\n[](https://api.reuse.software/info/git.fsfe.org/reuse/api)\n\nPython 3 scripts to allow license clearing automation using the\n[SW360](https://github.com/eclipse/sw360) software catalogue.\n\n## What is SW360?\n\n[SW360](https://github.com/eclipse/sw360) is a software component catalogue application designed to\nprovide a central place for sharing information about software components used by an organization.\nIt is designed to neatly integrate into existing infrastructures related to the management of\nsoftware artifacts and projects by providing separate backend services for distinct tasks and a set\nof portlets to access these services. A complete deployment unit exists (vagrant box or docker\ncontainer) that contains a complete configuration of all services and portlets.\n\nCompanies like Cariad, Siemens or Toshiba use SW360 to track their use of third party software components.\n\n## Why CaPyCli?\n\nSW360 is for software developers and software developers love to automate tasks. The SW360 user\ninterface is nice if you want to check a project or search for a single component. But if you have\na project with a JavaScript frontend and hundreds of components, you do not want to add all of them\nmanually. You want to be able to determine your software bill of materials (SBOM) and you want to\nmap this SBOM to the information that is already available of SW360.\n\nCaPyCli allows you to\n\n* determine your list of dependencies, your software bill of materials (SBOM)\n* determine meta-data for the SBOM items and download source files\n* map an SBOM to the data available on SW360\n* create all missing components and releases\n* create a project that contains all releases of your SBOM\n* track the progress on license compliance checks\n* show information about the project and its releases\n* show information about export control information and security vulnerabilities (if tracked via SW360)\n\n## Basic Syntax\n\n```code\nCaPyCli command [sub-command...] [options]\n\nCommands and Sub-Commands\n getdependencies dependency detection specific commands\n Nuget determine dependencies for a .Net/Nuget project\n Python determine dependencies for a Python project\n Javascript determine dependencies for a JavaScript project\n MavenPom determine dependencies for a Java/Maven project using the pom.xml file\n MavenList determine dependencies for a Java/Maven project using a Maven command\n\n bom bill of material (SBOM) specific commands\n Show display contents of a SBOM\n Convert convert SBOM formats\n Filter apply filter file to a SBOM\n Check check that all releases in the SBOM exist on target SW360 instance\n CheckItemStatus show additional information about SBOM items on SW360\n Map map a given SBOM to data on SW360\n CreateReleases create new releases for existing components on SW360\n CreateComponents create new components and releases on SW360 (use with care!)\n DownloadSources download source files from the URL specified in the SBOM\n Granularity check a bill of material for potential component granularity issues\n Diff compare two bills of material.\n Merge merge two bills of material.\n Findsources determine the source code for SBOM items.\n\n mapping\n ToHtml create a HTML page showing the mapping result\n ToXlsx create an Excel sheet showing the mapping result\n\n moverview\n ToHtml create a HTML page showing the mapping result overview\n ToXlsx create an Excel sheet showing the mapping result overview\n\n project\n Find find a project by name\n Prerequisites checks whether all prerequisites for a successfull\n software clearing are fulfilled\n Show show project details\n Licenses show licenses of all cleared compponents\n Create create or update a project on SW360\n Update update an exiting project, preserving linked releases\n GetLicenseInfo get license info of all project components\n CreateReadme create a Readme_OSS\n Vulnerabilities show security vulnerabilities of a project\n ECC Show export control status of a project\n\nOptions:\n command command and subcommand to process\n -h, --help show a help message and exit\n -i INPUTFILE, --inputfile INPUTFILE input file to read from\n -ri RAW_INPUT, --raw-input RAW_INPUT raw data input file to parse repository urls\n -o OUTPUTFILE, --outputfile OUTPUTFILE output file to write to\n -filterfile FILTERFILE filter file to use\n -v VERBOSE be verbose\n -t SW360_TOKEN, --token SW360_TOKEN use this token for access to SW360\n -oa, --oauth2 this is an oauth2 token\n -url SW360_URL use this URL for access to SW360\n --nocache NOCACHE do not use component cache\n -cf CACHEFILE, --cachefile CACHEFILE cache file name to use\n -rc REFRESH_CACHE, --refresh_cache REFRESH_CACHE refresh component cache\n -sc, --similar look for components with similar name\n -ov CREATE_OVERVIEW, --overview CREATE_OVERVIEW create an mapping overview JSON file\n -mr WRITE_MAPRESULT, --mapresult WRITE_MAPRESULT create a JSON file with the mapping details\n -name name of the project\n -version version of the project\n -id ID SW360 id of the project, supersedes name and \n version parameters\n -ncli NCLI, --no-overwrite-cli NCLI do not overwrite existing CLI files\n -nconf NCONF, --no-overwrite-config NCONF do not overwrite an existing configuration file\n -dest DESTINATION, --destination DESTINATION the destination folder\n -source SOURCE source folder or additional source file\n --dbx DBX relaxed handling of debian version numbers\n --download enable automatic download of missing sources\n --search-meta-data SEARCH_META_DATA search for component meta-data\n -old-version OLD_VERSION previous version\n -ex show exit code\n -rr RESULT_REQUIRED there must be a clearing result available\n -xml XML use XML format\n -package-source PACKAGE_SOURCE URL of the package manager to use\n -all show/use all items\n -format FORMAT format to use (text, json, xml)\n -fe FORCE_EXIT, --forceexit FORCE_EXIT force a specific exit code\n -m MODE, --mode MODE specific mode for some commands\n -if INPUTFORMAT Specify input file format\n -of OUTPUTFORMAT Specify output file format\n -X DEBUG Enable debug output\n --forceerror FORCE_ERROR force an error exit code in case of visual errors\n```\n\n## Use Cases\n\nOver the time we implemented more and more commands with more and more parameters. \nWe understand that it is hard for beginners to find the right command for the task\nthey want to do. Have a look at our [Use Case Overview](UseCaseOverview.md).\n\n## Software Clearing Approaches\n\nFrom time to time there are questions **why** a command has been implemented in this\nspecific way or why a command exists at all. Not all organization have the same\napproach when doing license compliance. Have a look at our\n[Software Clearing Approach Overview](SoftwareClearingApproachOverview.md) to see our\napproaches.\n\n## Note about Python Dependency Detection\n\nAt the moment there is only support for dependencies defined in a `requirements.txt` file. \nPoetry users can create the `requirements.txt` file via\n\n```sh\npoetry export --format requirements.txt -o requirements.txt --without-hashes\n```\n\nIf you are using pipenv, you can create the `requirements.txt` file via\n\n```sh\npipenv lock -r > requirements.txt\n```\n\nIf your dependencies are defined in `setup.py` you may take a look at\nhttps://dephell.readthedocs.io/cmd-deps-convert.html or\nhttps://github.com/jazzband/pip-tools#example-usage-for-pip-compile to generate\na `requirements.txt` file.\n\nProbably the best solution is if you enhance CaPyCli to support poetry, pipenv or setup.py\ndirectly and open a merge request.\n\n## Examples\n\n### Find project by name\n\nCommand:\n\n```sh\ncapycli project find -name \"tr-card\"\n- or -\npython -m capycli project find -name tr-card\n```\n\nResult\n\n```sh\nCaPyCli - Find a project by name\n\n Searching for projects by name\n TR-Card, 1.0 => ID = ff697cd18fe178b26fc601b60e00fcdf\n```\n\nMore examples and usage notes can be found in [examples.md](examples.md).\n\n## Prerequisites\n\n* Python 3\n* A SW360 read (and write) token, see next section.\n\n## API Access\n\nAccess to the SW360 REST API requires an access token.\nThe token can be requested on SW360/Preferences/REST API Token.\n\nThe scripts in this repository expect, that a valid token\nis stored in the environment variable ``SW360ProductionToken``.\nAlternatively you can specify a token using the `-t` option.\n\nFor proper access to an SW360 instance the correct url must be own.\nThe SW360 url can be specified on the commandline with the `-url`\nparameter, via the environment variable ``SW360ServerUrl`` or in the\nconfig file (`.capycli.cfg`).\n\n## SBOM Format\n\nThe software bill of materials (SBOM) is a crucial information for most operations.\nThere is no common description what a bill of materials should contain.\nThere are different formats available, for example the SBOM of CyCloneDX,\nnevertheless most tools have their own SBOM format.\nWe have decided also to have our own flavor of CycloneDX, see [SBOM](Readme_BOM.md),\nfocused on the information we need to handle components, releases and projects\non SW360. It is a simple JSON format. CaPyCli reads or writes exactly the\ninformation that is needed.\nConversion support from or to our SBOM format is available.\nFor converting CycloneDX (XML) to JSON or for converting SPDX SBOMs, we like\nto refer you to the oepn source tools from [CycloneDX](https://cyclonedx.org/).\n\n## Mapping a SBOM to SW360\n\nSBOM mapping is described in an extra file, see [SBOM Mapping](Readme_Mapping.md).\n\n## Project Management\n\nThis is a Python project managed using ```Poetry```.\n\n## Installation\n\n### From PyPi\n\n* using `pip`:\n\n ```shell\n pip install capycli\n ```\n\n## Copyright & License\n\nCopyright 2018-2024 Siemens\n\nThis program and the accompanying materials are made\navailable under the terms of the MIT License. \nSPDX-License-Identifier: MIT\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "CaPyCli - Clearing Automation Python Command Line Interface for SW360",
"version": "2.4.0",
"project_urls": {
"Homepage": "https://github.com/sw360/capycli",
"Repository": "https://github.com/sw360/capycli",
"issues": "https://github.com/sw360/capycli/issues"
},
"split_keywords": [
"sw360",
" cli",
" automation",
" license",
" compliance",
" clearing"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b7cd59a33eca4b31324d8151fd7b08e87c11c6fea22d289fbcbbdb397d8bc09f",
"md5": "3c911249e86c170d4a924181729b658e",
"sha256": "355cd2b0ff32fa0c83da01cd8d53ac75033675efcc0140586e0fba784e9aff76"
},
"downloads": -1,
"filename": "capycli-2.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3c911249e86c170d4a924181729b658e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 185140,
"upload_time": "2024-04-22T06:22:39",
"upload_time_iso_8601": "2024-04-22T06:22:39.521049Z",
"url": "https://files.pythonhosted.org/packages/b7/cd/59a33eca4b31324d8151fd7b08e87c11c6fea22d289fbcbbdb397d8bc09f/capycli-2.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6ca2c62fa4321546577fea77ee8463719a9e1ecbe995239407732ca437ff6750",
"md5": "f8ba935491a1244dc6326462715cd7d1",
"sha256": "50abded40d9b21472d6802cae535d36c851a28d6453ff35b9b1d5c7b029ee81a"
},
"downloads": -1,
"filename": "capycli-2.4.0.tar.gz",
"has_sig": false,
"md5_digest": "f8ba935491a1244dc6326462715cd7d1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8",
"size": 131606,
"upload_time": "2024-04-22T06:22:41",
"upload_time_iso_8601": "2024-04-22T06:22:41.687344Z",
"url": "https://files.pythonhosted.org/packages/6c/a2/c62fa4321546577fea77ee8463719a9e1ecbe995239407732ca437ff6750/capycli-2.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-22 06:22:41",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "sw360",
"github_project": "capycli",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"tox": true,
"lcname": "capycli"
}
JSON
