# NASA PDS DOI Service
[](https://doi.org/10.5281/zenodo.5764878) [](https://github.com/NASA-PDS/doi-service/actions/workflows/unstable-cicd.yaml) [](https://github.com/NASA-PDS/doi-service/actions/workflows/stable-cicd.yaml)
The Planetary Data System (PDS) Digital Object Identifier (DOI) Service provides tools for PDS operators to mint [DOI](https://www.doi.org/)s.
## Prerequisites
- Python 3.9 or above
- A login to the DOI Service Provider endpoint server (currently DataCite)
## User Documentation
Please visit the documentation at: https://nasa-pds.github.io/doi-service/
## Developers
[JPL Internal Wiki](https://wiki.jpl.nasa.gov/display/PDSEN/DOI+Service)
Get the code and work on a branch:
git clone ...
git checkout -b "#<issue number>"
Install a Python virtual environment, say in a `venv` directory:
python3 -m venv venv
source venv/bin/activate
Install the package and its dependencies for development into the virtual environment:
pip install --editable '.[dev]'
If you get an error like
src/types.h:36:2: error: You need a compatible libgit2 version (1.1.x)
then you're probably using [brew.sh](https://brew.sh)'s Python 3.10. Use their Python 3.9 instead.
Update your local configuration to access the DOI service provider's test server.
Create a file in the base directory of the project named `pds_doi_service.ini`; the following may be used as a template
[SERVICE]
# Should be set to DataCite (case-insensitive)
provider = datacite
[DATACITE]
# Select the appropriate URL endpoint for either a test or production deployment
url = https://api.test.datacite.org/dois
#url = https://api.datacite.org/dois
user = <contact [PDS Help Desk](https://pds.nasa.gov/?feedback=true)>
password = <contact [PDS Help Desk](https://pds.nasa.gov/?feedback=true)>
doi_prefix = 10.17189
validate_against_schema = True
[OSTI]
# This section is kept for posterity, but should be ignored as OSTI is no longer a supported endpoint
url = https://www.osti.gov/iad2test/api/records
#url = https://www.osti.gov/iad2/api/records
user = <contact [PDS Help Desk](https://pds.nasa.gov/?feedback=true)>
password = <contact [PDS Help Desk](https://pds.nasa.gov/?feedback=true)>
doi_prefix = 10.17189
validate_against_schema = True
[PDS4_DICTIONARY]
url = https://pds.nasa.gov/pds4/pds/v1/PDS4_PDS_JSON_1D00.JSON
pds_node_identifier = 0001_NASA_PDS_1.pds.Node.pds.name
[API_AUTHENTICATION]
# Add the issuer of the oauth tokens, for cognito https://cognito-idp.<aws-region>.amazonaws.com/<userpoolID>
jwt_issuer =
# Add the entire content of the JSON file at https://cognito-idp.<aws-region>.amazonaws.com/<userpoolID>/.well-known/jwks.json
json_web_key_set =
jwt_lifetime_seconds = 3600
jwt_algorithm = RS256
[OTHER]
logging_level = INFO
doi_publisher = NASA Planetary Data System
global_keyword_values = PDS,PDS4
pds_uri = https://pds.nasa.gov/pds4/pds/v1/
transaction_dir = ./transaction_history
db_file = doi.db
db_table = doi
api_host = 0.0.0.0
api_port = 8080
api_valid_referrers =
emailer_local_host = localhost
emailer_port = 25
emailer_sender = pdsen-doi-test@jpl.nasa.gov
emailer_receivers = pdsen-doi-test@jpl.nasa.gov
## Launch API server
To run the DOI API server, try:
```console
$ pip install pds-doi-service
$ pds-doi-api
```
The started service documentation is available on http://localhost:8080/PDS_APIs/pds_doi_api/0.2/ui/
ð **Note:** When the `api_valid_referrers` option is set in `pds_doi_service.ini`, this service documentation UI will be unavailable.
## Running with Docker
To run the server on a Docker container, please execute the following from the package directory:
```console
$ # building the image
$ docker image build --tag pds-doi-service .
$ # starting up a container
$ docker container run --publish 8080:8080 pds-doi-service
```
However, note that when launching the container via `docker container run`, all configuration values are derived from the default INI file bundled with the repository. To override the configuration, it is recommended to launch the service via a Docker Composition:
```console
$ # Make a copy of the docker composition environment template:
$ cp doi_service.env.in doi_service.env
$ # Edit the environment file, setting the credentials within:
$ vi doi_service.env
$ # Start the composition; on some systems, `docker compose` is `docker-compose`:
$ docker compose up
```
This will launch the DOI Service container using the top-level `docker-compose.yml` file, which specifies that environment variables be imported from `doi_service.env`. Modify `doi_service.env` (after copying it from `doi_service.env.in`) to define any configuration values to override when the service is launched.
## Test
Testing details are detailed in this section.
### Tox (for developers)
#### N.B. Updates to pip dependencies are not automatically applied to existing tox virtual environments, to keep unit testing fast. The simplest way to propagate dependency updates is to delete `./.tox` and run tox again.
[tox](https://tox.readthedocs.io/) is installed automatically during `pip install --editable .[dev]`, and provides virtual environments and run configurations for
- unit/functional testing
- linting
- building the rich documentation.
To launch the full set of tests, simply run:
tox
You can also run individual components:
```console
$ tox -e tests # Run unit, functional, and integration tests
$ tox -e lint # Run flake8, mypy, and black code reformatting
$ tox -e docs # Build the documentation to see if that works
```
It is strongly recommended to add `tox -e lint` to your `pre-commit` [git hook](https://www.atlassian.com/git/tutorials/git-hooks), and `tox -e tests` in a `pre-push` hook, as only linted and test-passing PRs will be merged.
The following linting example is provided for ease of use:
```bash
STAGED_FILES=$(git diff --cached --name-only --diff-filter=ACM | grep ".py$")
echo "Linting files"
tox -e lint
git add $STAGED_FILES # add any lint-related changes to the current commit
if [ $? -ne 0 ]
then
echo "Initial lint detected errors, re-linting to determine whether errors remain"
tox -e lint
if [ $? -ne 0 ]
then
exit 1
fi
fi
exit 0
```
You can also run `pytest`, `sphinx-build`, `mypy`, etc., if that's more your speed.
### ~~Behavioral testing (for Integration & Testing)~~
~~Behavioral tests are also pre-installed in the Python virtual environment when you run `pip install --editable .[dev]`. Launch those by running:~~
behave
~~Note this will download reference test data. If they need to be updated you have to first remove your local copy of the reference data (`test/aaDOI_production_submitted_labels`)~~
~~You can also run them for a nicer reporting:~~
behave -f allure_behave.formatter:AllureFormatter -o ./allure ./features
allure service allure
~~ð **Note:** This assumes you have [Allure Test Reporting](http://allure.qatools.ru/) framework installed.~~
#### Testrail Reporting
Test reports can be pushed to [Testrail](https://cae-testrail.jpl.nasa.gov/testrail/)
Project: Planetary Data System (PDS)
Test suite: pds-doi-service
Set your environment:
export TESTRAIL_USER=<your email in testrail>
export TESTRAIL_KEY=<your API key in tesrail>
Run the tests:
behave
See the results in https://cae-testrail.jpl.nasa.gov/testrail/index.php?/projects/overview/168
ð **Note:** This assumes you have access to the [Jet Propulsion Laboratory's Testrail installation](https://opencae.jpl.nasa.gov/portal/#/tool-detail/site__18_5_3_83a025f_1554392171681_999533_17603_cover).
## Documentation Management
Documentation about the documentation is described in this section.
### Design
See in this repository:
https://github.com/NASA-PDS/pds-doi-service/tree/main/docs
or the `docs` directory in the source package.
### User Documentation
User documentation is managed with Sphinx, which is also installed in your Python virtual environment when you run `pip install --editable .[dev]`. You can use `tox` as described above to make the docs, or by hand at any time by running:
sphinx-build -ab html docs/source docs/build
## Build & Release
The build and release process is managed by [GitHub Actions](https://github.com/features/actions) and the [Roundup](https://github.com/NASA-PDS/roundup-action).
Raw data
{
"_id": null,
"home_page": "https://github.com/NASA-PDS/pds-doi-service",
"name": "pds-doi-service",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "",
"keywords": "pds,doi,osti,dataCite",
"author": "PDS",
"author_email": "pds_operator@jpl.nasa.gov",
"download_url": "https://github.com/NASA-PDS/pds-doi-service/releases/",
"platform": null,
"description": "# NASA PDS DOI Service\n\n[](https://doi.org/10.5281/zenodo.5764878) [](https://github.com/NASA-PDS/doi-service/actions/workflows/unstable-cicd.yaml) [](https://github.com/NASA-PDS/doi-service/actions/workflows/stable-cicd.yaml)\n\nThe Planetary Data System (PDS) Digital Object Identifier (DOI) Service provides tools for PDS operators to mint [DOI](https://www.doi.org/)s.\n\n\n## Prerequisites\n\n- Python 3.9 or above\n- A login to the DOI Service Provider endpoint server (currently DataCite)\n\n\n## User Documentation\n\nPlease visit the documentation at: https://nasa-pds.github.io/doi-service/\n\n\n## Developers\n\n[JPL Internal Wiki](https://wiki.jpl.nasa.gov/display/PDSEN/DOI+Service)\n\nGet the code and work on a branch:\n\n git clone ...\n git checkout -b \"#<issue number>\"\n\nInstall a Python virtual environment, say in a `venv` directory:\n\n python3 -m venv venv\n source venv/bin/activate\n\nInstall the package and its dependencies for development into the virtual environment:\n\n pip install --editable '.[dev]'\n\nIf you get an error like\n\n src/types.h:36:2: error: You need a compatible libgit2 version (1.1.x)\n\nthen you're probably using [brew.sh](https://brew.sh)'s Python 3.10. Use their Python 3.9 instead.\n\nUpdate your local configuration to access the DOI service provider's test server.\n\nCreate a file in the base directory of the project named `pds_doi_service.ini`; the following may be used as a template\n\n [SERVICE]\n # Should be set to DataCite (case-insensitive)\n provider = datacite\n\n [DATACITE]\n # Select the appropriate URL endpoint for either a test or production deployment\n url = https://api.test.datacite.org/dois\n #url = https://api.datacite.org/dois\n user = <contact [PDS Help Desk](https://pds.nasa.gov/?feedback=true)>\n password = <contact [PDS Help Desk](https://pds.nasa.gov/?feedback=true)>\n doi_prefix = 10.17189\n validate_against_schema = True\n\n [OSTI]\n # This section is kept for posterity, but should be ignored as OSTI is no longer a supported endpoint\n url = https://www.osti.gov/iad2test/api/records\n #url = https://www.osti.gov/iad2/api/records\n user = <contact [PDS Help Desk](https://pds.nasa.gov/?feedback=true)>\n password = <contact [PDS Help Desk](https://pds.nasa.gov/?feedback=true)>\n doi_prefix = 10.17189\n validate_against_schema = True\n\n [PDS4_DICTIONARY]\n url = https://pds.nasa.gov/pds4/pds/v1/PDS4_PDS_JSON_1D00.JSON\n pds_node_identifier = 0001_NASA_PDS_1.pds.Node.pds.name\n\n [API_AUTHENTICATION]\n # Add the issuer of the oauth tokens, for cognito https://cognito-idp.<aws-region>.amazonaws.com/<userpoolID>\n jwt_issuer =\n # Add the entire content of the JSON file at https://cognito-idp.<aws-region>.amazonaws.com/<userpoolID>/.well-known/jwks.json\n json_web_key_set =\n jwt_lifetime_seconds = 3600\n jwt_algorithm = RS256\n\n [OTHER]\n logging_level = INFO\n doi_publisher = NASA Planetary Data System\n global_keyword_values = PDS,PDS4\n pds_uri = https://pds.nasa.gov/pds4/pds/v1/\n transaction_dir = ./transaction_history\n db_file = doi.db\n db_table = doi\n api_host = 0.0.0.0\n api_port = 8080\n api_valid_referrers =\n emailer_local_host = localhost\n emailer_port = 25\n emailer_sender = pdsen-doi-test@jpl.nasa.gov\n emailer_receivers = pdsen-doi-test@jpl.nasa.gov\n\n\n## Launch API server\n\nTo run the DOI API server, try:\n\n```console\n$ pip install pds-doi-service\n$ pds-doi-api\n```\n\nThe started service documentation is available on http://localhost:8080/PDS_APIs/pds_doi_api/0.2/ui/\n\n\ud83d\udc49 **Note:** When the `api_valid_referrers` option is set in `pds_doi_service.ini`, this service documentation UI will be unavailable.\n\n\n## Running with Docker\n\nTo run the server on a Docker container, please execute the following from the package directory:\n\n```console\n$ # building the image\n$ docker image build --tag pds-doi-service .\n$ # starting up a container\n$ docker container run --publish 8080:8080 pds-doi-service\n```\n\nHowever, note that when launching the container via `docker container run`, all configuration values are derived from the default INI file bundled with the repository. To override the configuration, it is recommended to launch the service via a Docker Composition:\n\n```console\n$ # Make a copy of the docker composition environment template:\n$ cp doi_service.env.in doi_service.env\n$ # Edit the environment file, setting the credentials within:\n$ vi doi_service.env\n$ # Start the composition; on some systems, `docker compose` is `docker-compose`:\n$ docker compose up\n```\n\nThis will launch the DOI Service container using the top-level `docker-compose.yml` file, which specifies that environment variables be imported from `doi_service.env`. Modify `doi_service.env` (after copying it from `doi_service.env.in`) to define any configuration values to override when the service is launched.\n\n## Test\n\nTesting details are detailed in this section.\n\n\n### Tox (for developers)\n\n#### N.B. Updates to pip dependencies are not automatically applied to existing tox virtual environments, to keep unit testing fast. The simplest way to propagate dependency updates is to delete `./.tox` and run tox again.\n\n[tox](https://tox.readthedocs.io/) is installed automatically during `pip install --editable .[dev]`, and provides virtual environments and run configurations for\n- unit/functional testing\n- linting\n- building the rich documentation.\n\nTo launch the full set of tests, simply run:\n\n tox\n\nYou can also run individual components:\n\n```console\n$ tox -e tests # Run unit, functional, and integration tests\n$ tox -e lint # Run flake8, mypy, and black code reformatting\n$ tox -e docs # Build the documentation to see if that works\n```\n\nIt is strongly recommended to add `tox -e lint` to your `pre-commit` [git hook](https://www.atlassian.com/git/tutorials/git-hooks), and `tox -e tests` in a `pre-push` hook, as only linted and test-passing PRs will be merged.\n\nThe following linting example is provided for ease of use:\n\n```bash\nSTAGED_FILES=$(git diff --cached --name-only --diff-filter=ACM | grep \".py$\")\n\necho \"Linting files\"\ntox -e lint\ngit add $STAGED_FILES # add any lint-related changes to the current commit\n\nif [ $? -ne 0 ]\nthen\n echo \"Initial lint detected errors, re-linting to determine whether errors remain\"\n tox -e lint\n if [ $? -ne 0 ]\n then\n exit 1\n fi\nfi\n\nexit 0\n```\n\nYou can also run `pytest`, `sphinx-build`, `mypy`, etc., if that's more your speed.\n\n\n### ~~Behavioral testing (for Integration & Testing)~~\n\n~~Behavioral tests are also pre-installed in the Python virtual environment when you run `pip install --editable .[dev]`. Launch those by running:~~\n\n behave\n\n~~Note this will download reference test data. If they need to be updated you have to first remove your local copy of the reference data (`test/aaDOI_production_submitted_labels`)~~\n\n~~You can also run them for a nicer reporting:~~\n\n behave -f allure_behave.formatter:AllureFormatter -o ./allure ./features\n allure service allure\n\n~~\ud83d\udc49 **Note:** This assumes you have [Allure Test Reporting](http://allure.qatools.ru/) framework installed.~~\n\n\n#### Testrail Reporting\n\nTest reports can be pushed to [Testrail](https://cae-testrail.jpl.nasa.gov/testrail/)\n\nProject: Planetary Data System (PDS)\nTest suite: pds-doi-service\n\nSet your environment:\n\n export TESTRAIL_USER=<your email in testrail>\n export TESTRAIL_KEY=<your API key in tesrail>\n\nRun the tests:\n\n behave\n\nSee the results in https://cae-testrail.jpl.nasa.gov/testrail/index.php?/projects/overview/168\n\n\ud83d\udc49 **Note:** This assumes you have access to the [Jet Propulsion Laboratory's Testrail installation](https://opencae.jpl.nasa.gov/portal/#/tool-detail/site__18_5_3_83a025f_1554392171681_999533_17603_cover).\n\n\n## Documentation Management\n\nDocumentation about the documentation is described in this section.\n\n\n### Design\n\nSee in this repository:\n\n https://github.com/NASA-PDS/pds-doi-service/tree/main/docs\n\nor the `docs` directory in the source package.\n\n\n### User Documentation\n\nUser documentation is managed with Sphinx, which is also installed in your Python virtual environment when you run `pip install --editable .[dev]`. You can use `tox` as described above to make the docs, or by hand at any time by running:\n\n sphinx-build -ab html docs/source docs/build\n\n\n## Build & Release\n\nThe build and release process is managed by [GitHub Actions](https://github.com/features/actions) and the [Roundup](https://github.com/NASA-PDS/roundup-action).\n",
"bugtrack_url": null,
"license": "apache-2.0",
"summary": "Digital Object Identifier service for the Planetary Data System",
"version": "2.4.0",
"project_urls": {
"Download": "https://github.com/NASA-PDS/pds-doi-service/releases/",
"Homepage": "https://github.com/NASA-PDS/pds-doi-service"
},
"split_keywords": [
"pds",
"doi",
"osti",
"datacite"
],
"urls": [
{
"comment_text": "\ud83e\udd20 Yee-haw! This here ar-tee-fact got done uploaded by the Roundup!",
"digests": {
"blake2b_256": "80753ba3a7224c6763b2ec0d397f3d962f617aeb643ff96f95c036ef5cb8d40b",
"md5": "ac473cb89cde56ec0ae38ae88477852d",
"sha256": "f18efe4fd37f177f33d7a6fb42c1f66a6e16f2d50c7f4bae89f844e2d9bd5ad5"
},
"downloads": -1,
"filename": "pds_doi_service-2.4.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ac473cb89cde56ec0ae38ae88477852d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 354244,
"upload_time": "2023-10-16T21:47:29",
"upload_time_iso_8601": "2023-10-16T21:47:29.344888Z",
"url": "https://files.pythonhosted.org/packages/80/75/3ba3a7224c6763b2ec0d397f3d962f617aeb643ff96f95c036ef5cb8d40b/pds_doi_service-2.4.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-16 21:47:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "NASA-PDS",
"github_project": "pds-doi-service",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "pds-doi-service"
}
JSON
