# Preservation Status Database Builder
Returns the preservation status of a Crossref DOI matched against mainstream digital preservation platforms.
  <a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
    
This application allows you to build a database of digital preservation sources and then to match a DOI against common digital preservation systems.
## Installation
The easiest install is via pip:
pip install preservation-database
Then add "preservationdatabase" (no hyphen) to your list of INSTALLED_APPS.
## Usage
export DJANGO_SETTINGS_MODULE=import_settings.settings
Usage: python -m preservationdatabase.cli [OPTIONS] COMMAND [ARGS]...
Options:
--help Show this message and exit.
Commands:
clear-cache Clear the import cache
import-all Download and import all data (excluding...
import-cariniana Download and import data from Cariniana
import-clockss Download and import data from CLOCKSS
import-hathi Import data from Hathi (requires local...
import-internet-archive Import data from Internet Archive...
import-internet-archive-items Import item data from Internet Archive
import-issnl Import ISSN-L mappings
import-lockss Download and import data from LOCKSS
import-ocul Import data from Ocul (requires local...
import-pkp Download and import data from PKP's...
import-portico Download and import data from Portico
random-samples Return random samples that occur in and...
show-archives Clear the import cache
show-cache Show last fill date/times and cache status
show-issn Show preservation items that match an ISSN
show-preservation Determine whether a DOI is preserved
stamp-cache-today Mark the latest imports as today
## Features
* Cariniana import.
* CLOCKSS import.
* HathiTrust import.
* Internet Archive import.
* Internet Archive item-level import.
* LOCKSS import.
* PKP PLN import.
* Portico import.
* Crossref DOI lookup.
## First-Run Setup
First, copy example_settings.py to settings.py and check settings.py to ensure that the database you want to use is set correctly. The default is db.sqlite. You should carefully read and check all of settings.py.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': BASE_DIR / 'db.sqlite3',
}
}
Next, run the database build commands:
python3 manage.py makemigrations
python3 manage.py makemigrations preservation-database
python3 manage.py migrate
You should then have a working database into which you can import new preservation data.
## Archive Notes
### Internet Archive
The Internet Archive gives a KBART file for the Keepers Registry that we use as a primary ingest source: https://archive.org/details/ia-keepers-registry-kbart. However, this source is not the total coverage of the Internet Archive. However, sadly, the Internet Archive snapshots do not contain external identifiers and the container-level snapshots do not present coverage extent. While it is possible to download the entire 217GB FATCAT database snapshot, this will not be viable for many users. We have therefore stuck with the KBART file that Keepers uses. Extent of coverage in the Internet Archive may, therefore, be under-reported.
# Credits
* [Django](https://www.djangoproject.com/) for the ORM.
* [Git](https://git-scm.com/) from Linus Torvalds _et al_.
* [.gitignore](https://github.com/github/gitignore) from Github.
* [Rich](https://github.com/Textualize/rich) from Textualize.
© Crossref 2023
Raw data
{
"_id": null,
"home_page": "https://gitlab.com/crossref/labs/preservationDatabase",
"name": "preservation-database",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "Martin Paul Eve <meve@crossref.org>",
"keywords": "digital preservation, academic",
"author": "Martin Paul Eve",
"author_email": "meve@crossref.org",
"download_url": "https://files.pythonhosted.org/packages/71/60/91715c4624be8b219a3bb43ee3e5d3dd727fa4d3e15506753b01cf2259fd/preservation_database-0.0.106.tar.gz",
"platform": null,
"description": "# Preservation Status Database Builder\nReturns the preservation status of a Crossref DOI matched against mainstream digital preservation platforms.\n\n  <a href=\"https://github.com/psf/black\"><img alt=\"Code style: black\" src=\"https://img.shields.io/badge/code%20style-black-000000.svg\"></a>\n\n    \n\nThis application allows you to build a database of digital preservation sources and then to match a DOI against common digital preservation systems.\n\n## Installation\nThe easiest install is via pip:\n \n pip install preservation-database\n\nThen add \"preservationdatabase\" (no hyphen) to your list of INSTALLED_APPS.\n\n## Usage\n\n export DJANGO_SETTINGS_MODULE=import_settings.settings\n\n Usage: python -m preservationdatabase.cli [OPTIONS] COMMAND [ARGS]...\n \n Options:\n --help Show this message and exit.\n \n Commands:\n clear-cache Clear the import cache\n import-all Download and import all data (excluding...\n import-cariniana Download and import data from Cariniana\n import-clockss Download and import data from CLOCKSS\n import-hathi Import data from Hathi (requires local...\n import-internet-archive Import data from Internet Archive...\n import-internet-archive-items Import item data from Internet Archive\n import-issnl Import ISSN-L mappings\n import-lockss Download and import data from LOCKSS\n import-ocul Import data from Ocul (requires local...\n import-pkp Download and import data from PKP's...\n import-portico Download and import data from Portico\n random-samples Return random samples that occur in and...\n show-archives Clear the import cache\n show-cache Show last fill date/times and cache status\n show-issn Show preservation items that match an ISSN\n show-preservation Determine whether a DOI is preserved\n stamp-cache-today Mark the latest imports as today\n\n## Features\n* Cariniana import.\n* CLOCKSS import.\n* HathiTrust import.\n* Internet Archive import.\n* Internet Archive item-level import.\n* LOCKSS import.\n* PKP PLN import.\n* Portico import.\n* Crossref DOI lookup.\n\n## First-Run Setup\nFirst, copy example_settings.py to settings.py and check settings.py to ensure that the database you want to use is set correctly. The default is db.sqlite. You should carefully read and check all of settings.py.\n\n DATABASES = {\n 'default': {\n 'ENGINE': 'django.db.backends.sqlite3',\n 'NAME': BASE_DIR / 'db.sqlite3',\n }\n }\n\nNext, run the database build commands:\n\n python3 manage.py makemigrations\n python3 manage.py makemigrations preservation-database\n python3 manage.py migrate \n\nYou should then have a working database into which you can import new preservation data.\n\n## Archive Notes\n\n### Internet Archive\nThe Internet Archive gives a KBART file for the Keepers Registry that we use as a primary ingest source: https://archive.org/details/ia-keepers-registry-kbart. However, this source is not the total coverage of the Internet Archive. However, sadly, the Internet Archive snapshots do not contain external identifiers and the container-level snapshots do not present coverage extent. While it is possible to download the entire 217GB FATCAT database snapshot, this will not be viable for many users. We have therefore stuck with the KBART file that Keepers uses. Extent of coverage in the Internet Archive may, therefore, be under-reported.\n\n# Credits\n* [Django](https://www.djangoproject.com/) for the ORM.\n* [Git](https://git-scm.com/) from Linus Torvalds _et al_.\n* [.gitignore](https://github.com/github/gitignore) from Github.\n* [Rich](https://github.com/Textualize/rich) from Textualize.\n\n© Crossref 2023\n",
"bugtrack_url": null,
"license": null,
"summary": "A database builder for digital preservation information.",
"version": "0.0.106",
"project_urls": {
"Homepage": "https://gitlab.com/crossref/labs/preservationDatabase",
"changelog": "https://gitlab.com/crossref/labs/preservation-data/-/blob/main/CHANGELOG.md",
"documentation": "https://labs.crossref.org",
"homepage": "https://labs.crossref.org",
"repository": "https://gitlab.com/crossref/labs/preservationDatabase"
},
"split_keywords": [
"digital preservation",
" academic"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f90fcd255f2d9fd94598ab8c33a0b0cf0531a5278ca040fc8a62ceeff73b9cd3",
"md5": "e0019bd6db01e7520a89bc4ade1a37d6",
"sha256": "ca4a91a2305829a58e7817e138a7012e4f3d02cc8e3dd328aeb5f9d3355418bb"
},
"downloads": -1,
"filename": "preservation_database-0.0.106-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e0019bd6db01e7520a89bc4ade1a37d6",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 31319,
"upload_time": "2024-06-17T10:48:35",
"upload_time_iso_8601": "2024-06-17T10:48:35.057898Z",
"url": "https://files.pythonhosted.org/packages/f9/0f/cd255f2d9fd94598ab8c33a0b0cf0531a5278ca040fc8a62ceeff73b9cd3/preservation_database-0.0.106-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "716091715c4624be8b219a3bb43ee3e5d3dd727fa4d3e15506753b01cf2259fd",
"md5": "477dd4152df0a9e9d57f3bad76d64a3e",
"sha256": "225514afeaedfe5e0bd93d7f9fc24e2de422340083c844fb44ee891c6086a70d"
},
"downloads": -1,
"filename": "preservation_database-0.0.106.tar.gz",
"has_sig": false,
"md5_digest": "477dd4152df0a9e9d57f3bad76d64a3e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 28649,
"upload_time": "2024-06-17T10:48:44",
"upload_time_iso_8601": "2024-06-17T10:48:44.364602Z",
"url": "https://files.pythonhosted.org/packages/71/60/91715c4624be8b219a3bb43ee3e5d3dd727fa4d3e15506753b01cf2259fd/preservation_database-0.0.106.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-17 10:48:44",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "crossref",
"gitlab_project": "labs",
"lcname": "preservation-database"
}
JSON