# Work-set Clustering
[](https://zenodo.org/doi/10.5281/zenodo.10011416)
A Python script to perform a clustering based on descriptive keys.
It can be used to identify _work_ clusters for _manifestations_ according to the FRBR (IFLA-LRM) model.
This tool only performs the clustering. It needs a list of manifestation identifiers and their descriptive keys as input.
If already computed cluster identifiers and descriptive keys from a previous run are provided, they can be reused.
## Usage via the command line
Create and activate a Python virtual environment
```bash
# Create a new Python virtual environment
python3 -m venv py-request-isni-env
# Activate the virtual environment
source py-request-isni-env/bin/activate
# There are no depdendenies to install
# install the tool
pip install .
```
Available options:
```
usage: clustering.py [-h] -i INPUT_FILE -o OUTPUT_FILE --id-column ID_COLUMN --key-column KEY_COLUMN [--delimiter DELIMITER] [--existing-clusters EXISTING_CLUSTERS]
[--existing-clusters-keys EXISTING_CLUSTERS_KEYS]
optional arguments:
-h, --help show this help message and exit
-i INPUT_FILE, --input-file INPUT_FILE
The CSV file(s) with columns for elements and descriptive keys, one row is one element and descriptive key relationship
-o OUTPUT_FILE, --output-file OUTPUT_FILE
The name of the output CSV file containing two columns: elementID and clusterID
--id-column ID_COLUMN
The name of the column with element identifiers
--key-column KEY_COLUMN
The name of the column that contains a descriptive key
--delimiter DELIMITER
Optional delimiter of the input/output CSV, default is ','
--existing-clusters EXISTING_CLUSTERS
Optional file with existing element-cluster mapping
--existing-clusters-keys EXISTING_CLUSTERS_KEYS
Optional file with element-descriptive key mapping for existing clusters mapping
```
### Clustering from scratch
Given a CSV file where each row contains the relationship
between one manifestation identifier and one descriptive key,
the tool can be called the following to create cluster assignments.
```python
python -m work_set_clustering.clustering \
--input-file "descriptive-keys.csv" \
--output-file "clusters.csv" \
--id-column "elementID" \
--key-column "descriptiveKey"
```
Example CSV which should result in two clusters, one for book1 and book2 (due to a similar key) and one for book3:
|elementID|descriptiveKey|
|---------|--------------|
|book1|theTitle/author1|
|book1|isbnOfTheBook/author1|
|book2|isbnOfTheBook/author1|
|book3|otherBookTitle/author1|
The script can also read descriptive keys that are distributed across several files.
Therefore you only have to use the `--input-file` parameter several times.
Please note that all of those input files should have the same column names specified with `--id-column` and `--key-column`.
You can find more examples of cluster input in the `test/resources` directory.
### Reuse existing clusters
You can reuse the clusters created from an earlier run,
but you also have to provide the mapping between the previous elements and optionally their descriptive keys.
```python
python -m work_set_clustering.clustering \
--input-file "descriptive-keys.csv" \
--output-file "clusters.csv" \
--id-column "elementID" \
--key-column "descriptiveKey" \
--existing-clusters "existing-clusters.csv" \
--existing-cluster-keys "initial-descriptive-keys.csv"
```
Please note that with the two parameters `--existing-clusters` and `--existing-cluster-keys`
the data from a previous run are provided.
Similar to the initial clustering, you can provide several input files.
> [!NOTE]
> When skipping existing descriptive keys, existing cluster identifiers and assigments are kept, even if their elements have overlapping descriptive keys. Additionally, none of the new elements can be mapped to the existing clusters, because no descriptive keys are provided (more info in https://github.com/kbrbe/work-set-clustering/issues/9)
## Usage as a library
The tool can also be used as a library within another Python script or a Jupyter notebook.
```python
from work_set_clustering.clustering import clusterFromScratch as clustering
clustering(
inputFilename=["descriptive-keys.csv"],
outputFilename="cluster-assignments.csv",
idColumnName="elementID",
keyColumnName="descriptiveKey",
delimiter=',')
```
Or if you want to reuse existing clusters:
```python
from work_set_clustering.clustering import updateClusters as clustering
clustering(
inputFilename=["descriptive-keys.csv"],
outputFilename="cluster-assignments.csv",
idColumnName="elementID",
keyColumnName="descriptiveKey",
delimiter=',',
existingClustersFilename="existing-clusters.csv",
existingClusterKeysFilename="initial-descriptive-keys.csv")
```
## Software Tests
* You can execute the unit tests of the `lib.py` file with the following command: `python work_set_clustering.lib`.
* You can execute the integration tests with the following command: `python -m unittest discover -s test`
## Contact
Sven Lieber - Sven.Lieber@kbr.be - Royal Library of Belgium (KBR) - https://www.kbr.be/en/
Raw data
{
"_id": null,
"home_page": "https://github.com/kbrbe/work-set-clustering",
"name": "work-set-clustering",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "FRBRization FRBR work-set-clustering",
"author": "Sven Lieber",
"author_email": "Sven.Lieber@kbr.be",
"download_url": "https://files.pythonhosted.org/packages/89/46/0f2906b1d9f2b87cf8d52c3f149e6626b0e227f3c7ac2b78cec903a8f939/work_set_clustering-0.4.0.tar.gz",
"platform": null,
"description": "# Work-set Clustering\n\n[](https://zenodo.org/doi/10.5281/zenodo.10011416)\n\nA Python script to perform a clustering based on descriptive keys.\nIt can be used to identify _work_ clusters for _manifestations_ according to the FRBR (IFLA-LRM) model.\n\nThis tool only performs the clustering. It needs a list of manifestation identifiers and their descriptive keys as input.\nIf already computed cluster identifiers and descriptive keys from a previous run are provided, they can be reused.\n\n\n## Usage via the command line\n\nCreate and activate a Python virtual environment\n```bash\n\n# Create a new Python virtual environment\npython3 -m venv py-request-isni-env\n\n# Activate the virtual environment\nsource py-request-isni-env/bin/activate\n\n# There are no depdendenies to install\n\n# install the tool\npip install .\n```\n\nAvailable options:\n\n```\nusage: clustering.py [-h] -i INPUT_FILE -o OUTPUT_FILE --id-column ID_COLUMN --key-column KEY_COLUMN [--delimiter DELIMITER] [--existing-clusters EXISTING_CLUSTERS]\n [--existing-clusters-keys EXISTING_CLUSTERS_KEYS]\n\noptional arguments:\n -h, --help show this help message and exit\n -i INPUT_FILE, --input-file INPUT_FILE\n The CSV file(s) with columns for elements and descriptive keys, one row is one element and descriptive key relationship\n -o OUTPUT_FILE, --output-file OUTPUT_FILE\n The name of the output CSV file containing two columns: elementID and clusterID\n --id-column ID_COLUMN\n The name of the column with element identifiers\n --key-column KEY_COLUMN\n The name of the column that contains a descriptive key\n --delimiter DELIMITER\n Optional delimiter of the input/output CSV, default is ','\n --existing-clusters EXISTING_CLUSTERS\n Optional file with existing element-cluster mapping\n --existing-clusters-keys EXISTING_CLUSTERS_KEYS\n Optional file with element-descriptive key mapping for existing clusters mapping\n\n```\n\n### Clustering from scratch\nGiven a CSV file where each row contains the relationship\nbetween one manifestation identifier and one descriptive key,\nthe tool can be called the following to create cluster assignments.\n\n```python\npython -m work_set_clustering.clustering \\\n --input-file \"descriptive-keys.csv\" \\\n --output-file \"clusters.csv\" \\\n --id-column \"elementID\" \\\n --key-column \"descriptiveKey\"\n```\n\nExample CSV which should result in two clusters, one for book1 and book2 (due to a similar key) and one for book3:\n\n|elementID|descriptiveKey|\n|---------|--------------|\n|book1|theTitle/author1|\n|book1|isbnOfTheBook/author1|\n|book2|isbnOfTheBook/author1|\n|book3|otherBookTitle/author1|\n\nThe script can also read descriptive keys that are distributed across several files.\nTherefore you only have to use the `--input-file` parameter several times.\nPlease note that all of those input files should have the same column names specified with `--id-column` and `--key-column`.\n\nYou can find more examples of cluster input in the `test/resources` directory.\n\n### Reuse existing clusters\n\nYou can reuse the clusters created from an earlier run,\nbut you also have to provide the mapping between the previous elements and optionally their descriptive keys.\n\n\n```python\npython -m work_set_clustering.clustering \\\n --input-file \"descriptive-keys.csv\" \\\n --output-file \"clusters.csv\" \\\n --id-column \"elementID\" \\\n --key-column \"descriptiveKey\" \\\n --existing-clusters \"existing-clusters.csv\" \\\n --existing-cluster-keys \"initial-descriptive-keys.csv\"\n```\n\nPlease note that with the two parameters `--existing-clusters` and `--existing-cluster-keys`\nthe data from a previous run are provided.\n\nSimilar to the initial clustering, you can provide several input files.\n\n> [!NOTE]\n> When skipping existing descriptive keys, existing cluster identifiers and assigments are kept, even if their elements have overlapping descriptive keys. Additionally, none of the new elements can be mapped to the existing clusters, because no descriptive keys are provided (more info in https://github.com/kbrbe/work-set-clustering/issues/9)\n\n## Usage as a library\n\nThe tool can also be used as a library within another Python script or a Jupyter notebook.\n\n```python\nfrom work_set_clustering.clustering import clusterFromScratch as clustering\n\nclustering(\n inputFilename=[\"descriptive-keys.csv\"],\n outputFilename=\"cluster-assignments.csv\",\n idColumnName=\"elementID\",\n keyColumnName=\"descriptiveKey\",\n delimiter=',')\n```\n\nOr if you want to reuse existing clusters:\n\n```python\nfrom work_set_clustering.clustering import updateClusters as clustering\n\nclustering(\n inputFilename=[\"descriptive-keys.csv\"],\n outputFilename=\"cluster-assignments.csv\",\n idColumnName=\"elementID\",\n keyColumnName=\"descriptiveKey\",\n delimiter=',',\n existingClustersFilename=\"existing-clusters.csv\",\n existingClusterKeysFilename=\"initial-descriptive-keys.csv\")\n```\n\n## Software Tests\n\n* You can execute the unit tests of the `lib.py` file with the following command: `python work_set_clustering.lib`.\n* You can execute the integration tests with the following command: `python -m unittest discover -s test`\n\n## Contact\n\nSven Lieber - Sven.Lieber@kbr.be - Royal Library of Belgium (KBR) - https://www.kbr.be/en/\n\n",
"bugtrack_url": null,
"license": "AGPL-3.0",
"summary": "A Python script to perform a clustering based on descriptive keys.",
"version": "0.4.0",
"project_urls": {
"Homepage": "https://github.com/kbrbe/work-set-clustering"
},
"split_keywords": [
"frbrization",
"frbr",
"work-set-clustering"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "89460f2906b1d9f2b87cf8d52c3f149e6626b0e227f3c7ac2b78cec903a8f939",
"md5": "92752c35740889db522ffcb6c607bff1",
"sha256": "5f72c4039c0ccd272a7e65b5cc2753c46b4b669241496d2a247845b77bbc7581"
},
"downloads": -1,
"filename": "work_set_clustering-0.4.0.tar.gz",
"has_sig": false,
"md5_digest": "92752c35740889db522ffcb6c607bff1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 21662,
"upload_time": "2024-06-28T12:44:27",
"upload_time_iso_8601": "2024-06-28T12:44:27.398491Z",
"url": "https://files.pythonhosted.org/packages/89/46/0f2906b1d9f2b87cf8d52c3f149e6626b0e227f3c7ac2b78cec903a8f939/work_set_clustering-0.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-06-28 12:44:27",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kbrbe",
"github_project": "work-set-clustering",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "work-set-clustering"
}
JSON
