<a href="https://explosion.ai"><img src="https://explosion.ai/assets/img/logo.svg" width="125" height="125" align="right" /></a>
# catalogue: Super lightweight function registries for your library
`catalogue` is a tiny, zero-dependencies library that makes it easy to **add
function (or object) registries** to your code. Function registries are helpful
when you have objects that need to be both easily serializable and fully
customizable. Instead of passing a function into your object, you pass in an
identifier name, which the object can use to lookup the function from the
registry. This makes the object easy to serialize, because the name is a simple
string. If you instead saved the function, you'd have to use Pickle for
serialization, which has many drawbacks.
[](https://github.com/explosion/catalogue/actions/workflows/tests.yml)
[](https://github.com/explosion/catalogue/releases)
[](https://pypi.org/project/catalogue/)
[](https://anaconda.org/conda-forge/catalogue)
[](https://github.com/ambv/black)
## ⏳ Installation
```bash
pip install catalogue
```
```bash
conda install -c conda-forge catalogue
```
> ⚠️ **Important note:** `catalogue` v2.0+ is only compatible with Python 3.6+.
> For Python 2.7+ compatibility, use `catalogue` v1.x.
## 👩💻 Usage
Let's imagine you're developing a Python package that needs to load data
somewhere. You've already implemented some loader functions for the most common
data types, but you want to allow the user to easily add their own. Using
`catalogue.create` you can create a new registry under the namespace
`your_package` → `loaders`.
```python
# YOUR PACKAGE
import catalogue
loaders = catalogue.create("your_package", "loaders")
```
This gives you a `loaders.register` decorator that your users can import and
decorate their custom loader functions with.
```python
# USER CODE
from your_package import loaders
@loaders.register("custom_loader")
def custom_loader(data):
# Load something here...
return data
```
The decorated function will be registered automatically and in your package,
you'll be able to access all loaders by calling `loaders.get_all`.
```python
# YOUR PACKAGE
def load_data(data, loader_id):
print("All loaders:", loaders.get_all()) # {"custom_loader": <custom_loader>}
loader = loaders.get(loader_id)
return loader(data)
```
The user can now refer to their custom loader using only its string name
(`"custom_loader"`) and your application will know what to do and will use their
custom function.
```python
# USER CODE
from your_package import load_data
load_data(data, loader_id="custom_loader")
```
## ❓ FAQ
#### But can't the user just pass in the `custom_loader` function directly?
Sure, that's the more classic callback approach. Instead of a string ID,
`load_data` could also take a function, in which case you wouldn't need a
package like this. `catalogue` helps you when you need to produce a serializable
record of which functions were passed in. For instance, you might want to write
a log message, or save a config to load back your object later. With
`catalogue`, your functions can be parameterized by strings, so logging and
serialization remains easy – while still giving you full extensibility.
#### How do I make sure all of the registration decorators have run?
Decorators normally run when modules are imported. Relying on this side-effect
can sometimes lead to confusion, especially if there's no other reason the
module would be imported. One solution is to use
[entry points](https://packaging.python.org/specifications/entry-points/).
For instance, in [spaCy](https://spacy.io) we're starting to use function
registries to make the pipeline components much more customizable. Let's say one
user, Jo, develops a better tagging model using new machine learning research.
End-users of Jo's package should be able to write
`spacy.load("jo_tagging_model")`. They shouldn't need to remember to write
`import jos_tagged_model` first, just to run the function registries as a
side-effect. With entry points, the registration happens at install time – so
you don't need to rely on the import side-effects.
## 🎛 API
### <kbd>function</kbd> `catalogue.create`
Create a new registry for a given namespace. Returns a setter function that can
be used as a decorator or called with a name and `func` keyword argument. If
`entry_points=True` is set, the registry will check for
[Python entry points](https://packaging.python.org/tutorials/packaging-projects/#entry-points)
advertised for the given namespace, e.g. the entry point group
`spacy_architectures` for the namespace `"spacy", "architectures"`, in
`Registry.get` and `Registry.get_all`. This allows other packages to
auto-register functions.
| Argument | Type | Description |
| -------------- | ---------- | ---------------------------------------------------------------------------------------------- |
| `*namespace` | str | The namespace, e.g. `"spacy"` or `"spacy", "architectures"`. |
| `entry_points` | bool | Whether to check for entry points of the given namespace and pre-populate the global registry. |
| **RETURNS** | `Registry` | The `Registry` object with methods to register and retrieve functions. |
```python
architectures = catalogue.create("spacy", "architectures")
# Use as decorator
@architectures.register("custom_architecture")
def custom_architecture():
pass
# Use as regular function
architectures.register("custom_architecture", func=custom_architecture)
```
### <kbd>class</kbd> `Registry`
The registry object that can be used to register and retrieve functions. It's
usually created internally when you call `catalogue.create`.
#### <kbd>method</kbd> `Registry.__init__`
Initialize a new registry. If `entry_points=True` is set, the registry will
check for
[Python entry points](https://packaging.python.org/tutorials/packaging-projects/#entry-points)
advertised for the given namespace, e.g. the entry point group
`spacy_architectures` for the namespace `"spacy", "architectures"`, in
`Registry.get` and `Registry.get_all`.
| Argument | Type | Description |
| -------------- | ---------- | -------------------------------------------------------------------------------- |
| `namespace` | Tuple[str] | The namespace, e.g. `"spacy"` or `"spacy", "architectures"`. |
| `entry_points` | bool | Whether to check for entry points of the given namespace in `get` and `get_all`. |
| **RETURNS** | `Registry` | The newly created object. |
```python
# User-facing API
architectures = catalogue.create("spacy", "architectures")
# Internal API
architectures = Registry(("spacy", "architectures"))
```
#### <kbd>method</kbd> `Registry.__contains__`
Check whether a name is in the registry.
| Argument | Type | Description |
| ----------- | ---- | ------------------------------------ |
| `name` | str | The name to check. |
| **RETURNS** | bool | Whether the name is in the registry. |
```python
architectures = catalogue.create("spacy", "architectures")
@architectures.register("custom_architecture")
def custom_architecture():
pass
assert "custom_architecture" in architectures
```
#### <kbd>method</kbd> `Registry.__call__`
Register a function in the registry's namespace. Can be used as a decorator or
called as a function with the `func` keyword argument supplying the function to
register. Delegates to `Registry.register`.
#### <kbd>method</kbd> `Registry.register`
Register a function in the registry's namespace. Can be used as a decorator or
called as a function with the `func` keyword argument supplying the function to
register.
| Argument | Type | Description |
| ----------- | -------- | --------------------------------------------------------- |
| `name` | str | The name to register under the namespace. |
| `func` | Any | Optional function to register (if not used as decorator). |
| **RETURNS** | Callable | The decorator that takes one argument, the name. |
```python
architectures = catalogue.create("spacy", "architectures")
# Use as decorator
@architectures.register("custom_architecture")
def custom_architecture():
pass
# Use as regular function
architectures.register("custom_architecture", func=custom_architecture)
```
#### <kbd>method</kbd> `Registry.get`
Get a function registered in the namespace.
| Argument | Type | Description |
| ----------- | ---- | ------------------------ |
| `name` | str | The name. |
| **RETURNS** | Any | The registered function. |
```python
custom_architecture = architectures.get("custom_architecture")
```
#### <kbd>method</kbd> `Registry.get_all`
Get all functions in the registry's namespace.
| Argument | Type | Description |
| ----------- | -------------- | ---------------------------------------- |
| **RETURNS** | Dict[str, Any] | The registered functions, keyed by name. |
```python
all_architectures = architectures.get_all()
# {"custom_architecture": <custom_architecture>}
```
#### <kbd>method</kbd> `Registry.get_entry_points`
Get registered entry points from other packages for this namespace. The name of
the entry point group is the namespace joined by `_`.
| Argument | Type | Description |
| ----------- | -------------- | --------------------------------------- |
| **RETURNS** | Dict[str, Any] | The loaded entry points, keyed by name. |
```python
architectures = catalogue.create("spacy", "architectures", entry_points=True)
# Will get all entry points of the group "spacy_architectures"
all_entry_points = architectures.get_entry_points()
```
#### <kbd>method</kbd> `Registry.get_entry_point`
Check if registered entry point is available for a given name in the namespace
and load it. Otherwise, return the default value.
| Argument | Type | Description |
| ----------- | ---- | ------------------------------------------------ |
| `name` | str | Name of entry point to load. |
| `default` | Any | The default value to return. Defaults to `None`. |
| **RETURNS** | Any | The loaded entry point or the default value. |
```python
architectures = catalogue.create("spacy", "architectures", entry_points=True)
# Will get entry point "custom_architecture" of the group "spacy_architectures"
custom_architecture = architectures.get_entry_point("custom_architecture")
```
#### <kbd>method</kbd> `Registry.find`
Find the information about a registered function, including the module and path
to the file it's defined in, the line number and the docstring, if available.
| Argument | Type | Description |
| ----------- | -------------------------- | ----------------------------------- |
| `name` | str | Name of the registered function. |
| **RETURNS** | Dict[str, Union[str, int]] | The information about the function. |
```python
import catalogue
architectures = catalogue.create("spacy", "architectures", entry_points=True)
@architectures("my_architecture")
def my_architecture():
"""This is an architecture"""
pass
info = architectures.find("my_architecture")
# {'module': 'your_package.architectures',
# 'file': '/path/to/your_package/architectures.py',
# 'line_no': 5,
# 'docstring': 'This is an architecture'}
```
### <kbd>function</kbd> `catalogue.check_exists`
Check if a namespace exists.
| Argument | Type | Description |
| ------------ | ---- | ------------------------------------------------------------ |
| `*namespace` | str | The namespace, e.g. `"spacy"` or `"spacy", "architectures"`. |
| **RETURNS** | bool | Whether the namespace exists. |
Raw data
{
"_id": null,
"home_page": "https://github.com/explosion/catalogue",
"name": "catalogue",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6",
"maintainer_email": "",
"keywords": "",
"author": "Explosion",
"author_email": "contact@explosion.ai",
"download_url": "https://files.pythonhosted.org/packages/38/b4/244d58127e1cdf04cf2dc7d9566f0d24ef01d5ce21811bab088ecc62b5ea/catalogue-2.0.10.tar.gz",
"platform": null,
"description": "<a href=\"https://explosion.ai\"><img src=\"https://explosion.ai/assets/img/logo.svg\" width=\"125\" height=\"125\" align=\"right\" /></a>\n\n# catalogue: Super lightweight function registries for your library\n\n`catalogue` is a tiny, zero-dependencies library that makes it easy to **add\nfunction (or object) registries** to your code. Function registries are helpful\nwhen you have objects that need to be both easily serializable and fully\ncustomizable. Instead of passing a function into your object, you pass in an\nidentifier name, which the object can use to lookup the function from the\nregistry. This makes the object easy to serialize, because the name is a simple\nstring. If you instead saved the function, you'd have to use Pickle for\nserialization, which has many drawbacks.\n\n[](https://github.com/explosion/catalogue/actions/workflows/tests.yml)\n[](https://github.com/explosion/catalogue/releases)\n[](https://pypi.org/project/catalogue/)\n[](https://anaconda.org/conda-forge/catalogue)\n[](https://github.com/ambv/black)\n\n## \u23f3 Installation\n\n```bash\npip install catalogue\n```\n\n```bash\nconda install -c conda-forge catalogue\n```\n\n> \u26a0\ufe0f **Important note:** `catalogue` v2.0+ is only compatible with Python 3.6+.\n> For Python 2.7+ compatibility, use `catalogue` v1.x.\n\n## \ud83d\udc69\u200d\ud83d\udcbb Usage\n\nLet's imagine you're developing a Python package that needs to load data\nsomewhere. You've already implemented some loader functions for the most common\ndata types, but you want to allow the user to easily add their own. Using\n`catalogue.create` you can create a new registry under the namespace\n`your_package` → `loaders`.\n\n```python\n# YOUR PACKAGE\nimport catalogue\n\nloaders = catalogue.create(\"your_package\", \"loaders\")\n```\n\nThis gives you a `loaders.register` decorator that your users can import and\ndecorate their custom loader functions with.\n\n```python\n# USER CODE\nfrom your_package import loaders\n\n@loaders.register(\"custom_loader\")\ndef custom_loader(data):\n # Load something here...\n return data\n```\n\nThe decorated function will be registered automatically and in your package,\nyou'll be able to access all loaders by calling `loaders.get_all`.\n\n```python\n# YOUR PACKAGE\ndef load_data(data, loader_id):\n print(\"All loaders:\", loaders.get_all()) # {\"custom_loader\": <custom_loader>}\n loader = loaders.get(loader_id)\n return loader(data)\n```\n\nThe user can now refer to their custom loader using only its string name\n(`\"custom_loader\"`) and your application will know what to do and will use their\ncustom function.\n\n```python\n# USER CODE\nfrom your_package import load_data\n\nload_data(data, loader_id=\"custom_loader\")\n```\n\n## \u2753 FAQ\n\n#### But can't the user just pass in the `custom_loader` function directly?\n\nSure, that's the more classic callback approach. Instead of a string ID,\n`load_data` could also take a function, in which case you wouldn't need a\npackage like this. `catalogue` helps you when you need to produce a serializable\nrecord of which functions were passed in. For instance, you might want to write\na log message, or save a config to load back your object later. With\n`catalogue`, your functions can be parameterized by strings, so logging and\nserialization remains easy \u2013 while still giving you full extensibility.\n\n#### How do I make sure all of the registration decorators have run?\n\nDecorators normally run when modules are imported. Relying on this side-effect\ncan sometimes lead to confusion, especially if there's no other reason the\nmodule would be imported. One solution is to use\n[entry points](https://packaging.python.org/specifications/entry-points/).\n\nFor instance, in [spaCy](https://spacy.io) we're starting to use function\nregistries to make the pipeline components much more customizable. Let's say one\nuser, Jo, develops a better tagging model using new machine learning research.\nEnd-users of Jo's package should be able to write\n`spacy.load(\"jo_tagging_model\")`. They shouldn't need to remember to write\n`import jos_tagged_model` first, just to run the function registries as a\nside-effect. With entry points, the registration happens at install time \u2013 so\nyou don't need to rely on the import side-effects.\n\n## \ud83c\udf9b API\n\n### <kbd>function</kbd> `catalogue.create`\n\nCreate a new registry for a given namespace. Returns a setter function that can\nbe used as a decorator or called with a name and `func` keyword argument. If\n`entry_points=True` is set, the registry will check for\n[Python entry points](https://packaging.python.org/tutorials/packaging-projects/#entry-points)\nadvertised for the given namespace, e.g. the entry point group\n`spacy_architectures` for the namespace `\"spacy\", \"architectures\"`, in\n`Registry.get` and `Registry.get_all`. This allows other packages to\nauto-register functions.\n\n| Argument | Type | Description |\n| -------------- | ---------- | ---------------------------------------------------------------------------------------------- |\n| `*namespace` | str | The namespace, e.g. `\"spacy\"` or `\"spacy\", \"architectures\"`. |\n| `entry_points` | bool | Whether to check for entry points of the given namespace and pre-populate the global registry. |\n| **RETURNS** | `Registry` | The `Registry` object with methods to register and retrieve functions. |\n\n```python\narchitectures = catalogue.create(\"spacy\", \"architectures\")\n\n# Use as decorator\n@architectures.register(\"custom_architecture\")\ndef custom_architecture():\n pass\n\n# Use as regular function\narchitectures.register(\"custom_architecture\", func=custom_architecture)\n```\n\n### <kbd>class</kbd> `Registry`\n\nThe registry object that can be used to register and retrieve functions. It's\nusually created internally when you call `catalogue.create`.\n\n#### <kbd>method</kbd> `Registry.__init__`\n\nInitialize a new registry. If `entry_points=True` is set, the registry will\ncheck for\n[Python entry points](https://packaging.python.org/tutorials/packaging-projects/#entry-points)\nadvertised for the given namespace, e.g. the entry point group\n`spacy_architectures` for the namespace `\"spacy\", \"architectures\"`, in\n`Registry.get` and `Registry.get_all`.\n\n| Argument | Type | Description |\n| -------------- | ---------- | -------------------------------------------------------------------------------- |\n| `namespace` | Tuple[str] | The namespace, e.g. `\"spacy\"` or `\"spacy\", \"architectures\"`. |\n| `entry_points` | bool | Whether to check for entry points of the given namespace in `get` and `get_all`. |\n| **RETURNS** | `Registry` | The newly created object. |\n\n```python\n# User-facing API\narchitectures = catalogue.create(\"spacy\", \"architectures\")\n# Internal API\narchitectures = Registry((\"spacy\", \"architectures\"))\n```\n\n#### <kbd>method</kbd> `Registry.__contains__`\n\nCheck whether a name is in the registry.\n\n| Argument | Type | Description |\n| ----------- | ---- | ------------------------------------ |\n| `name` | str | The name to check. |\n| **RETURNS** | bool | Whether the name is in the registry. |\n\n```python\narchitectures = catalogue.create(\"spacy\", \"architectures\")\n\n@architectures.register(\"custom_architecture\")\ndef custom_architecture():\n pass\n\nassert \"custom_architecture\" in architectures\n```\n\n#### <kbd>method</kbd> `Registry.__call__`\n\nRegister a function in the registry's namespace. Can be used as a decorator or\ncalled as a function with the `func` keyword argument supplying the function to\nregister. Delegates to `Registry.register`.\n\n#### <kbd>method</kbd> `Registry.register`\n\nRegister a function in the registry's namespace. Can be used as a decorator or\ncalled as a function with the `func` keyword argument supplying the function to\nregister.\n\n| Argument | Type | Description |\n| ----------- | -------- | --------------------------------------------------------- |\n| `name` | str | The name to register under the namespace. |\n| `func` | Any | Optional function to register (if not used as decorator). |\n| **RETURNS** | Callable | The decorator that takes one argument, the name. |\n\n```python\narchitectures = catalogue.create(\"spacy\", \"architectures\")\n\n# Use as decorator\n@architectures.register(\"custom_architecture\")\ndef custom_architecture():\n pass\n\n# Use as regular function\narchitectures.register(\"custom_architecture\", func=custom_architecture)\n```\n\n#### <kbd>method</kbd> `Registry.get`\n\nGet a function registered in the namespace.\n\n| Argument | Type | Description |\n| ----------- | ---- | ------------------------ |\n| `name` | str | The name. |\n| **RETURNS** | Any | The registered function. |\n\n```python\ncustom_architecture = architectures.get(\"custom_architecture\")\n```\n\n#### <kbd>method</kbd> `Registry.get_all`\n\nGet all functions in the registry's namespace.\n\n| Argument | Type | Description |\n| ----------- | -------------- | ---------------------------------------- |\n| **RETURNS** | Dict[str, Any] | The registered functions, keyed by name. |\n\n```python\nall_architectures = architectures.get_all()\n# {\"custom_architecture\": <custom_architecture>}\n```\n\n#### <kbd>method</kbd> `Registry.get_entry_points`\n\nGet registered entry points from other packages for this namespace. The name of\nthe entry point group is the namespace joined by `_`.\n\n| Argument | Type | Description |\n| ----------- | -------------- | --------------------------------------- |\n| **RETURNS** | Dict[str, Any] | The loaded entry points, keyed by name. |\n\n```python\narchitectures = catalogue.create(\"spacy\", \"architectures\", entry_points=True)\n# Will get all entry points of the group \"spacy_architectures\"\nall_entry_points = architectures.get_entry_points()\n```\n\n#### <kbd>method</kbd> `Registry.get_entry_point`\n\nCheck if registered entry point is available for a given name in the namespace\nand load it. Otherwise, return the default value.\n\n| Argument | Type | Description |\n| ----------- | ---- | ------------------------------------------------ |\n| `name` | str | Name of entry point to load. |\n| `default` | Any | The default value to return. Defaults to `None`. |\n| **RETURNS** | Any | The loaded entry point or the default value. |\n\n```python\narchitectures = catalogue.create(\"spacy\", \"architectures\", entry_points=True)\n# Will get entry point \"custom_architecture\" of the group \"spacy_architectures\"\ncustom_architecture = architectures.get_entry_point(\"custom_architecture\")\n```\n\n#### <kbd>method</kbd> `Registry.find`\n\nFind the information about a registered function, including the module and path\nto the file it's defined in, the line number and the docstring, if available.\n\n| Argument | Type | Description |\n| ----------- | -------------------------- | ----------------------------------- |\n| `name` | str | Name of the registered function. |\n| **RETURNS** | Dict[str, Union[str, int]] | The information about the function. |\n\n```python\nimport catalogue\n\narchitectures = catalogue.create(\"spacy\", \"architectures\", entry_points=True)\n\n@architectures(\"my_architecture\")\ndef my_architecture():\n \"\"\"This is an architecture\"\"\"\n pass\n\ninfo = architectures.find(\"my_architecture\")\n# {'module': 'your_package.architectures',\n# 'file': '/path/to/your_package/architectures.py',\n# 'line_no': 5,\n# 'docstring': 'This is an architecture'}\n```\n\n### <kbd>function</kbd> `catalogue.check_exists`\n\nCheck if a namespace exists.\n\n| Argument | Type | Description |\n| ------------ | ---- | ------------------------------------------------------------ |\n| `*namespace` | str | The namespace, e.g. `\"spacy\"` or `\"spacy\", \"architectures\"`. |\n| **RETURNS** | bool | Whether the namespace exists. |\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Super lightweight function registries for your library",
"version": "2.0.10",
"project_urls": {
"Homepage": "https://github.com/explosion/catalogue"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "9e96d32b941a501ab566a16358d68b6eb4e4acc373fab3c3c4d7d9e649f7b4bb",
"md5": "ee30fc462b57aa308ef07385806070df",
"sha256": "58c2de0020aa90f4a2da7dfad161bf7b3b054c86a5f09fcedc0b2b740c109a9f"
},
"downloads": -1,
"filename": "catalogue-2.0.10-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ee30fc462b57aa308ef07385806070df",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6",
"size": 17325,
"upload_time": "2023-09-25T06:29:23",
"upload_time_iso_8601": "2023-09-25T06:29:23.337576Z",
"url": "https://files.pythonhosted.org/packages/9e/96/d32b941a501ab566a16358d68b6eb4e4acc373fab3c3c4d7d9e649f7b4bb/catalogue-2.0.10-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "38b4244d58127e1cdf04cf2dc7d9566f0d24ef01d5ce21811bab088ecc62b5ea",
"md5": "c7f2d1babd56156fadc071558791c483",
"sha256": "4f56daa940913d3f09d589c191c74e5a6d51762b3a9e37dd53b7437afd6cda15"
},
"downloads": -1,
"filename": "catalogue-2.0.10.tar.gz",
"has_sig": false,
"md5_digest": "c7f2d1babd56156fadc071558791c483",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.6",
"size": 19561,
"upload_time": "2023-09-25T06:29:24",
"upload_time_iso_8601": "2023-09-25T06:29:24.962308Z",
"url": "https://files.pythonhosted.org/packages/38/b4/244d58127e1cdf04cf2dc7d9566f0d24ef01d5ce21811bab088ecc62b5ea/catalogue-2.0.10.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-25 06:29:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "explosion",
"github_project": "catalogue",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "catalogue"
}
JSON
