# django-tenants-celery-beat
Support for celery beat in multitenant Django projects. Schedule periodic tasks for a
specific tenant, with flexibility to run tasks with respect to each tenant's timezone.
For use with [django-tenants](https://github.com/django-tenants/django-tenants) and
[tenant-schemas-celery](https://github.com/maciej-gol/tenant-schemas-celery).
Features:
- Configure static periodic tasks in `app.conf.beat_schedule` automatically for all
tenants, optionally in their own timezones
- Django admin modified to show and give you control over the tenant a task will run in
- Filter the admin based on tenants
- Tenant-level admin (e.g. tenant.domain.com) will only show tasks for that tenant
## Installation
Install via pip:
```commandline
pip install django-tenants-celery-beat
```
## Usage
Follow the instructions for [django-tenants](https://github.com/django-tenants/django-tenants)
and [tenant-schemas-celery](https://github.com/maciej-gol/tenant-schemas-celery).
In your `SHARED_APPS` (_not_ your `TENANT_APPS`):
```python
SHARED_APPS = [
# ...
"django_celery_results",
"django_celery_beat",
"django_tenants_celery_beat",
# ...
]
```
Depending on your setup, you may also put `django_celery_results` in your `TENANT_APPS`.
(Assuming you have followed the instructions for
[django-tenants](https://github.com/django-tenants/django-tenants)
all your `SHARED_APPS` will also appear in your `INSTALLED_APPS`.)
`django-tenants-celery-beat` requires your `Tenant` model to have a `timezone` field in
order to control periodic task scheduling. To this end, we provide a `TenantTimezoneMixin`
that you should inherit from in your `Tenant` model, e.g.:
```python
from django_tenants.models import TenantMixin
from django_tenants_celery_beat.models import TenantTimezoneMixin
class Tenant(TenantTimezoneMixin, TenantMixin):
pass
```
You can configure whether the timezones are displayed with the GMT offset, i.e.
`Australia/Sydney` vs. `GMT+11:00 Australia/Sydney`, using the setting
`TENANT_TIMEZONE_DISPLAY_GMT_OFFSET`. By default, the GMT offset is not shown.
(If you later change this setting, you will need to run `makemigrations` to see any effect.)
Ensure that `DJANGO_CELERY_BEAT_TZ_AWARE` is True (the default) for any timezone aware
scheduling to work.
In order to make the link between your `Tenant` model and `PeriodicTask`, the app comes
with an abstract model. You simply need create a class that inherits from this mixin and
does nothing else. Having this model in your own first-party app means that the migrations
can be managed properly.
```python
from django_tenants_celery_beat.models import PeriodicTaskTenantLinkMixin
class PeriodicTaskTenantLink(PeriodicTaskTenantLinkMixin):
pass
```
You need to register which model is acting as the link. If your tenancy models live in
an app called `tenancy` and the model is named as above, you need the following in your
project settings:
```python
PERIODIC_TASK_TENANT_LINK_MODEL = "tenancy.PeriodicTaskTenantLink"
```
Once this has been done, you will need to run `makemigrations`. This will create the
necessary migrations for your `Tenant` and `PeriodicTaskTenantLink` models.
To apply the migrations, run:
```commandline
python manage.py migrate_schemas --shared
```
### Setting up a `beat_schedule`
For statically configured periodic tasks assigned via `app.conf.beat_schedule`, there
is a helper utility function to produce a valid tenant-aware `beat_schedule`. You can take
an existing `beat_schedule` and make minor modifications to achieve the desired behaviour.
The `generate_beat_schedule` function takes a dict that looks exactly like the usual
`beat_schedule` dict, but each task contains an additional entry with the key `tenancy_options`.
Here you can specify three things:
- Should the task run in the `public` schema?
- Should the task run on all tenant schemas?
- Should the task scheduling use the tenant's timezone?
All of these are False by default, so you only need to include them if you set them to True,
though you may prefer to keep them there to be explicit about your intentions. At least one
of the `public` or `all_tenants` keys must be True, otherwise the entry is ignored.
Additionally, if the `tenancy_option` key is missing from an entry, that entry will be ignored.
Example usage:
```python
app.conf.beat_schedule = generate_beat_schedule(
{
"tenant_task": {
"task": "app.tasks.tenant_task",
"schedule": crontab(minute=0, hour=12, day_of_week=1),
"tenancy_options": {
"public": False,
"all_tenants": True,
"use_tenant_timezone": True,
}
},
"hourly_tenant_task": {
"task": "app.tasks.hourly_tenant_task",
"schedule": crontab(minute=0),
"tenancy_options": {
"public": False,
"all_tenants": True,
"use_tenant_timezone": False,
}
},
"public_task": {
"task": "app.tasks.tenant_task",
"schedule": crontab(minute=0, hour=0, day_of_month=1),
"tenancy_options": {
"public": True,
"all_tenants": False,
}
}
}
)
```
This `beat_schedule` will actually produce an entry for each tenant with the schema name
as a prefix. For example, `tenant1: celery.backend_cleanup`. For public tasks, there is
no prefix added to the name.
This function also sets some AMQP message headers, which is how the schema and timezone
settings are configured.
#### Configuring `celery.backend_cleanup`
Note that in many cases, tasks should not be both run on the `public` schema and on all
tenant schemas, as the database tables are often very different. One example that most
likely should is the `celery.backend_cleanup` task that is automatically added. If you
do nothing with it, it will run only in the public schema, which may or may not suit your
needs. Assuming you have `django_celery_results` in `TENANT_APPS` you will need this task to
be run on all tenants, and if you also have it in `SHARED_APPS`, you will need it to run
on the `public` schema too. This task is also a case where you will likely want it to run
in the tenant's timezone so it always runs during a quiet time.
Using the utility function, this is how we could set up the `celery.backend_cleanup` task:
```python
from django_tenants_celery_beat.utils import generate_beat_schedule
# ...
app.conf.beat_schedule = generate_beat_schedule(
{
"celery.backend_cleanup": {
"task": "celery.backend_cleanup",
"schedule": crontab("0", "4", "*"),
"options": {"expire_seconds": 12 * 3600},
"tenancy_options": {
"public": True,
"all_tenants": True,
"use_tenant_timezone": True,
}
}
}
)
```
This will prevent the automatically created one being added, though the settings are
identical to the automatic one as of `django-celery-beat==2.2.0`. You could also set
`public` to False here for exactly the same resulting schedule, as the public one will
be automatically created by `django-celery-beat`.
### Modifying Periodic Tasks in the Django Admin
You can further manage periodic tasks in the Django admin.
The public schema admin will display the periodic tasks for each tenant as well as the
public tenant.
When on a tenant-level admin (e.g. `tenant.domain.com`), you can only see
the tasks for the given tenant, and any filters are hidden so as to not show a list of
tenants.
When editing a `PeriodicTask`, there is an inline form for the `OneToOneModel` added by
this package that connects a `PeriodicTask` to a `Tenant`. You can toggle the
`use_tenant_timezone` setting (but when restarting the beat service, the `beat_schedule`
will always take precedence). The tenant is shown as a read-only field, unless you are
on the public admin site, in which case you have the option edit the tenant. Editing the
tenant here will take precedence over the `beat_schedule`.
## Developer Setup
To set up the example app:
1. Navigate into the `example` directory
2. Create a virtual environment and install the requirements in `requirements.txt`
3. Create a postgres database according to the `example.settings.DATABASES["default"]` (edit the settings if necessary)
4. Run `python manage.py migrate_schemas` to create the public schema
5. Run `python manage.py create_tenant` to create the public tenant and any other tenants
6. Create superusers with `python manage.py create_tenant_superuser`
7. Run `celery -A example beat --loglevel=INFO` to run the beat scheduler
8. Run `celery -A example worker --loglevel=INFO` (add `--pool=solo` if on Windows)
Raw data
{
"_id": null,
"home_page": "https://github.com/QuickRelease/django-tenants-celery-beat",
"name": "django-tenants-celery-beat",
"maintainer": "Quick Release (Automotive) Ltd.",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "django tenants celery beat multitenancy postgres postgresql",
"author": "David Vaughan",
"author_email": "david.vaughan@quickrelease.co.uk",
"download_url": "https://files.pythonhosted.org/packages/8d/4d/00eb1c61c77dbb570750f9dcd1e858319baa22b40b3c48f120b752ad51dd/django-tenants-celery-beat-0.2.1.tar.gz",
"platform": null,
"description": "# django-tenants-celery-beat\n\nSupport for celery beat in multitenant Django projects. Schedule periodic tasks for a\nspecific tenant, with flexibility to run tasks with respect to each tenant's timezone. \n\nFor use with [django-tenants](https://github.com/django-tenants/django-tenants) and\n[tenant-schemas-celery](https://github.com/maciej-gol/tenant-schemas-celery).\n\nFeatures:\n- Configure static periodic tasks in `app.conf.beat_schedule` automatically for all\ntenants, optionally in their own timezones\n- Django admin modified to show and give you control over the tenant a task will run in\n- Filter the admin based on tenants\n- Tenant-level admin (e.g. tenant.domain.com) will only show tasks for that tenant\n\n## Installation\n\nInstall via pip:\n```commandline\npip install django-tenants-celery-beat\n```\n\n## Usage\n\nFollow the instructions for [django-tenants](https://github.com/django-tenants/django-tenants)\nand [tenant-schemas-celery](https://github.com/maciej-gol/tenant-schemas-celery).\n\nIn your `SHARED_APPS` (_not_ your `TENANT_APPS`):\n```python\nSHARED_APPS = [\n # ...\n \"django_celery_results\",\n \"django_celery_beat\",\n \"django_tenants_celery_beat\",\n # ...\n]\n```\nDepending on your setup, you may also put `django_celery_results` in your `TENANT_APPS`.\n(Assuming you have followed the instructions for\n[django-tenants](https://github.com/django-tenants/django-tenants)\nall your `SHARED_APPS` will also appear in your `INSTALLED_APPS`.)\n\n`django-tenants-celery-beat` requires your `Tenant` model to have a `timezone` field in\norder to control periodic task scheduling. To this end, we provide a `TenantTimezoneMixin`\nthat you should inherit from in your `Tenant` model, e.g.:\n```python\nfrom django_tenants.models import TenantMixin\nfrom django_tenants_celery_beat.models import TenantTimezoneMixin\n\nclass Tenant(TenantTimezoneMixin, TenantMixin):\n pass\n```\nYou can configure whether the timezones are displayed with the GMT offset, i.e.\n`Australia/Sydney` vs. `GMT+11:00 Australia/Sydney`, using the setting\n`TENANT_TIMEZONE_DISPLAY_GMT_OFFSET`. By default, the GMT offset is not shown.\n(If you later change this setting, you will need to run `makemigrations` to see any effect.)\n\nEnsure that `DJANGO_CELERY_BEAT_TZ_AWARE` is True (the default) for any timezone aware\nscheduling to work.\n\nIn order to make the link between your `Tenant` model and `PeriodicTask`, the app comes\nwith an abstract model. You simply need create a class that inherits from this mixin and\ndoes nothing else. Having this model in your own first-party app means that the migrations\ncan be managed properly.\n```python\nfrom django_tenants_celery_beat.models import PeriodicTaskTenantLinkMixin\n\nclass PeriodicTaskTenantLink(PeriodicTaskTenantLinkMixin):\n pass\n```\nYou need to register which model is acting as the link. If your tenancy models live in\nan app called `tenancy` and the model is named as above, you need the following in your\nproject settings:\n```python\nPERIODIC_TASK_TENANT_LINK_MODEL = \"tenancy.PeriodicTaskTenantLink\"\n```\n\nOnce this has been done, you will need to run `makemigrations`. This will create the\nnecessary migrations for your `Tenant` and `PeriodicTaskTenantLink` models.\nTo apply the migrations, run:\n```commandline\npython manage.py migrate_schemas --shared\n```\n\n### Setting up a `beat_schedule`\n\nFor statically configured periodic tasks assigned via `app.conf.beat_schedule`, there\nis a helper utility function to produce a valid tenant-aware `beat_schedule`. You can take\nan existing `beat_schedule` and make minor modifications to achieve the desired behaviour.\n\nThe `generate_beat_schedule` function takes a dict that looks exactly like the usual\n`beat_schedule` dict, but each task contains an additional entry with the key `tenancy_options`.\nHere you can specify three things:\n- Should the task run in the `public` schema?\n- Should the task run on all tenant schemas?\n- Should the task scheduling use the tenant's timezone?\n\nAll of these are False by default, so you only need to include them if you set them to True,\nthough you may prefer to keep them there to be explicit about your intentions. At least one\nof the `public` or `all_tenants` keys must be True, otherwise the entry is ignored.\nAdditionally, if the `tenancy_option` key is missing from an entry, that entry will be ignored.\n\nExample usage:\n```python\napp.conf.beat_schedule = generate_beat_schedule(\n {\n \"tenant_task\": {\n \"task\": \"app.tasks.tenant_task\",\n \"schedule\": crontab(minute=0, hour=12, day_of_week=1),\n \"tenancy_options\": {\n \"public\": False,\n \"all_tenants\": True,\n \"use_tenant_timezone\": True,\n }\n },\n \"hourly_tenant_task\": {\n \"task\": \"app.tasks.hourly_tenant_task\",\n \"schedule\": crontab(minute=0),\n \"tenancy_options\": {\n \"public\": False,\n \"all_tenants\": True,\n \"use_tenant_timezone\": False,\n }\n },\n \"public_task\": {\n \"task\": \"app.tasks.tenant_task\",\n \"schedule\": crontab(minute=0, hour=0, day_of_month=1),\n \"tenancy_options\": {\n \"public\": True,\n \"all_tenants\": False,\n }\n }\n }\n)\n```\nThis `beat_schedule` will actually produce an entry for each tenant with the schema name\nas a prefix. For example, `tenant1: celery.backend_cleanup`. For public tasks, there is\nno prefix added to the name.\n\nThis function also sets some AMQP message headers, which is how the schema and timezone\nsettings are configured.\n\n#### Configuring `celery.backend_cleanup`\n\nNote that in many cases, tasks should not be both run on the `public` schema and on all\ntenant schemas, as the database tables are often very different. One example that most\nlikely should is the `celery.backend_cleanup` task that is automatically added. If you\ndo nothing with it, it will run only in the public schema, which may or may not suit your\nneeds. Assuming you have `django_celery_results` in `TENANT_APPS` you will need this task to\nbe run on all tenants, and if you also have it in `SHARED_APPS`, you will need it to run\non the `public` schema too. This task is also a case where you will likely want it to run\nin the tenant's timezone so it always runs during a quiet time.\n\nUsing the utility function, this is how we could set up the `celery.backend_cleanup` task:\n```python\nfrom django_tenants_celery_beat.utils import generate_beat_schedule\n\n# ...\n\napp.conf.beat_schedule = generate_beat_schedule(\n {\n \"celery.backend_cleanup\": {\n \"task\": \"celery.backend_cleanup\",\n \"schedule\": crontab(\"0\", \"4\", \"*\"),\n \"options\": {\"expire_seconds\": 12 * 3600},\n \"tenancy_options\": {\n \"public\": True,\n \"all_tenants\": True,\n \"use_tenant_timezone\": True,\n }\n }\n }\n)\n```\nThis will prevent the automatically created one being added, though the settings are\nidentical to the automatic one as of `django-celery-beat==2.2.0`. You could also set\n`public` to False here for exactly the same resulting schedule, as the public one will\nbe automatically created by `django-celery-beat`.\n\n### Modifying Periodic Tasks in the Django Admin\n\nYou can further manage periodic tasks in the Django admin.\n\nThe public schema admin will display the periodic tasks for each tenant as well as the\npublic tenant.\n\nWhen on a tenant-level admin (e.g. `tenant.domain.com`), you can only see\nthe tasks for the given tenant, and any filters are hidden so as to not show a list of\ntenants.\n\nWhen editing a `PeriodicTask`, there is an inline form for the `OneToOneModel` added by\nthis package that connects a `PeriodicTask` to a `Tenant`. You can toggle the\n`use_tenant_timezone` setting (but when restarting the beat service, the `beat_schedule`\nwill always take precedence). The tenant is shown as a read-only field, unless you are\non the public admin site, in which case you have the option edit the tenant. Editing the\ntenant here will take precedence over the `beat_schedule`.\n\n## Developer Setup\n\nTo set up the example app:\n1. Navigate into the `example` directory\n2. Create a virtual environment and install the requirements in `requirements.txt`\n3. Create a postgres database according to the `example.settings.DATABASES[\"default\"]` (edit the settings if necessary)\n4. Run `python manage.py migrate_schemas` to create the public schema\n5. Run `python manage.py create_tenant` to create the public tenant and any other tenants\n6. Create superusers with `python manage.py create_tenant_superuser`\n7. Run `celery -A example beat --loglevel=INFO` to run the beat scheduler\n8. Run `celery -A example worker --loglevel=INFO` (add `--pool=solo` if on Windows)\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Support for celery beat in multitenant Django projects",
"version": "0.2.1",
"project_urls": {
"Homepage": "https://github.com/QuickRelease/django-tenants-celery-beat"
},
"split_keywords": [
"django",
"tenants",
"celery",
"beat",
"multitenancy",
"postgres",
"postgresql"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "f40df2a72ec5cfd555b4d3e057ed107e64ad385ac2c726da988a90e86c2f3525",
"md5": "e45920e0dfd36a6910b4bc14e00e864d",
"sha256": "cf283096cf65ef2d9f1eca86a5ade6c99da86c950d9c0e4761108e45e4565846"
},
"downloads": -1,
"filename": "django_tenants_celery_beat-0.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e45920e0dfd36a6910b4bc14e00e864d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 9905,
"upload_time": "2023-08-04T09:35:16",
"upload_time_iso_8601": "2023-08-04T09:35:16.962612Z",
"url": "https://files.pythonhosted.org/packages/f4/0d/f2a72ec5cfd555b4d3e057ed107e64ad385ac2c726da988a90e86c2f3525/django_tenants_celery_beat-0.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8d4d00eb1c61c77dbb570750f9dcd1e858319baa22b40b3c48f120b752ad51dd",
"md5": "77208fe2bc733a4e5271d69c17dcd6a5",
"sha256": "780ccfcd6ff0aa02905776ab5461d7b8a81a65bc0890c274f8cf88854081b35b"
},
"downloads": -1,
"filename": "django-tenants-celery-beat-0.2.1.tar.gz",
"has_sig": false,
"md5_digest": "77208fe2bc733a4e5271d69c17dcd6a5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11363,
"upload_time": "2023-08-04T09:35:17",
"upload_time_iso_8601": "2023-08-04T09:35:17.976362Z",
"url": "https://files.pythonhosted.org/packages/8d/4d/00eb1c61c77dbb570750f9dcd1e858319baa22b40b3c48f120b752ad51dd/django-tenants-celery-beat-0.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-04 09:35:17",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "QuickRelease",
"github_project": "django-tenants-celery-beat",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "django-tenants-celery-beat"
}
JSON
