| Name | apf-base JSON |
| Version |
2.5.2
JSON |
| download |
| home_page | |
| Summary | ALeRCE Alert Processing Framework. |
| upload_time | 2023-07-21 16:03:10 |
| maintainer | |
| docs_url | None |
| author | ALeRCE Team |
| requires_python | |
| license | |
| keywords |
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
Alert Processing Framework (*apf*)
================================================
[](https://apf.readthedocs.io/en/latest/?badge=latest) [](https://codecov.io/gh/alercebroker/APF)
*apf* is a framework developed to create a dockerized pipeline to
process an alert stream, that can be easily be deployed in a local
machine or distributed using [Kubernetes](https://kubernetes.io).
First developed to process [ZTF data](https://www.ztf.caltech.edu/) it
is capable to be used for any stream/static data processing pipeline.
Installing *apf*
================
*apf* installation can be done with *pip*
``` {.sourceCode .bash}
pip install apf_base
```
This will install the *apf* python package and *apf* command line
script.
*apf* design
============
*apf* is based on *steps* conected through [Apache
Kafka](https://kafka.apache.org/) topics.
Each *step* is composed by a `consumer` and is isolated from other
steps inside a docker container.
When running, the step calls the **execute()** method for each *message* or *message batch*
consumed. A step can have multiple producers and databases back-ends
plugins that can be accessed inside the *execute* method to have a more
complex logic.
<p align="center">
<img src="docs/source/_static/images/apf-flow.png">
</p>
This generic step greatly reduce the development of each component of
the pipeline and make it easier to test each component separately.
*apf* features
==============
- Automatic Metric Sender (KafkaMetrics)
- Automatic Code Generation (`apf new-step <step_name>`)
- Multiple Consumer Plugins:
- Kafka
- AVRO
- CSV
- JSON
- Producers:
- Kafka
- CSV
- Metrics:
- Kafka
*apf* Quickstart
===================
A **quick-start** guide to create a new step can be found [here](https://apf.readthedocs.io/en/latest/new_step.html).
Raw data
{
"_id": null,
"home_page": "",
"name": "apf-base",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "ALeRCE Team",
"author_email": "contact@alerce.online",
"download_url": "https://files.pythonhosted.org/packages/a1/0c/505c0bd9651fffd695649978f6ef5efe026e982c268662aadda5ba8dc299/apf_base-2.5.2.tar.gz",
"platform": null,
"description": "Alert Processing Framework (*apf*)\n================================================\n\n[](https://apf.readthedocs.io/en/latest/?badge=latest) [](https://codecov.io/gh/alercebroker/APF)\n\n\n*apf* is a framework developed to create a dockerized pipeline to\nprocess an alert stream, that can be easily be deployed in a local\nmachine or distributed using [Kubernetes](https://kubernetes.io).\n\nFirst developed to process [ZTF data](https://www.ztf.caltech.edu/) it\nis capable to be used for any stream/static data processing pipeline.\n\nInstalling *apf*\n================\n\n*apf* installation can be done with *pip*\n\n``` {.sourceCode .bash}\npip install apf_base\n```\n\nThis will install the *apf* python package and *apf* command line\nscript.\n\n*apf* design\n============\n\n*apf* is based on *steps* conected through [Apache\nKafka](https://kafka.apache.org/) topics.\n\nEach *step* is composed by a `consumer` and is isolated from other\nsteps inside a docker container.\n\nWhen running, the step calls the **execute()** method for each *message* or *message batch*\nconsumed. A step can have multiple producers and databases back-ends\nplugins that can be accessed inside the *execute* method to have a more\ncomplex logic.\n\n\n<p align=\"center\">\n <img src=\"docs/source/_static/images/apf-flow.png\">\n</p>\n\n\nThis generic step greatly reduce the development of each component of\nthe pipeline and make it easier to test each component separately.\n\n\n*apf* features\n==============\n\n- Automatic Metric Sender (KafkaMetrics)\n- Automatic Code Generation (`apf new-step <step_name>`)\n- Multiple Consumer Plugins:\n - Kafka\n - AVRO\n - CSV\n - JSON\n- Producers:\n - Kafka\n - CSV\n- Metrics:\n - Kafka\n\n*apf* Quickstart\n===================\n\nA **quick-start** guide to create a new step can be found [here](https://apf.readthedocs.io/en/latest/new_step.html).\n",
"bugtrack_url": null,
"license": "",
"summary": "ALeRCE Alert Processing Framework.",
"version": "2.5.2",
"project_urls": {
"Documentation": "https://apf.readthedocs.io/en/latest/index.html",
"Github": "https://github.com/alercebroker/APF"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c138feaf618aad398d8d1ca5d5eb8985bcfc35bc7448fb8c5daab4a3fda00a02",
"md5": "e739c3d4f49c472b421b90e8fb912c9c",
"sha256": "eb495838a5159fae030a514f901e4fe028e58c058dcf95bf0b9fc2c1a208343f"
},
"downloads": -1,
"filename": "apf_base-2.5.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e739c3d4f49c472b421b90e8fb912c9c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 27205,
"upload_time": "2023-07-21T16:03:09",
"upload_time_iso_8601": "2023-07-21T16:03:09.828463Z",
"url": "https://files.pythonhosted.org/packages/c1/38/feaf618aad398d8d1ca5d5eb8985bcfc35bc7448fb8c5daab4a3fda00a02/apf_base-2.5.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a10c505c0bd9651fffd695649978f6ef5efe026e982c268662aadda5ba8dc299",
"md5": "e32e616399c64b5236e38cf886f783ca",
"sha256": "c60b80db5793c6ee5499ac5286aaa734221e187813f6c37eb184ebd762685fde"
},
"downloads": -1,
"filename": "apf_base-2.5.2.tar.gz",
"has_sig": false,
"md5_digest": "e32e616399c64b5236e38cf886f783ca",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 18789,
"upload_time": "2023-07-21T16:03:10",
"upload_time_iso_8601": "2023-07-21T16:03:10.847300Z",
"url": "https://files.pythonhosted.org/packages/a1/0c/505c0bd9651fffd695649978f6ef5efe026e982c268662aadda5ba8dc299/apf_base-2.5.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-21 16:03:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "alercebroker",
"github_project": "APF",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "apf-base"
}
