- [Summary](#summary)
- [Project Structure](#project-structure)
- [Build and Command](#build-and-command)
- [Config Files](#config-files)
- [`config/py`](#configpy)
## Summary
dspw is a Python library that provides a unified message protocol baseline (MPB) and transport abstraction for ingesting, normalizing, and redistributing financial or journalistic content from heterogeneous online sources such as Wall Street Journal (WSJ), Financial Times (FT), Bloomberg, and others.
It defines:
A normalized message schema for news, feeds, threads, and reports objects, designed to work seamlessly across Kafka and NATS brokers.
A Source metadata class (with UUID, canonical group ID, origin identifiers, etc.) forming the core of the MPB.
Inter-broker serialization utilities to translate and validate message payloads between Kafka and NATS in a unified object format.
Publisher–Subscriber interfaces to streamline ingestion pipelines, transformation logic, and event-driven analytics integration.
A distribution-ready Python package (PEP 621-compliant) supporting build, wheel packaging, and deployment to public or private registries (e.g., PyPI, GitHub Packages, or JFrog Artifactory).
The project can be extended for:
Event-sourcing pipelines and real-time news aggregation systems,
Quantitative research ingestion frameworks,
Sentiment analysis feeds or market intelligence dashboards that rely on unified feed ingestion from multiple third-party sources.
## Project Structure
```
dspw/
├── **init**.py
├── schema/
│ ├── **init**.py
│ └── news_event.py
├── mpb/
│ ├── **init**.py
│ └── source_protocol.py # Message Protocol Baseline (MPB)
├── serializers/
│ ├── **init**.py
│ ├── kafka_serializer.py
│ ├── nats_serializer.py
│ └── utils.py
├── bridge.py
├── config.py
└── pyproject.toml
```
## Build and Command
```sh
# create the distribution wheel and sdist
python3 -m build
# check the wheel structure
twine check dist/*
# upload to test pypi
twine upload --repository dspw dist/*
# install for verification
pip install -i https://test.pypi.org/simple/ dspw
```
## Config Files
### [`config/py`](./config.py)
How to use the config file, E.g.:
```py
from dspw import config
print(config.KAFKA["bootstrap_servers"])
print(config.MPB["serialization_format"])
print(config.SOURCE_REGISTRY["canonical_sources"]["ft"])
```
Raw data
{
"_id": null,
"home_page": null,
"name": "dspw",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "news, message-broker, kafka, nats, event-driven, pubsub, news-ingestion, data-pipeline, real-time-analytics, source-metadata, serialization, schema-normalization, quant-research, feed-parser, python-library, distribution",
"author": null,
"author_email": "Lance M <lance.mk@bitstric.com>",
"download_url": "https://files.pythonhosted.org/packages/05/33/022df5759e59bf51f0ce946d8e43af0ac2a49bebce7db82c417495afaa21/dspw-0.2.7.tar.gz",
"platform": null,
"description": "- [Summary](#summary)\n- [Project Structure](#project-structure)\n- [Build and Command](#build-and-command)\n- [Config Files](#config-files)\n - [`config/py`](#configpy)\n\n## Summary\n\ndspw is a Python library that provides a unified message protocol baseline (MPB) and transport abstraction for ingesting, normalizing, and redistributing financial or journalistic content from heterogeneous online sources such as Wall Street Journal (WSJ), Financial Times (FT), Bloomberg, and others.\n\nIt defines:\n\nA normalized message schema for news, feeds, threads, and reports objects, designed to work seamlessly across Kafka and NATS brokers.\n\nA Source metadata class (with UUID, canonical group ID, origin identifiers, etc.) forming the core of the MPB.\n\nInter-broker serialization utilities to translate and validate message payloads between Kafka and NATS in a unified object format.\n\nPublisher\u2013Subscriber interfaces to streamline ingestion pipelines, transformation logic, and event-driven analytics integration.\n\nA distribution-ready Python package (PEP 621-compliant) supporting build, wheel packaging, and deployment to public or private registries (e.g., PyPI, GitHub Packages, or JFrog Artifactory).\n\nThe project can be extended for:\n\nEvent-sourcing pipelines and real-time news aggregation systems,\n\nQuantitative research ingestion frameworks,\n\nSentiment analysis feeds or market intelligence dashboards that rely on unified feed ingestion from multiple third-party sources.\n\n## Project Structure\n\n```\ndspw/\n\u251c\u2500\u2500 **init**.py\n\u251c\u2500\u2500 schema/\n\u2502 \u251c\u2500\u2500 **init**.py\n\u2502 \u2514\u2500\u2500 news_event.py\n\u251c\u2500\u2500 mpb/\n\u2502 \u251c\u2500\u2500 **init**.py\n\u2502 \u2514\u2500\u2500 source_protocol.py # Message Protocol Baseline (MPB)\n\u251c\u2500\u2500 serializers/\n\u2502 \u251c\u2500\u2500 **init**.py\n\u2502 \u251c\u2500\u2500 kafka_serializer.py\n\u2502 \u251c\u2500\u2500 nats_serializer.py\n\u2502 \u2514\u2500\u2500 utils.py\n\u251c\u2500\u2500 bridge.py\n\u251c\u2500\u2500 config.py\n\u2514\u2500\u2500 pyproject.toml\n```\n\n## Build and Command\n\n```sh\n# create the distribution wheel and sdist\npython3 -m build\n\n# check the wheel structure\ntwine check dist/*\n\n# upload to test pypi\ntwine upload --repository dspw dist/*\n\n# install for verification\npip install -i https://test.pypi.org/simple/ dspw\n\n```\n\n## Config Files\n\n### [`config/py`](./config.py)\n\nHow to use the config file, E.g.:\n\n```py\nfrom dspw import config\n\nprint(config.KAFKA[\"bootstrap_servers\"])\nprint(config.MPB[\"serialization_format\"])\nprint(config.SOURCE_REGISTRY[\"canonical_sources\"][\"ft\"])\n\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Data Stream Protocol Wire. A Python library that provides a unified message protocol baseline (MPB) and transport abstraction for ingesting, normalizing, and redistributing financial or journalistic content from heterogeneous online sources",
"version": "0.2.7",
"project_urls": {
"Homepage": "https://bitbucket.org/xft-software/dspw",
"Issues": "https://bitbucket.org/xft-software/dspw/issues",
"Repository": "https://bitbucket.org/xft-software/dspw"
},
"split_keywords": [
"news",
" message-broker",
" kafka",
" nats",
" event-driven",
" pubsub",
" news-ingestion",
" data-pipeline",
" real-time-analytics",
" source-metadata",
" serialization",
" schema-normalization",
" quant-research",
" feed-parser",
" python-library",
" distribution"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "8edb76e83c2b2dfac3b363eb540249321a343d557ded1cf3f070af03f2da829f",
"md5": "9b9200df1b4c4ff20065f5d104b48228",
"sha256": "227c340d8ad8230284a5c7d488b89b0791be805c66664faebdf4298bbac0d105"
},
"downloads": -1,
"filename": "dspw-0.2.7-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "9b9200df1b4c4ff20065f5d104b48228",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.9",
"size": 5746,
"upload_time": "2025-10-11T07:09:42",
"upload_time_iso_8601": "2025-10-11T07:09:42.509806Z",
"url": "https://files.pythonhosted.org/packages/8e/db/76e83c2b2dfac3b363eb540249321a343d557ded1cf3f070af03f2da829f/dspw-0.2.7-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "0533022df5759e59bf51f0ce946d8e43af0ac2a49bebce7db82c417495afaa21",
"md5": "faeb8b9931f3649f4cec7f8a3c069ea8",
"sha256": "fadba8c7945957f8bfc77de7d873b27b799fcb83cfe197286ff7194263cea9fb"
},
"downloads": -1,
"filename": "dspw-0.2.7.tar.gz",
"has_sig": false,
"md5_digest": "faeb8b9931f3649f4cec7f8a3c069ea8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 5117,
"upload_time": "2025-10-11T07:09:43",
"upload_time_iso_8601": "2025-10-11T07:09:43.831818Z",
"url": "https://files.pythonhosted.org/packages/05/33/022df5759e59bf51f0ce946d8e43af0ac2a49bebce7db82c417495afaa21/dspw-0.2.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-11 07:09:43",
"github": false,
"gitlab": false,
"bitbucket": true,
"codeberg": false,
"bitbucket_user": "xft-software",
"bitbucket_project": "dspw",
"lcname": "dspw"
}
JSON