# dds_py - Data driven software
Data-driven software (python implementation)
## Introduction
The DDS package solves the synchronization problem between code and data. It
allows programmers, scientists and data scientists to integrate code with data
and data with code without fear of stale data, disparate storage frameworks or
concurrency issues. DDS allows quick collaboration and data software reuse
without the complexity. In short, you do not have to think about changes in your
data pipelines.
## How to use
This package is published on PyPI:
```
pip install dds_py
```
This package is known to work on python 3.8, 3.9, 3.10, 3.11. No other versions
are officially supported. Python 3.4 and 3.5 might work but they are not
supported.
**Plotting dependencies** If you want to plot the graph of data dependencies,
you must install separately the `pydotplus` package, which requires `graphviz`
on your system to work properly. Consult the documentation of the `pydotplus`
package for more details. The `pydotplus` package is only required with the
`dds_export_graph` option.
**Databricks users:** If you want to use this package with Databricks, some
specific hooks for Spark are available. See this notebook for a complete
example:
https://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/7816249071411394/4492656151009213/5450662050199348/latest.html
## Documentation
API reference, tutorials and FAQs are located here:
https://tjhunter.github.io/dds_py/
## Example
Here is the Hello world example (using type annotations for clarity)
```python
import dds
import requests
@dds.data_function("/hello_data")
def data() -> str:
url = "https://gist.githubusercontent.com/bigsnarfdude/515849391ad37fe593997fe0db98afaa/raw/f663366d17b7d05de61a145bbce7b2b961b3b07f/weather.csv"
return requests.get(url=url, verify=False).content.decode("utf-8")
data()
```
This example does the following:
- it defines a source of data, here a piece of weather data from the internet.
This source is defined as the function `data_creator`
- it assigns the data produced by this source into a variable (`data`) and also
to a path in a storage system (`/hello_data`)
The DDS library guarantees the following after evaluation of the code:
1. the path `/hello_data` contains a copy of the data returned by
`data_creator`, as if the function `data_creator` had been called at this
moment
2. the function `data_creator` is only evaluated when its inputs, or its code,
are modified (referential transparency)
## License
The `dds` package is published under the Affero General Public License.
Raw data
{
"_id": null,
"home_page": "https://github.com/tjhunter/dds_py",
"name": "dds-py",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.6,<4",
"maintainer_email": "",
"keywords": "",
"author": "Tim Hunter",
"author_email": "tjhunter+dds@cs.stanford.edu",
"download_url": "",
"platform": null,
"description": "# dds_py - Data driven software\n\nData-driven software (python implementation)\n\n## Introduction\n\nThe DDS package solves the synchronization problem between code and data. It\nallows programmers, scientists and data scientists to integrate code with data\nand data with code without fear of stale data, disparate storage frameworks or\nconcurrency issues. DDS allows quick collaboration and data software reuse\nwithout the complexity. In short, you do not have to think about changes in your\ndata pipelines.\n\n## How to use\n\nThis package is published on PyPI:\n\n```\npip install dds_py\n```\n\nThis package is known to work on python 3.8, 3.9, 3.10, 3.11. No other versions\nare officially supported. Python 3.4 and 3.5 might work but they are not\nsupported.\n\n**Plotting dependencies** If you want to plot the graph of data dependencies,\nyou must install separately the `pydotplus` package, which requires `graphviz`\non your system to work properly. Consult the documentation of the `pydotplus`\npackage for more details. The `pydotplus` package is only required with the\n`dds_export_graph` option.\n\n**Databricks users:** If you want to use this package with Databricks, some\nspecific hooks for Spark are available. See this notebook for a complete\nexample:\n\nhttps://databricks-prod-cloudfront.cloud.databricks.com/public/4027ec902e239c93eaaa8714f173bcfc/7816249071411394/4492656151009213/5450662050199348/latest.html\n\n## Documentation\n\nAPI reference, tutorials and FAQs are located here:\nhttps://tjhunter.github.io/dds_py/\n\n## Example\n\nHere is the Hello world example (using type annotations for clarity)\n\n```python\nimport dds\nimport requests \n\n@dds.data_function(\"/hello_data\")\ndef data() -> str:\n url = \"https://gist.githubusercontent.com/bigsnarfdude/515849391ad37fe593997fe0db98afaa/raw/f663366d17b7d05de61a145bbce7b2b961b3b07f/weather.csv\"\n return requests.get(url=url, verify=False).content.decode(\"utf-8\")\n\ndata()\n```\n\nThis example does the following:\n\n- it defines a source of data, here a piece of weather data from the internet.\n This source is defined as the function `data_creator`\n- it assigns the data produced by this source into a variable (`data`) and also\n to a path in a storage system (`/hello_data`)\n\nThe DDS library guarantees the following after evaluation of the code:\n\n1. the path `/hello_data` contains a copy of the data returned by\n `data_creator`, as if the function `data_creator` had been called at this\n moment\n2. the function `data_creator` is only evaluated when its inputs, or its code,\n are modified (referential transparency)\n\n## License\n\nThe `dds` package is published under the Affero General Public License.\n",
"bugtrack_url": null,
"license": "AGPL",
"summary": "PyDDS: data-driven programing",
"version": "0.13.1",
"project_urls": {
"Bug Tracker": "https://github.com/tjhunter/dds_py/issues",
"Documentation": "https://tjhunter.github.io/dds_py/",
"Homepage": "https://github.com/tjhunter/dds_py",
"Source Code": "https://github.com/tjhunter/dds_py"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "32fcba9e9eafc2ee27c5efabc6dd9931859eca199c1c5df0bb89c68b4a8507d5",
"md5": "1cc9bdf4d7abcca50920fd978cc89cf0",
"sha256": "dadf366015938d984af516a7edd8a322373fa91b2fdd037906c2a6a12619f288"
},
"downloads": -1,
"filename": "dds_py-0.13.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1cc9bdf4d7abcca50920fd978cc89cf0",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.6,<4",
"size": 74592,
"upload_time": "2023-10-20T10:12:09",
"upload_time_iso_8601": "2023-10-20T10:12:09.325485Z",
"url": "https://files.pythonhosted.org/packages/32/fc/ba9e9eafc2ee27c5efabc6dd9931859eca199c1c5df0bb89c68b4a8507d5/dds_py-0.13.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-20 10:12:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "tjhunter",
"github_project": "dds_py",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "dds-py"
}
JSON
