Smart Citizen Data
=======
[](https://zenodo.org/badge/latestdoi/97752018)
[](https://mybinder.org/v2/gh/fablabbcn/smartcitizen-data-framework/master?filepath=%2Fexamples%2Fnotebooks)
[](https://badge.fury.io/py/scdata)
[](https://github.com/fablabbcn/smartcitizen-data/actions/workflows/python-multiple-versions.yml)
Welcome to **SmartCitizen Data**. This is a data analysis framework for working with sensor data in different ways:
- Interacting with several sensors APIs
- Clean data, export and calculate metrics
- Model sensor data and calibrate sensors
- Generate data visualisations - matplotlib, ~[plotly](https://plotly.com/)~ or [uplot](https://leeoniya.github.io/uPlot)
- Generate analysis reports in html or pdf and upload them to [Zenodo](http://zenodo.org)
## Installation
You can check it out in the [](https://mybinder.org/v2/gh/fablabbcn/smartcitizen-data-framework/master?filepath=%2Fexamples%2Fnotebooks) before installing if you want. Works with `Python 3.*` (tested between 3.9 and 3.11).
You can just run:
```
pip install scdata
```
### Work on the source code
Simply clone the repository with:
```
git clone https://github.com/fablabbcn/smartcitizen-data.git
cd smartcitizen-data
```
Install `scdata` package with requirements:
```
python setup.py install
```
Or if you want to edit:
```
cd scdata
pip install --editable .
```
### Tokens and config
A configuration file is available at `~/.config/scdata/config.yaml`, which contains a set of configurable variables to allow or not the local storage of relevant data in the data folder, normally in `~/.cache/scdata`:
```
data:
cached_data_margin: 2
load_cached_api: true
reload_metadata: true
store_cached_api: true
paths:
config: /Users/username/.config/scdata
data: /Users/username/.cache/scdata
export: /Users/username/.cache/scdata/export
interim: /Users/username/.cache/scdata/interim
inventory: ''
models: /Users/username/.cache/scdata/models
processed: /Users/username/.cache/scdata/processed
raw: /Users/username/.cache/scdata/raw
reports: /Users/username/.cache/scdata/reports
uploads: /Users/username/.cache/scdata/uploads
zenodo_real_base_url: https://zenodo.org
zenodo_sandbox_base_url: http://sandbox.zenodo.org
```
Also, `.env` files will be picked from `~/.cache/scdata`. An [env.example](env.example) is provided. If you want to upload data to [Zenodo](http://zenodo.org), you will need to fill set an environment variable called `ZENODO_TOKEN` in your environment. The `.env` file is not mandatory for `scdata` to work. It only serves for authentication if you want to `POST` or `GET` data without throttling. You can still do `GET` requests but they will be throttled if not authenticated/authorized by the platform. See [user roles](https://developer.smartcitizen.me/#authentication) for more info.
### Using with jupyter lab (optional)
It can also be used with `jupyter lab` or `jupyter`. For this [install juypterlab](https://github.com/jupyterlab/jupyterlab).
## Contribute
Issues and PR more than welcome!
## Funding
This work has received funding from the European Union's Horizon 2020 research and innovation program under the grant agreement [No. 689954](https://cordis.europa.eu/project/rcn/202639_en.html)
Raw data
{
"_id": null,
"home_page": "https://github.com/fablabbcn/smartcitizen-data",
"name": "scdata",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "air, sensors, Smart Citizen",
"author": "oscgonfer",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/d3/63/e6fd937ff0334c39f3e74cecbe996202adffbd1501ba3eb32dd8f81fc216/scdata-1.3.0.tar.gz",
"platform": null,
"description": "Smart Citizen Data\n=======\n\n[](https://zenodo.org/badge/latestdoi/97752018)\n[](https://mybinder.org/v2/gh/fablabbcn/smartcitizen-data-framework/master?filepath=%2Fexamples%2Fnotebooks)\n[](https://badge.fury.io/py/scdata)\n[](https://github.com/fablabbcn/smartcitizen-data/actions/workflows/python-multiple-versions.yml)\n\nWelcome to **SmartCitizen Data**. This is a data analysis framework for working with sensor data in different ways:\n\n- Interacting with several sensors APIs\n- Clean data, export and calculate metrics\n- Model sensor data and calibrate sensors\n- Generate data visualisations - matplotlib, ~[plotly](https://plotly.com/)~ or [uplot](https://leeoniya.github.io/uPlot)\n- Generate analysis reports in html or pdf and upload them to [Zenodo](http://zenodo.org)\n\n## Installation\n\nYou can check it out in the [](https://mybinder.org/v2/gh/fablabbcn/smartcitizen-data-framework/master?filepath=%2Fexamples%2Fnotebooks) before installing if you want. Works with `Python 3.*` (tested between 3.9 and 3.11).\n\nYou can just run:\n\n```\npip install scdata\n```\n\n### Work on the source code\n\nSimply clone the repository with:\n\n```\ngit clone https://github.com/fablabbcn/smartcitizen-data.git\ncd smartcitizen-data\n```\n\nInstall `scdata` package with requirements:\n\n```\npython setup.py install\n```\n\nOr if you want to edit:\n\n```\ncd scdata\npip install --editable .\n```\n\n### Tokens and config\n\nA configuration file is available at `~/.config/scdata/config.yaml`, which contains a set of configurable variables to allow or not the local storage of relevant data in the data folder, normally in `~/.cache/scdata`:\n\n```\ndata:\n cached_data_margin: 2\n load_cached_api: true\n reload_metadata: true\n store_cached_api: true\npaths:\n config: /Users/username/.config/scdata\n data: /Users/username/.cache/scdata\n export: /Users/username/.cache/scdata/export\n interim: /Users/username/.cache/scdata/interim\n inventory: ''\n models: /Users/username/.cache/scdata/models\n processed: /Users/username/.cache/scdata/processed\n raw: /Users/username/.cache/scdata/raw\n reports: /Users/username/.cache/scdata/reports\n uploads: /Users/username/.cache/scdata/uploads\nzenodo_real_base_url: https://zenodo.org\nzenodo_sandbox_base_url: http://sandbox.zenodo.org\n```\n\nAlso, `.env` files will be picked from `~/.cache/scdata`. An [env.example](env.example) is provided. If you want to upload data to [Zenodo](http://zenodo.org), you will need to fill set an environment variable called `ZENODO_TOKEN` in your environment. The `.env` file is not mandatory for `scdata` to work. It only serves for authentication if you want to `POST` or `GET` data without throttling. You can still do `GET` requests but they will be throttled if not authenticated/authorized by the platform. See [user roles](https://developer.smartcitizen.me/#authentication) for more info.\n\n### Using with jupyter lab (optional)\n\nIt can also be used with `jupyter lab` or `jupyter`. For this [install juypterlab](https://github.com/jupyterlab/jupyterlab).\n\n## Contribute\n\nIssues and PR more than welcome!\n\n## Funding\n\nThis work has received funding from the European Union's Horizon 2020 research and innovation program under the grant agreement [No. 689954](https://cordis.europa.eu/project/rcn/202639_en.html)\n",
"bugtrack_url": null,
"license": "GNU-GPL3.0",
"summary": "Analysis of sensors and time series data",
"version": "1.3.0",
"project_urls": {
"Documentation": "https://docs.smartcitizen.me/",
"Homepage": "https://github.com/fablabbcn/smartcitizen-data",
"Source Code": "https://github.com/fablabbcn/smartcitizen-data"
},
"split_keywords": [
"air",
" sensors",
" smart citizen"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "d9cb08a6bf1271cba13dc134be113ae5bd8ce09eaa08ef6e7a2f02b2d13d55ef",
"md5": "d41c72774a70870e0314e605994a9c0b",
"sha256": "09e14007c3a40dd26cfc7d5246d50e1ffbec7bbe9a234ed1c40525a814c7c108"
},
"downloads": -1,
"filename": "scdata-1.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "d41c72774a70870e0314e605994a9c0b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 333627,
"upload_time": "2025-10-18T09:06:26",
"upload_time_iso_8601": "2025-10-18T09:06:26.489491Z",
"url": "https://files.pythonhosted.org/packages/d9/cb/08a6bf1271cba13dc134be113ae5bd8ce09eaa08ef6e7a2f02b2d13d55ef/scdata-1.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "d363e6fd937ff0334c39f3e74cecbe996202adffbd1501ba3eb32dd8f81fc216",
"md5": "2d553848bcd2df5dd504481238e70b8c",
"sha256": "bd1aa9b0f9653b475c38a987c8ec5d5e3ee3a9fced79840bd4213fcebb68e92d"
},
"downloads": -1,
"filename": "scdata-1.3.0.tar.gz",
"has_sig": false,
"md5_digest": "2d553848bcd2df5dd504481238e70b8c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 310024,
"upload_time": "2025-10-18T09:06:28",
"upload_time_iso_8601": "2025-10-18T09:06:28.054044Z",
"url": "https://files.pythonhosted.org/packages/d3/63/e6fd937ff0334c39f3e74cecbe996202adffbd1501ba3eb32dd8f81fc216/scdata-1.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-10-18 09:06:28",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "fablabbcn",
"github_project": "smartcitizen-data",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "branca",
"specs": [
[
"~=",
"0.4.0"
]
]
},
{
"name": "Flask",
"specs": [
[
"~=",
"2.2.2"
]
]
},
{
"name": "folium",
"specs": [
[
"~=",
"0.12.1"
]
]
},
{
"name": "geopy",
"specs": [
[
"~=",
"1.21.0"
]
]
},
{
"name": "Jinja2",
"specs": [
[
"~=",
"3.1.2"
]
]
},
{
"name": "matplotlib",
"specs": []
},
{
"name": "pandas",
"specs": [
[
"~=",
"2.2.2"
]
]
},
{
"name": "pydantic",
"specs": []
},
{
"name": "pytest",
"specs": []
},
{
"name": "PyYAML",
"specs": [
[
"~=",
"6.0.1"
]
]
},
{
"name": "requests",
"specs": []
},
{
"name": "scipy",
"specs": []
},
{
"name": "scikit-learn",
"specs": []
},
{
"name": "seaborn",
"specs": []
},
{
"name": "smartcitizen-connector",
"specs": []
},
{
"name": "termcolor",
"specs": [
[
"==",
"1.1.0"
]
]
},
{
"name": "tqdm",
"specs": [
[
"~=",
"4.50.2"
]
]
},
{
"name": "timezonefinder",
"specs": [
[
"~=",
"6.1.9"
]
]
},
{
"name": "urllib3",
"specs": []
},
{
"name": "boto3",
"specs": []
},
{
"name": "awswrangler",
"specs": []
}
],
"lcname": "scdata"
}
JSON
