The `harvest_basket` extension comes with a few custom **harvesters** for different data portals.
The list of available harvesters, that must be enabled with `ckan.plugins` in your config:
- `dkan_harvester`
- `junar_harvester`
- `socrata_harvester`
- `arcgis_harvester`
This extension also adds some features to extend the basic harvester:
1. Source checkup preview. During the source creation stage the harvester will try to access the remote portal and harvest one dataset to check if it's accessible or not. (TODO: the source checkup interface will be documented soon).
2. Restriction for anonymous users to visit harveser pages
## Installation
To install ckanext-harvest-basket:
1. Activate your CKAN virtual environment, for example:
. /usr/lib/ckan/default/bin/activate
2. Clone the source and install it on the virtualenv
git clone https://github.com/mutantsan/ckanext-harvest-basket.git
cd ckanext-harvest-basket
pip install -e .
pip install -r requirements.txt
3. Add `harvest_basket` to the `ckan.plugins` setting in your CKAN
config file (by default the config file is located at
`/etc/ckan/default/ckan.ini`).
4. Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:
sudo service apache2 reload
## Config settings
Available config options:
# You can disallow accessing harvester pages for anonymous users.
# (optional, default: 1).
ckanext.harvest_basket.allow_anonymous = 0
## Developer installation
To install ckanext-harvest-basket for development, activate your CKAN virtualenv and
do:
git clone https://github.com/mutantsan/ckanext-harvest-basket.git
cd ckanext-harvest-basket
python setup.py develop
pip install -r requirements.txt
## Tests
To run the tests, do:
pytest --ckan-ini=test.ini
## License
[AGPL](https://www.gnu.org/licenses/agpl-3.0.en.html)
Raw data
{
"_id": null,
"home_page": "https://github.com/mutantsan/ckanext-harvest-basket",
"name": "ckanext-harvest-basket",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "CKAN harvest harvester",
"author": "Alexandr Cherniavskyi",
"author_email": "mutantsan@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/00/20/27cea62ab80425c16a96fa602cb08d62ed5edc1a6189a793624d10af804e/ckanext-harvest-basket-1.3.6.tar.gz",
"platform": null,
"description": "The `harvest_basket` extension comes with a few custom **harvesters** for different data portals.\nThe list of available harvesters, that must be enabled with `ckan.plugins` in your config:\n- `dkan_harvester`\n- `junar_harvester`\n- `socrata_harvester`\n- `arcgis_harvester`\n\nThis extension also adds some features to extend the basic harvester:\n1. Source checkup preview. During the source creation stage the harvester will try to access the remote portal and harvest one dataset to check if it's accessible or not. (TODO: the source checkup interface will be documented soon).\n2. Restriction for anonymous users to visit harveser pages\n\n## Installation\n\n\nTo install ckanext-harvest-basket:\n\n1. Activate your CKAN virtual environment, for example:\n\n . /usr/lib/ckan/default/bin/activate\n\n2. Clone the source and install it on the virtualenv\n\n git clone https://github.com/mutantsan/ckanext-harvest-basket.git\n cd ckanext-harvest-basket\n pip install -e .\n\tpip install -r requirements.txt\n\n3. Add `harvest_basket` to the `ckan.plugins` setting in your CKAN\n config file (by default the config file is located at\n `/etc/ckan/default/ckan.ini`).\n\n4. Restart CKAN. For example if you've deployed CKAN with Apache on Ubuntu:\n\n sudo service apache2 reload\n\n\n## Config settings\nAvailable config options:\n\n\t# You can disallow accessing harvester pages for anonymous users.\n\t# (optional, default: 1).\n\tckanext.harvest_basket.allow_anonymous = 0\n\n\n## Developer installation\n\nTo install ckanext-harvest-basket for development, activate your CKAN virtualenv and\ndo:\n\n git clone https://github.com/mutantsan/ckanext-harvest-basket.git\n cd ckanext-harvest-basket\n python setup.py develop\n pip install -r requirements.txt\n\n\n## Tests\n\nTo run the tests, do:\n\n pytest --ckan-ini=test.ini\n\n## License\n\n[AGPL](https://www.gnu.org/licenses/agpl-3.0.en.html)\n",
"bugtrack_url": null,
"license": "AGPL",
"summary": "A bunch of custom harvesters for ckanext-harvest",
"version": "1.3.6",
"split_keywords": [
"ckan",
"harvest",
"harvester"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "9ff3c97218eb4a5674df2d01a2dc945549861cbb8dd1bc4693238a2f10a3ee75",
"md5": "b2545d5ebb6ca0b9f88f0f18945bf3e5",
"sha256": "9d45353ca0d97e3312338b7079f21f6be0a5b6e022b4b0501c4700525518e6f8"
},
"downloads": -1,
"filename": "ckanext_harvest_basket-1.3.6-py3-none-any.whl",
"has_sig": false,
"md5_digest": "b2545d5ebb6ca0b9f88f0f18945bf3e5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 40995,
"upload_time": "2023-04-27T13:26:12",
"upload_time_iso_8601": "2023-04-27T13:26:12.522157Z",
"url": "https://files.pythonhosted.org/packages/9f/f3/c97218eb4a5674df2d01a2dc945549861cbb8dd1bc4693238a2f10a3ee75/ckanext_harvest_basket-1.3.6-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "002027cea62ab80425c16a96fa602cb08d62ed5edc1a6189a793624d10af804e",
"md5": "5bc196891bbc496d564eba069a3bb190",
"sha256": "277741fce1e8d49fcdad0961465b978c22a999d4166bf1ff72b92dd09e9e7c71"
},
"downloads": -1,
"filename": "ckanext-harvest-basket-1.3.6.tar.gz",
"has_sig": false,
"md5_digest": "5bc196891bbc496d564eba069a3bb190",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 33375,
"upload_time": "2023-04-27T13:26:14",
"upload_time_iso_8601": "2023-04-27T13:26:14.178113Z",
"url": "https://files.pythonhosted.org/packages/00/20/27cea62ab80425c16a96fa602cb08d62ed5edc1a6189a793624d10af804e/ckanext-harvest-basket-1.3.6.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-04-27 13:26:14",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "mutantsan",
"github_project": "ckanext-harvest-basket",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [
{
"name": "html2markdown",
"specs": []
},
{
"name": "geojson",
"specs": [
[
"~=",
"2.5.0"
]
]
},
{
"name": "jsonschema",
"specs": []
},
{
"name": "ckanext-transmute",
"specs": [
[
"~=",
"1.4.7"
]
]
}
],
"lcname": "ckanext-harvest-basket"
}
JSON
