[](https://travis-ci.com/jnhansen/esahub)
[](https://badge.fury.io/py/esahub)
# esahub: Downloading data from ESA scihub
`esahub` provides a simple interface for downloading satellite data from the European Sentinel missions.
It allows multiple downloads to be performed in parallel, from multiple data servers.
The full range of Copernicus data servers are supported, including:
* `https://scihub.copernicus.eu/dhus`
* `https://scihub.copernicus.eu/apihub`
* `https://scihub.copernicus.eu/s3` (guest access)
* `https://s5phub.copernicus.eu/dhus` (guest access)
* `https://tmphub.copernicus.eu/dhus`
* `https://colhub.copernicus.eu/dhus`
* `https://colhub2.copernicus.eu/dhus`
## <a name="setup"></a>Setup
Install `esahub`:
```
$ pip install esahub
```
This will also create a YAML configuration file in `~/.esahub.conf` (unless that file exists) by copying `config.yaml`. The configuration settings in `~/.esahub.conf` have precedence over the settings in `config.yaml`.
You should overwrite the required settings in your `~/.esahub.conf`, especially:
* `GENERAL.DATA_DIR`
For the majority of the data servers you will need to provide your own authentication details in `SERVERS`.
## <a name="test"></a>Testing the installation
The recommended way of running tests is:
```
$ python setup.py test
```
*Note:* Running the tests may take a while as it includes testing live downloads from SciHub (although with very small files).
## <a name="usage"></a>Command Line Usage
```
$ esahub [cmd] [args] ...
```
### Available commands:
| Command | Description
|:-------------|:-----------------------------------------------------------------------------------
| `ls` | Queries SciHub for archives matching the specified query parameters. Prints the total number of files and data size.
| `get` | Queries SciHub like `ls`, but then downloads the files.
| `doctor` | Checks local satellite products for consistency, either by validating the zip/NetCDF format or by comparing to the MD5 checksum from SciHub. Allows to either delete or repair broken files.
### Options
| Option | Argument | Available for | Description
|:-----------------|:------------------------------|:--------------|:------------------
| | <code><SAT></code> | all | Satellite to query, e.g. S1A, S1B, S2A, S2B, S3A
| `-d`, `--dir` | <code><DIR></code> | all | raw data directory (defaults to config `GENERAL.DATA_DIR`)
| `-o`, `--out` | <code><FILE></code> | `ls` | write files to JSON
| `-i`, `--in` | <code><FILE></code> | `get` | read files from JSON
| `-m`, `--mission`| <code><MISSION></code> | `ls`, `get` | e.g. `Sentinel-1`, `Sentinel-2`, `Sentinel-3`
| `-g`, `--geo` | <code><WKT></code> | `ls`, `get` | geospatial location in WKT format
| `--location` | <code><LOCATION></code> | `ls`, `get` | location as defined in config `LOCATIONS`
| `-t`, `--time` | <code><ARG></code> | `ls`, `get` | Supports a variety of datetime string formats.
| `--type` | <code><TYPE></code> | `ls`, `get` | e.g. `GRD`
| `--orbit` | <code><ORBIT></code> | `ls`, `get` | `ASC` or `DESC`
| `--id` | <code><ID></code> | `ls`, `get` | product identifier, may include wildcards (`*`), e.g. `*SDV*`
| `-q`, `--query` | <code><QUERY></code> | `ls`, `get` | custom query for SciHub, e.g. for single archive: `identifier:...`
| `--restart` | | `get` | Force restart incomplete downloads
| `--log` | | all | write log file
| `--quiet` | | all | Suppress terminal output
| `--mode` | <code><MODE></code> | `doctor` | <code>zip|file</code>
| `--delete` | | `doctor` | delete corrupt files
| `--repair` | | `doctor` | redownload corrupt files
| `--email` | | all | send email report
##### Datetime parsing
The following are examples of datetime formats that will be automatically parsed into a date or date range:
The following single dates will be explicitly converted to the date range covering the given year, month, or day:
* `--time 2016`
* `--time 06/2018`
* `--time 2018/06`
* `--time "Sep 1, 2018"`
Date ranges may also be specified explicitly:
* `--time "2016 to 2017"`
* `--time "Jan 2016 - Feb 2016"`
* `--time "01/01/2016, 14/01/2016"`
One-sided date ranges are also possible:
* `--time "to 2017"`
* `--time "01/2017-"`
* `--time "01/12/2017,"`
### Examples
**Ex 1.** Retrieve the number of archives and total file size of Sentinel-3 archives uploaded to SciHub during the past midnight-to-midnight period intersecting Ireland (only works if `Ireland` is defined in the config item `LOCATIONS`):
```
$ esahub ls -t yesterday --location=Ireland
```
**Ex 2.** Download the archives uploaded yesterday for four locations.
```
$ esahub get -t yesterday --location=Ireland_Mace_Head --location=Namibia_Gobabeb --location=Italy_Rome_Tor_Vergata --location=France_La_Crau
```
**Ex 3.** Query SciHub for all available Sentinel-2 data for Ireland and write the result to a JSON file. Then read that JSON file by the `get` command, thus downloading the specified files. _Note:_ Since the JSON file may be edited manually, this approach offers the most flexibility.
```
$ esahub ls --location=Ireland --mission=Sentinel-2 --out=Sen2_IE.json
$ esahub get --in=Sen2_IE.json --log
```
**Ex 4.** Check all zip archives in a custom directory for MD5 consistency and generate a log file.
```
$ esahub doctor --dir=/path/to/dir/ --mode=md5 --log
```
## Python API
```python
from esahub import scihub
query = {'mission': 'Sentinel-1',
'geo': 'POINT(-9.0 53.0)',
'time': 'today'}
files = scihub.search(query)
scihub.download(files)
```
## <a name="dependencies"></a>Dependencies
### Required
* `pyyaml`
* `numpy`
* `lxml`
* `shapely`
* `python-dateutil`
* `pytz`
* `tqdm`
### Optional
* `pyproj`
* `netCDF4`
### Libraries
* `libgeos_c`
Raw data
{
"_id": null,
"home_page": "https://github.com/wegaw/esahub",
"name": "esahub-wegaw",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "esahub,scihub,sentinel,satellite,copernicus",
"author": "Gonzalo Alvarez",
"author_email": "gonzalo.alvarez@wegaw.com",
"download_url": "https://files.pythonhosted.org/packages/5c/65/45a243b5e91af87f9079b763a60cc7aba6c2233bbb78fc3901911bddfa28/esahub-wegaw-0.1.0.tar.gz",
"platform": null,
"description": "[](https://travis-ci.com/jnhansen/esahub)\n[](https://badge.fury.io/py/esahub)\n\n# esahub: Downloading data from ESA scihub\n`esahub` provides a simple interface for downloading satellite data from the European Sentinel missions.\n\nIt allows multiple downloads to be performed in parallel, from multiple data servers.\nThe full range of Copernicus data servers are supported, including:\n* `https://scihub.copernicus.eu/dhus`\n* `https://scihub.copernicus.eu/apihub`\n* `https://scihub.copernicus.eu/s3` (guest access)\n* `https://s5phub.copernicus.eu/dhus` (guest access)\n* `https://tmphub.copernicus.eu/dhus`\n* `https://colhub.copernicus.eu/dhus`\n* `https://colhub2.copernicus.eu/dhus`\n\n\n## <a name=\"setup\"></a>Setup\nInstall `esahub`:\n```\n$ pip install esahub\n```\n\nThis will also create a YAML configuration file in `~/.esahub.conf` (unless that file exists) by copying `config.yaml`. The configuration settings in `~/.esahub.conf` have precedence over the settings in `config.yaml`.\n\nYou should overwrite the required settings in your `~/.esahub.conf`, especially:\n* `GENERAL.DATA_DIR`\n\nFor the majority of the data servers you will need to provide your own authentication details in `SERVERS`.\n\n\n## <a name=\"test\"></a>Testing the installation\nThe recommended way of running tests is:\n```\n$ python setup.py test\n```\n\n*Note:* Running the tests may take a while as it includes testing live downloads from SciHub (although with very small files).\n\n## <a name=\"usage\"></a>Command Line Usage\n```\n$ esahub [cmd] [args] ...\n```\n\n### Available commands:\n\n| Command | Description\n|:-------------|:-----------------------------------------------------------------------------------\n| `ls` | Queries SciHub for archives matching the specified query parameters. Prints the total number of files and data size.\n| `get` | Queries SciHub like `ls`, but then downloads the files.\n| `doctor` | Checks local satellite products for consistency, either by validating the zip/NetCDF format or by comparing to the MD5 checksum from SciHub. Allows to either delete or repair broken files.\n\n\n### Options\n\n| Option | Argument | Available for | Description\n|:-----------------|:------------------------------|:--------------|:------------------\n| | <code><SAT></code> | all | Satellite to query, e.g. S1A, S1B, S2A, S2B, S3A\n| `-d`, `--dir` | <code><DIR></code> | all | raw data directory (defaults to config `GENERAL.DATA_DIR`)\n| `-o`, `--out` | <code><FILE></code> | `ls` | write files to JSON\n| `-i`, `--in` | <code><FILE></code> | `get` | read files from JSON\n| `-m`, `--mission`| <code><MISSION></code> | `ls`, `get` | e.g. `Sentinel-1`, `Sentinel-2`, `Sentinel-3`\n| `-g`, `--geo` | <code><WKT></code> | `ls`, `get` | geospatial location in WKT format\n| `--location` | <code><LOCATION></code> | `ls`, `get` | location as defined in config `LOCATIONS`\n| `-t`, `--time` | <code><ARG></code> | `ls`, `get` | Supports a variety of datetime string formats.\n| `--type` | <code><TYPE></code> | `ls`, `get` | e.g. `GRD`\n| `--orbit` | <code><ORBIT></code> | `ls`, `get` | `ASC` or `DESC`\n| `--id` | <code><ID></code> | `ls`, `get` | product identifier, may include wildcards (`*`), e.g. `*SDV*`\n| `-q`, `--query` | <code><QUERY></code> | `ls`, `get` | custom query for SciHub, e.g. for single archive: `identifier:...`\n| `--restart` | | `get` | Force restart incomplete downloads\n| `--log` | | all | write log file\n| `--quiet` | | all | Suppress terminal output\n| `--mode` | <code><MODE></code> | `doctor` | <code>zip|file</code>\n| `--delete` | | `doctor` | delete corrupt files\n| `--repair` | | `doctor` | redownload corrupt files\n| `--email` | | all | send email report\n\n\n##### Datetime parsing\nThe following are examples of datetime formats that will be automatically parsed into a date or date range:\n\nThe following single dates will be explicitly converted to the date range covering the given year, month, or day:\n* `--time 2016`\n* `--time 06/2018`\n* `--time 2018/06`\n* `--time \"Sep 1, 2018\"`\n\nDate ranges may also be specified explicitly:\n* `--time \"2016 to 2017\"`\n* `--time \"Jan 2016 - Feb 2016\"`\n* `--time \"01/01/2016, 14/01/2016\"`\n\nOne-sided date ranges are also possible:\n* `--time \"to 2017\"`\n* `--time \"01/2017-\"`\n* `--time \"01/12/2017,\"`\n\n\n### Examples\n**Ex 1.** Retrieve the number of archives and total file size of Sentinel-3 archives uploaded to SciHub during the past midnight-to-midnight period intersecting Ireland (only works if `Ireland` is defined in the config item `LOCATIONS`):\n```\n$ esahub ls -t yesterday --location=Ireland\n```\n\n**Ex 2.** Download the archives uploaded yesterday for four locations.\n```\n$ esahub get -t yesterday --location=Ireland_Mace_Head --location=Namibia_Gobabeb --location=Italy_Rome_Tor_Vergata --location=France_La_Crau\n```\n\n**Ex 3.** Query SciHub for all available Sentinel-2 data for Ireland and write the result to a JSON file. Then read that JSON file by the `get` command, thus downloading the specified files. _Note:_ Since the JSON file may be edited manually, this approach offers the most flexibility.\n```\n$ esahub ls --location=Ireland --mission=Sentinel-2 --out=Sen2_IE.json\n$ esahub get --in=Sen2_IE.json --log\n```\n\n**Ex 4.** Check all zip archives in a custom directory for MD5 consistency and generate a log file.\n```\n$ esahub doctor --dir=/path/to/dir/ --mode=md5 --log\n```\n\n\n## Python API\n```python\nfrom esahub import scihub\nquery = {'mission': 'Sentinel-1',\n 'geo': 'POINT(-9.0 53.0)',\n 'time': 'today'}\nfiles = scihub.search(query)\nscihub.download(files)\n```\n\n\n## <a name=\"dependencies\"></a>Dependencies\n\n### Required\n* `pyyaml`\n* `numpy`\n* `lxml`\n* `shapely`\n* `python-dateutil`\n* `pytz`\n* `tqdm`\n\n### Optional\n* `pyproj`\n* `netCDF4`\n\n### Libraries\n* `libgeos_c`\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A Python module for downloading ESA satellite data, based on esahub",
"version": "0.1.0",
"project_urls": {
"Homepage": "https://github.com/wegaw/esahub"
},
"split_keywords": [
"esahub",
"scihub",
"sentinel",
"satellite",
"copernicus"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "5c6545a243b5e91af87f9079b763a60cc7aba6c2233bbb78fc3901911bddfa28",
"md5": "89bb8697d2938976fb5bbc9256536ad8",
"sha256": "eb7b5336e085398199565e65f9020b969ddb326be9637827a94f3dff78857693"
},
"downloads": -1,
"filename": "esahub-wegaw-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "89bb8697d2938976fb5bbc9256536ad8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 35994,
"upload_time": "2023-08-01T09:43:22",
"upload_time_iso_8601": "2023-08-01T09:43:22.949039Z",
"url": "https://files.pythonhosted.org/packages/5c/65/45a243b5e91af87f9079b763a60cc7aba6c2233bbb78fc3901911bddfa28/esahub-wegaw-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-01 09:43:22",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "wegaw",
"github_project": "esahub",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"lcname": "esahub-wegaw"
}
JSON
