FTPsubsetMO


NameFTPsubsetMO JSON
Version 0.5.5 PyPI version JSON
download
home_pagehttps://github.com/carmelosammarco/FTPsubsetMO
SummaryPython module able to download a file from FTP and subset it using time-range,bounding-box,variables and depths
upload_time2020-06-11 08:17:17
maintainer
docs_urlNone
authorCarmelo Sammarco
requires_python>=3
licensegpl-3.0
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI
coveralls test coverage No coveralls.
            # FTPsubsetMO - FTP subsetting service 

[![Build Status](https://travis-ci.com/carmelosammarco/FTPsubsetMO.png)](https://travis-ci.com/carmelosammarco/FTPsubsetMO) [![Build status](https://ci.appveyor.com/api/projects/status/y4glc7d7ccjb8diq?svg=true)](https://ci.appveyor.com/project/carmelosammarco/FTPsubsetMO) [![PyPi](https://img.shields.io/badge/PyPi-Project-yellow.svg)](https://pypi.org/project/FTPsubsetMO/) 

**I developed this software because motivated to improve my efficiency and productivity. It is just an attemp/idea/prototype and it is not fully optimased or considered stable.**

**Carmelo Sammarco**


## Be aware that:

The tool is in development so it can be possible find bugs, errors and imprecisions. Please to report them if you find one.

In the Database just the MY datasets are avaiable (still missing NRT datasets).

For more detailed information about the MULTI YEAR datasets please to look the [MY_datasets](FTPsubsetMO/Database/datasets_MY.pdf) file.

## Introduction:

Python software able to download files over FTP protocol and subset the files retrieved by parameters as time-range, bounding box, variables and single/range Depth levels (below the GUI interface displayed on a Linux system).

<p align="center">
   <img width="400" height="" src="FTPsubsetMO/IMAGES/GUI.png">
</p>

Together with this tool is distribuited a database which store all the information needed to download the files from each datasets (type of data-set (NRT/MY), time steps (DAILY/MONTLY) and other two parameters needed to correctly identify and select the files prior the download. The key value to retrive such information is the FTP URL of the targeted dataset. It was ideated and implemented by me to adress the download automatization. However I really push the Producers to uniform all the data_structure/file_names and Metadata info (which will make easier the database creation/updates).

After the download which uses the python module "ftplib" (The files are downloaded in the same directory where the tool run) all the subsequent analyses are mainly performed with xarray (another python module). Below the full list of dependencies required (They are installed automatically during the installation if not avaiable in the python environment used):

- [x] xarray
- [x] ftputil>=3.4
- [x] netCDF4
- [x] pandas
- [x] datetime
- [x] os
- [x] json
- [x] hdf5
- [x] h5py
- [x] h5netcdf

## What the user needs to input?

1. **CMEMS personal login credential**

- Username
- Password

2. **FTP Link of the dataset** (Our key value to extract from the data-base all the parameters needed to make the Tool works) as example below:

```
/Core/GLOBAL_REANALYSIS_PHY_001_025/global-reanalysis-phy-001-025-monthly/
```

3. **Time range**

- Date start
- Date end

Date format as YYYY-MM-DD also in the case of the MONTHLY dataset where the term "DD" can be set to any real value.


4. **Geographic bounding box** (if interested to subset by geographic area)

5. **Variables name** (if interested in extract a selection rather than all)

6. **Depths** information parameter values (if interested in a SINGLE/RANGE  or all the depths)


## Python module/script installation:

Just type in the terminal/command-prompt:

```
pip install FTPsubsetMO
```

After that you are free to decide if you want to use the GUI interface or exexute it as script. The latter will allow the maximum customization but as cons it is not very user friendly. 

-----

**To use the GUI interface**  just type on terminal/command-prompt:

```
FTPsubsetMO
```

After that just type on the pop upped screen the parameters asked/required. Finally just click on the red download button to start the download/subsetting process.

-----

**To use the program as a script** and then be able to be free in look/modify/customise the code please to:

1. Open the Terminal/command_prompt in the location where you desire download the files or anyway have the script

2. Activate your python environment and import the module:

```
from FTPsubsetMO import script
```
3. Run the function "script" as follow: 

```
script()
```

The above function will allow you to add, in the path folder where you run the command, the files needed  to run the subsetting process in a pure scripting way. "FTPsubsetMO.py" is the only file to modify based on your data request needs. The script's inputs are highlighted with **""**. More information can be found as form of comments in FTPsubsetMO.py script.
            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/carmelosammarco/FTPsubsetMO",
    "name": "FTPsubsetMO",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3",
    "maintainer_email": "",
    "keywords": "",
    "author": "Carmelo Sammarco",
    "author_email": "sammarcocarmelo@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/ee/30/9570b0426ae0db96d073ad8bcdc0b376d9ebfccedb8b5481c9c996185718/FTPsubsetMO-0.5.5.tar.gz",
    "platform": "OS Independent",
    "description": "# FTPsubsetMO - FTP subsetting service \n\n[![Build Status](https://travis-ci.com/carmelosammarco/FTPsubsetMO.png)](https://travis-ci.com/carmelosammarco/FTPsubsetMO) [![Build status](https://ci.appveyor.com/api/projects/status/y4glc7d7ccjb8diq?svg=true)](https://ci.appveyor.com/project/carmelosammarco/FTPsubsetMO) [![PyPi](https://img.shields.io/badge/PyPi-Project-yellow.svg)](https://pypi.org/project/FTPsubsetMO/) \n\n**I developed this software because motivated to improve my efficiency and productivity. It is just an attemp/idea/prototype and it is not fully optimased or considered stable.**\n\n**Carmelo Sammarco**\n\n\n## Be aware that:\n\nThe tool is in development so it can be possible find bugs, errors and imprecisions. Please to report them if you find one.\n\nIn the Database just the MY datasets are avaiable (still missing NRT datasets).\n\nFor more detailed information about the MULTI YEAR datasets please to look the [MY_datasets](FTPsubsetMO/Database/datasets_MY.pdf) file.\n\n## Introduction:\n\nPython software able to download files over FTP protocol and subset the files retrieved by parameters as time-range, bounding box, variables and single/range Depth levels (below the GUI interface displayed on a Linux system).\n\n<p align=\"center\">\n   <img width=\"400\" height=\"\" src=\"FTPsubsetMO/IMAGES/GUI.png\">\n</p>\n\nTogether with this tool is distribuited a database which store all the information needed to download the files from each datasets (type of data-set (NRT/MY), time steps (DAILY/MONTLY) and other two parameters needed to correctly identify and select the files prior the download. The key value to retrive such information is the FTP URL of the targeted dataset. It was ideated and implemented by me to adress the download automatization. However I really push the Producers to uniform all the data_structure/file_names and Metadata info (which will make easier the database creation/updates).\n\nAfter the download which uses the python module \"ftplib\" (The files are downloaded in the same directory where the tool run) all the subsequent analyses are mainly performed with xarray (another python module). Below the full list of dependencies required (They are installed automatically during the installation if not avaiable in the python environment used):\n\n- [x] xarray\n- [x] ftputil>=3.4\n- [x] netCDF4\n- [x] pandas\n- [x] datetime\n- [x] os\n- [x] json\n- [x] hdf5\n- [x] h5py\n- [x] h5netcdf\n\n## What the user needs to input?\n\n1. **CMEMS personal login credential**\n\n- Username\n- Password\n\n2. **FTP Link of the dataset** (Our key value to extract from the data-base all the parameters needed to make the Tool works) as example below:\n\n```\n/Core/GLOBAL_REANALYSIS_PHY_001_025/global-reanalysis-phy-001-025-monthly/\n```\n\n3. **Time range**\n\n- Date start\n- Date end\n\nDate format as YYYY-MM-DD also in the case of the MONTHLY dataset where the term \"DD\" can be set to any real value.\n\n\n4. **Geographic bounding box** (if interested to subset by geographic area)\n\n5. **Variables name** (if interested in extract a selection rather than all)\n\n6. **Depths** information parameter values (if interested in a SINGLE/RANGE  or all the depths)\n\n\n## Python module/script installation:\n\nJust type in the terminal/command-prompt:\n\n```\npip install FTPsubsetMO\n```\n\nAfter that you are free to decide if you want to use the GUI interface or exexute it as script. The latter will allow the maximum customization but as cons it is not very user friendly. \n\n-----\n\n**To use the GUI interface**  just type on terminal/command-prompt:\n\n```\nFTPsubsetMO\n```\n\nAfter that just type on the pop upped screen the parameters asked/required. Finally just click on the red download button to start the download/subsetting process.\n\n-----\n\n**To use the program as a script** and then be able to be free in look/modify/customise the code please to:\n\n1. Open the Terminal/command_prompt in the location where you desire download the files or anyway have the script\n\n2. Activate your python environment and import the module:\n\n```\nfrom FTPsubsetMO import script\n```\n3. Run the function \"script\" as follow: \n\n```\nscript()\n```\n\nThe above function will allow you to add, in the path folder where you run the command, the files needed  to run the subsetting process in a pure scripting way. \"FTPsubsetMO.py\" is the only file to modify based on your data request needs. The script's inputs are highlighted with **\"\"**. More information can be found as form of comments in FTPsubsetMO.py script.",
    "bugtrack_url": null,
    "license": "gpl-3.0",
    "summary": "Python module able to download a file from FTP and subset it using time-range,bounding-box,variables and depths",
    "version": "0.5.5",
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "78c854de66dae7d2c2a1b21fea3fce91",
                "sha256": "600bd37b53a72982d5088e4ca828760639747d4bc09111c5122cbb083706a366"
            },
            "downloads": -1,
            "filename": "FTPsubsetMO-0.5.5.tar.gz",
            "has_sig": false,
            "md5_digest": "78c854de66dae7d2c2a1b21fea3fce91",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3",
            "size": 8818,
            "upload_time": "2020-06-11T08:17:17",
            "upload_time_iso_8601": "2020-06-11T08:17:17.481177Z",
            "url": "https://files.pythonhosted.org/packages/ee/30/9570b0426ae0db96d073ad8bcdc0b376d9ebfccedb8b5481c9c996185718/FTPsubsetMO-0.5.5.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2020-06-11 08:17:17",
    "github": true,
    "github_user": "carmelosammarco",
    "github_project": "FTPsubsetMO",
    "travis_ci": true,
    "coveralls": false,
    "lcname": "ftpsubsetmo"
}
        
Elapsed time: 0.10963s