<div align="center">
<img src="https://github.com/AntoinePinto/easyenvi/blob/master/img/logo.png?raw=true"
raw=true" alt="drawing" width="400"/>><br>
</div>
# Easy environment : easy-to-use Python environment management toolkit
**Easy Environment** is a Python tool that provides **easy-to-use functionality for managing files and data in different environments.** It offers a class that simplifies file operations on the local disk and cloud services such as Google Cloud (Google Cloud Storage and Big Query) or SharePoint.
## Features
* **Multi-format loading and saving**: Load and save files in various formats with one command line
* **Default supported formats**: csv, docx, jpg, json, md, parquet, pdf, pickle, png, pptx, sql, toml, txt, xlsx, xml, yaml, yml
* **Unsupported formats**: Customisable. See [Customise supported formats](https://antoinepinto.gitbook.io/easyenvi/extra/customise-supported-formats).
* **Multi-environment management**:
* **Local disk**: Loading/saving and management.
* **Google Cloud Storage**: Loading/saving and management.
* **Big Query**: Append, write, and run queries on Big Query tables.
* **SharePoint**: Download, upload, and manage files on SharePoint.
<p align="center">
<img src="https://github.com/AntoinePinto/easyenvi/blob/master/img/table_support.png?raw=true" alt="drawing" width="800"/>
</p>
Start by installing `easyenvi` :
```python
pip install easyenvi==1.0.7
```
## Multi-format loading and saving
Load or save a large variety of format : csv, docx, jpg, json, md, parquet, pdf, pickle, png, pptx, sql, toml, txt, xlsx, xml, yaml, yml
```python
from easyenvi import file
secrets = file.load('my_path/secrets.toml')
config = file.load('my_path/config.json')
query = file.load('my_path/titanic.sql')
file.save(df, 'my_path/titanic.csv')
file.save(df, 'my_path/my_dict.parquet')
file.save(my_dict, 'my_path/my_dict.pickle')
```
## Multi-environment management
To use Easy Environment, create an instance of the `EasyEnvironment` class. All the parameters in the `EasyEnvironment` class are optional: it depends on which environment you need to access.
```python
from easyenvi import EasyEnvironment
envi = EasyEnvironment(
local_path="", # Optional
gcloud_project_id="your-project-id", # Optional
gcloud_credential_path="path/to/credentials.json", # Optional
GCS_path="gs://your-bucket-name/", # Optional
sharepoint_site_url="https://{tenant}.sharepoint.com/sites/{site}", # Optional
sharepoint_username="your-username", # Optional
sharepoint_user_password="your-password" # Optional
)
```
Specifying certain parameters means certain dependencies:
* For using **local operation**, `local_path` is the path from which local operations should be executed - specify an empty string if you want to use the current directory.
* For using **Google Cloud**, it is necessary to specify the project ID, the path to a credential .json file, and, in case of interaction with Google Cloud Storage, the path to the GCS folder (see [Google Cloud Initialisation](https://antoinepinto.gitbook.io/easyenvi/google-cloud-environment/google-cloud-initialisation)).
* For using **SharePoint**, it is necessary to specify the SharePoint site to interact with, as well as authentication credentials: either the client_id/client_secret pair or the username/user_password pair (see [SharePoint Initialisation](https://antoinepinto.gitbook.io/easyenvi/sharepoint-environment/sharepoint-initialisation)).
## Examples of use
### Local features
```python
# Load any file format
my_dict = envi.local.load(path='inputs/my_dictionnary.pickle')
my_logo = envi.local.load(path='inputs/my_logo.png')
dataset = envi.local.load(path='inputs/dataset.csv')
# Save any file format
envi.local.save(obj=my_dict, path='outputs/my_dictionnary.pickle')
envi.local.save(obj=my_logo, path='outputs/my_logo.png')
envi.local.save(obj=dataset, path='outputs/dataset.csv')
```
### Google Cloud Storage features
```python
# Load any file format
my_dict = envi.gcloud.GCS.load(path='inputs/my_dictionnary.pickle')
my_logo = envi.gcloud.GCS.load(path='inputs/my_logo.png')
dataset = envi.gcloud.GCS.load(path='inputs/dataset.csv')
# Save any file format
envi.gcloud.GCS.save(obj=my_dict, path='outputs/my_dictionnary.pickle')
envi.gcloud.GCS.save(obj=my_logo, path='outputs/my_logo.png')
envi.gcloud.GCS.save(obj=dataset, path='outputs/dataset.csv')
```
### Big Query features
```python
df = pd.DataFrame(data={'age': [21, 52, 30], 'wage': [12, 17, 11]})
# Create a new table
envi.gcloud.BQ.write(dataset, 'mydata.mytable')
# Append an existing table
envi.gcloud.BQ.append(dataset, 'mydata.mytable')
# Run queries
query = """
SELECT *
FROM mydata.mytable
WHERE age < 40
"""
new_dataset = envi.gcloud.BQ.query(query).to_dataframe()
```
### SharePoint features
```python
# Download a file
envi.sharepoint.download(input_path="/Document partages/sharepoint_folder/my_file.txt",
output_path="local_folder/my_file.txt")
# Upload a file
envi.sharepoint.upload(input_path="local_folder/my_file.txt",
output_path="Document partages/folder/my_file.txt")
# List files
envi.sharepoint.list_files(folder="local_folder")
```
## Documentation
The documentation is available here : [Easy Environment - Documentation](https://antoinepinto.gitbook.io/easyenvi/)
## Credits
* Thanks to [Herve Mignot](https://github.com/HerveMignot) for his advice on using `fsspec`.
* Thanks to [Nizar Fawal](https://www.linkedin.com/in/nizar-fawal/) for encouraging me to deploy this package as a Pypi library.
* Thanks to [Julien Lambert](https://www.linkedin.com/in/julien--lambert/) for the code snippets and feedback.
## Future Improvements
Future releases of Easy Environment will include support for additional cloud storage providers, including Amazon Web Services (AWS) and Microsoft Azure.
Raw data
{
"_id": null,
"home_page": "https://github.com/AntoinePinto/easyenvi",
"name": "easyenvi",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "file-manager,easy-to-use,environment,cloud,gcp,sharepoint",
"author": "Antoine PINTO",
"author_email": "antoine.pinto1@outlook.fr",
"download_url": "https://files.pythonhosted.org/packages/13/40/4135ee63c7c64990457bc08d0c81dd3fa3dda857ed1f097b07c1a95064af/easyenvi-1.0.7.tar.gz",
"platform": null,
"description": "<div align=\"center\">\r\n <img src=\"https://github.com/AntoinePinto/easyenvi/blob/master/img/logo.png?raw=true\"\r\n raw=true\" alt=\"drawing\" width=\"400\"/>><br>\r\n</div>\r\n\r\n# Easy environment : easy-to-use Python environment management toolkit\r\n\r\n**Easy Environment** is a Python tool that provides **easy-to-use functionality for managing files and data in different environments.** It offers a class that simplifies file operations on the local disk and cloud services such as Google Cloud (Google Cloud Storage and Big Query) or SharePoint.\r\n\r\n## Features\r\n\r\n* **Multi-format loading and saving**: Load and save files in various formats with one command line\r\n * **Default supported formats**: csv, docx, jpg, json, md, parquet, pdf, pickle, png, pptx, sql, toml, txt, xlsx, xml, yaml, yml\r\n * **Unsupported formats**: Customisable. See [Customise supported formats](https://antoinepinto.gitbook.io/easyenvi/extra/customise-supported-formats).\r\n* **Multi-environment management**:\r\n * **Local disk**: Loading/saving and management.\r\n * **Google Cloud Storage**: Loading/saving and management.\r\n * **Big Query**: Append, write, and run queries on Big Query tables.\r\n * **SharePoint**: Download, upload, and manage files on SharePoint.\r\n\r\n<p align=\"center\">\r\n <img src=\"https://github.com/AntoinePinto/easyenvi/blob/master/img/table_support.png?raw=true\" alt=\"drawing\" width=\"800\"/>\r\n</p>\r\n\r\nStart by installing `easyenvi` :\r\n\r\n```python\r\npip install easyenvi==1.0.7\r\n```\r\n\r\n## Multi-format loading and saving\r\n\r\nLoad or save a large variety of format : csv, docx, jpg, json, md, parquet, pdf, pickle, png, pptx, sql, toml, txt, xlsx, xml, yaml, yml\r\n\r\n```python\r\nfrom easyenvi import file\r\n\r\nsecrets = file.load('my_path/secrets.toml')\r\nconfig = file.load('my_path/config.json')\r\nquery = file.load('my_path/titanic.sql')\r\n\r\nfile.save(df, 'my_path/titanic.csv')\r\nfile.save(df, 'my_path/my_dict.parquet')\r\nfile.save(my_dict, 'my_path/my_dict.pickle')\r\n```\r\n\r\n## Multi-environment management\r\n\r\nTo use Easy Environment, create an instance of the `EasyEnvironment` class. All the parameters in the `EasyEnvironment` class are optional: it depends on which environment you need to access.\r\n\r\n```python\r\nfrom easyenvi import EasyEnvironment\r\n\r\nenvi = EasyEnvironment(\r\n local_path=\"\", # Optional\r\n\r\n gcloud_project_id=\"your-project-id\", # Optional\r\n gcloud_credential_path=\"path/to/credentials.json\", # Optional\r\n GCS_path=\"gs://your-bucket-name/\", # Optional\r\n\r\n sharepoint_site_url=\"https://{tenant}.sharepoint.com/sites/{site}\", # Optional\r\n sharepoint_username=\"your-username\", # Optional\r\n sharepoint_user_password=\"your-password\" # Optional\r\n )\r\n```\r\n\r\nSpecifying certain parameters means certain dependencies: \r\n* For using **local operation**, `local_path` is the path from which local operations should be executed - specify an empty string if you want to use the current directory.\r\n* For using **Google Cloud**, it is necessary to specify the project ID, the path to a credential .json file, and, in case of interaction with Google Cloud Storage, the path to the GCS folder (see [Google Cloud Initialisation](https://antoinepinto.gitbook.io/easyenvi/google-cloud-environment/google-cloud-initialisation)). \r\n* For using **SharePoint**, it is necessary to specify the SharePoint site to interact with, as well as authentication credentials: either the client_id/client_secret pair or the username/user_password pair (see [SharePoint Initialisation](https://antoinepinto.gitbook.io/easyenvi/sharepoint-environment/sharepoint-initialisation)).\r\n\r\n## Examples of use\r\n\r\n### Local features\r\n\r\n```python\r\n# Load any file format\r\nmy_dict = envi.local.load(path='inputs/my_dictionnary.pickle')\r\nmy_logo = envi.local.load(path='inputs/my_logo.png')\r\ndataset = envi.local.load(path='inputs/dataset.csv')\r\n\r\n# Save any file format\r\nenvi.local.save(obj=my_dict, path='outputs/my_dictionnary.pickle')\r\nenvi.local.save(obj=my_logo, path='outputs/my_logo.png')\r\nenvi.local.save(obj=dataset, path='outputs/dataset.csv')\r\n```\r\n\r\n### Google Cloud Storage features\r\n\r\n```python\r\n# Load any file format\r\nmy_dict = envi.gcloud.GCS.load(path='inputs/my_dictionnary.pickle')\r\nmy_logo = envi.gcloud.GCS.load(path='inputs/my_logo.png')\r\ndataset = envi.gcloud.GCS.load(path='inputs/dataset.csv')\r\n\r\n# Save any file format\r\nenvi.gcloud.GCS.save(obj=my_dict, path='outputs/my_dictionnary.pickle')\r\nenvi.gcloud.GCS.save(obj=my_logo, path='outputs/my_logo.png')\r\nenvi.gcloud.GCS.save(obj=dataset, path='outputs/dataset.csv')\r\n```\r\n\r\n### Big Query features\r\n\r\n```python\r\ndf = pd.DataFrame(data={'age': [21, 52, 30], 'wage': [12, 17, 11]})\r\n\r\n# Create a new table\r\nenvi.gcloud.BQ.write(dataset, 'mydata.mytable')\r\n\r\n# Append an existing table\r\nenvi.gcloud.BQ.append(dataset, 'mydata.mytable')\r\n\r\n# Run queries\r\nquery = \"\"\"\r\nSELECT *\r\nFROM mydata.mytable\r\nWHERE age < 40\r\n\"\"\"\r\n\r\nnew_dataset = envi.gcloud.BQ.query(query).to_dataframe()\r\n```\r\n\r\n### SharePoint features\r\n\r\n```python\r\n# Download a file\r\nenvi.sharepoint.download(input_path=\"/Document partages/sharepoint_folder/my_file.txt\",\r\n output_path=\"local_folder/my_file.txt\")\r\n \r\n# Upload a file\r\nenvi.sharepoint.upload(input_path=\"local_folder/my_file.txt\",\r\n output_path=\"Document partages/folder/my_file.txt\")\r\n \r\n# List files\r\nenvi.sharepoint.list_files(folder=\"local_folder\")\r\n```\r\n\r\n## Documentation\r\n\r\nThe documentation is available here : [Easy Environment - Documentation](https://antoinepinto.gitbook.io/easyenvi/)\r\n\r\n## Credits\r\n\r\n* Thanks to [Herve Mignot](https://github.com/HerveMignot) for his advice on using `fsspec`.\r\n* Thanks to [Nizar Fawal](https://www.linkedin.com/in/nizar-fawal/) for encouraging me to deploy this package as a Pypi library.\r\n* Thanks to [Julien Lambert](https://www.linkedin.com/in/julien--lambert/) for the code snippets and feedback.\r\n\r\n## Future Improvements\r\n\r\nFuture releases of Easy Environment will include support for additional cloud storage providers, including Amazon Web Services (AWS) and Microsoft Azure.\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Easy-to-use functionality for managing files and data in different environments",
"version": "1.0.7",
"project_urls": {
"Documentation": "https://antoinepinto.gitbook.io/easyenvi/",
"Homepage": "https://github.com/AntoinePinto/easyenvi",
"Source Code": "https://github.com/AntoinePinto/easyenvi"
},
"split_keywords": [
"file-manager",
"easy-to-use",
"environment",
"cloud",
"gcp",
"sharepoint"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "13404135ee63c7c64990457bc08d0c81dd3fa3dda857ed1f097b07c1a95064af",
"md5": "cbce3b15647ff2db56a0383615d30966",
"sha256": "9b9d1deb0ead6f6d580cb2ffa7aebba313325adfe073d3302ed77b4ca178567d"
},
"downloads": -1,
"filename": "easyenvi-1.0.7.tar.gz",
"has_sig": false,
"md5_digest": "cbce3b15647ff2db56a0383615d30966",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 10837,
"upload_time": "2024-01-21T14:11:12",
"upload_time_iso_8601": "2024-01-21T14:11:12.980662Z",
"url": "https://files.pythonhosted.org/packages/13/40/4135ee63c7c64990457bc08d0c81dd3fa3dda857ed1f097b07c1a95064af/easyenvi-1.0.7.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-21 14:11:12",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "AntoinePinto",
"github_project": "easyenvi",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "easyenvi"
}
JSON
