| Name | unifs JSON |
| Version |
1.1.0
JSON |
| download |
| home_page | |
| Summary | Unified FS-like CLI for S3, GCS, ADLS, HDFS, SMB, Dropbox, Google Drive, and dozens of other file systems |
| upload_time | 2023-01-22 19:30:21 |
| maintainer | |
| docs_url | None |
| author | candidtim |
| requires_python | >=3.7 |
| license | BSD 3-Clause License Copyright (c) 2023 candidtim Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
| keywords |
fs
shell
|
| VCS |
|
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
# unifs
Unified FS-like CLI for S3, GCS, ADLS, HDFS, SMB, Dropbox, Google Drive, and
dozens of other "file systems".
unifs conf use my-s3-bucket
unifs ls -l /
unifs mv /foo.txt /bar.txt
unifs download /bar.txt ~/Downloads/local.copy.txt
`unifs` uses the term "file system" in an open sense for anything that can be
represented as a set of files and directories and be manipulated with the
commands like `ls`, `cat`, `cp`, and `mv` for example (list is not exhaustive).
`unifs` also allows data upload and download when working with remote back-ends
(e.g., a cloud-based BLOB storage).
`unifs` supports multiple back-ends, such as a local file system, (S)FTP,
Google Drive, various blob storage such as S3, GCS, ADLS, and dozens of other
implementations. Use `unifs impl list` to list supported protocols, but know
that other protocols can be added, including any custom implementations users
may provide.
`unifs` is different from FUSE implementations in that it doesn't mount a file
system. Instead, it provides a unified CLI that uses target back-end API to
execute the issued commands.
## Installation
`unifs` is a Python package:
pip install unifs
Default `unifs` installation only supports a few basic protocols (e.g., a local
file system). To support other protocols you may need to install their
implementation packages. Because there are too many, `unifs` doesn't install
them for you by default, but it will tell which packages are missing if you
attempt to use a protocol that is not supported out of the box.
For example, to add the support for the GCS:
pip install gcsfs
Make sure to install the additional packages to the same (virtual) environment
where `unifs` is installed.
To list known implementations and their prerequisites, use:
unifs impl list
unifs impl info NAME
To avoid conflicts with other Python packages, it is recommended to install
this application into a dedicated virtual environment. For example, you may use
`pipx`, or create a virtual environment manually. At very least, install with a
`--user` option (`pip install --user unifs`).
## Quick start
By default, `unifs` will use the local file system and will behave much like
issuing the similar commands directly in the shell:
unifs ls -l /
unifs cat /tmp/foo.txt
unifs mv /tmp/foo.tx /tmp/bar.txt
unifs --help
You need to configure `unifs` to let it know about other file systems you will
use.
## Configuration
You may either modify the configuration file, or use `unifs conf` command to
manipulate it.
### Using `unifs conf`
Get the list of configured file systems (currently active one is highlighted):
unifs conf list
Set the active file system:
unifs conf use NAME
### Configuration file
`unifs` configuration is stored in the default OS configuration directory. You
can obtain a configuration file path with:
unifs conf path
Alternatively, you can pin the configuration file location with a
`UNIFS_CONFIG_PATH` environment variable.
If you didn't change your default OS settings, most likely it will
be:
~/.local/share/unifs/config.toml # Linux
~/Library/Application Support/unifs/config.toml # MacOS
~\AppData\Local\unifs\Config\config.toml # Windows
Configuration file is a TOML file that consists of:
- a single `[unifs]` section where the currently active file system is set
- any number of `[unifs.fs.NAME]` sections that declare the file systems
Example:
[unifs]
current = "local"
[unifs.fs.local]
protocol = "file"
auto_mkdir = false
File system configuration is a set of key-value pairs. `protocol` key is
mandatory and is used to select the implementation, all other values are passed
to the specific implementation. Use `unifs impl info NAME` to the list of
accepted parameters for any protocol.
For example, for a GCS bucket:
[unifs.fs.my-gcs-bucket]
protocol = gcs
project = "my-gcp-project"
token = "/path/to/token.json"
## Status
Available `unifs` features are considered stable. `unifs` is being actively
developed and more features are coming.
## Error reporting
If you happen to encounter an error ("An unexpected error" in the output),
please, feel free to report it on the Issues page. In this case, you may find
the detailed error message in the log file located in the same directory as the
application configuration file.
## Word of caution
Beware that `unifs` may change (copy, move, remove, etc.) the data in a "file
system" (as understood above). `unifs` is only a command-line layer between the
user and the target "file system". `unifs` tries its best to prevent errors
(e.g., uses interactive confirmations for some commands), but ultimately the
user is responsible for the operations performed by this program.
`unifs` is designed to be used in an interactive shell, not in headless mode.
Raw data
{
"_id": null,
"home_page": "",
"name": "unifs",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "fs,shell",
"author": "candidtim",
"author_email": "",
"download_url": "",
"platform": null,
"description": "# unifs\n\nUnified FS-like CLI for S3, GCS, ADLS, HDFS, SMB, Dropbox, Google Drive, and\ndozens of other \"file systems\".\n\n unifs conf use my-s3-bucket\n unifs ls -l /\n unifs mv /foo.txt /bar.txt\n unifs download /bar.txt ~/Downloads/local.copy.txt\n\n`unifs` uses the term \"file system\" in an open sense for anything that can be\nrepresented as a set of files and directories and be manipulated with the\ncommands like `ls`, `cat`, `cp`, and `mv` for example (list is not exhaustive).\n`unifs` also allows data upload and download when working with remote back-ends\n(e.g., a cloud-based BLOB storage).\n\n`unifs` supports multiple back-ends, such as a local file system, (S)FTP,\nGoogle Drive, various blob storage such as S3, GCS, ADLS, and dozens of other\nimplementations. Use `unifs impl list` to list supported protocols, but know\nthat other protocols can be added, including any custom implementations users\nmay provide.\n\n`unifs` is different from FUSE implementations in that it doesn't mount a file\nsystem. Instead, it provides a unified CLI that uses target back-end API to\nexecute the issued commands.\n\n## Installation\n\n`unifs` is a Python package:\n\n pip install unifs\n\nDefault `unifs` installation only supports a few basic protocols (e.g., a local\nfile system). To support other protocols you may need to install their\nimplementation packages. Because there are too many, `unifs` doesn't install\nthem for you by default, but it will tell which packages are missing if you\nattempt to use a protocol that is not supported out of the box.\n\nFor example, to add the support for the GCS:\n\n pip install gcsfs\n\nMake sure to install the additional packages to the same (virtual) environment\nwhere `unifs` is installed.\n\nTo list known implementations and their prerequisites, use:\n\n unifs impl list\n unifs impl info NAME\n\nTo avoid conflicts with other Python packages, it is recommended to install\nthis application into a dedicated virtual environment. For example, you may use\n`pipx`, or create a virtual environment manually. At very least, install with a\n`--user` option (`pip install --user unifs`).\n\n## Quick start\n\nBy default, `unifs` will use the local file system and will behave much like\nissuing the similar commands directly in the shell:\n\n unifs ls -l /\n unifs cat /tmp/foo.txt\n unifs mv /tmp/foo.tx /tmp/bar.txt\n unifs --help\n\nYou need to configure `unifs` to let it know about other file systems you will\nuse.\n\n## Configuration\n\nYou may either modify the configuration file, or use `unifs conf` command to\nmanipulate it.\n\n### Using `unifs conf`\n\nGet the list of configured file systems (currently active one is highlighted):\n\n unifs conf list\n\nSet the active file system:\n\n unifs conf use NAME\n\n### Configuration file\n\n`unifs` configuration is stored in the default OS configuration directory. You\ncan obtain a configuration file path with:\n\n unifs conf path\n\nAlternatively, you can pin the configuration file location with a\n`UNIFS_CONFIG_PATH` environment variable.\n\nIf you didn't change your default OS settings, most likely it will\nbe:\n\n ~/.local/share/unifs/config.toml # Linux\n ~/Library/Application Support/unifs/config.toml # MacOS\n ~\\AppData\\Local\\unifs\\Config\\config.toml # Windows\n\nConfiguration file is a TOML file that consists of:\n\n - a single `[unifs]` section where the currently active file system is set\n - any number of `[unifs.fs.NAME]` sections that declare the file systems\n\nExample:\n\n [unifs]\n current = \"local\"\n\n [unifs.fs.local]\n protocol = \"file\"\n auto_mkdir = false\n\nFile system configuration is a set of key-value pairs. `protocol` key is\nmandatory and is used to select the implementation, all other values are passed\nto the specific implementation. Use `unifs impl info NAME` to the list of\naccepted parameters for any protocol.\n\nFor example, for a GCS bucket:\n\n [unifs.fs.my-gcs-bucket]\n protocol = gcs\n project = \"my-gcp-project\"\n token = \"/path/to/token.json\"\n\n## Status\n\nAvailable `unifs` features are considered stable. `unifs` is being actively\ndeveloped and more features are coming.\n\n## Error reporting\n\nIf you happen to encounter an error (\"An unexpected error\" in the output),\nplease, feel free to report it on the Issues page. In this case, you may find\nthe detailed error message in the log file located in the same directory as the\napplication configuration file.\n\n## Word of caution\n\nBeware that `unifs` may change (copy, move, remove, etc.) the data in a \"file\nsystem\" (as understood above). `unifs` is only a command-line layer between the\nuser and the target \"file system\". `unifs` tries its best to prevent errors\n(e.g., uses interactive confirmations for some commands), but ultimately the\nuser is responsible for the operations performed by this program.\n\n`unifs` is designed to be used in an interactive shell, not in headless mode.\n",
"bugtrack_url": null,
"license": "BSD 3-Clause License Copyright (c) 2023 candidtim Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS \"AS IS\" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. ",
"summary": "Unified FS-like CLI for S3, GCS, ADLS, HDFS, SMB, Dropbox, Google Drive, and dozens of other file systems",
"version": "1.1.0",
"split_keywords": [
"fs",
"shell"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6accf5a7222357a2bcdf1336f4b138c8774c81367ffdf6e127deb94e65e8eb35",
"md5": "9393a712b4a4962be687d9f00175828e",
"sha256": "16429973519d2545f43cfd360d68c0ee1f5431a661e8a6bb7c36e9767d82c4e6"
},
"downloads": -1,
"filename": "unifs-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "9393a712b4a4962be687d9f00175828e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 16112,
"upload_time": "2023-01-22T19:30:21",
"upload_time_iso_8601": "2023-01-22T19:30:21.330460Z",
"url": "https://files.pythonhosted.org/packages/6a/cc/f5a7222357a2bcdf1336f4b138c8774c81367ffdf6e127deb94e65e8eb35/unifs-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-01-22 19:30:21",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "unifs"
}
