.. contents::
=========
Filestore
=========
The filestore is an easy way to to process files with multiple processes
without needing locks.
Initialize a FileStore
======================
Create a filestore in a temporary area:
>>> import tempfile, os
>>> temp_dir = tempfile.mkdtemp()
>>> store_dir = os.path.join(temp_dir, 'store1')
>>> os.mkdir(store_dir)
>>> from gocept.filestore import FileStore
>>> filestore = FileStore(store_dir)
>>> filestore
<gocept.filestore.filestore.FileStore object at 0x...>
So far nothing has happend:
>>> import os
>>> os.listdir(store_dir)
[]
Before using the store we need to prepare it:
>>> filestore.prepare()
Prepare has created the tmp/new/cur directory structure:
>>> sorted(os.listdir(store_dir))
['cur', 'new', 'tmp']
Calling prepare again does nothing:
>>> filestore.prepare()
>>> sorted(os.listdir(store_dir))
['cur', 'new', 'tmp']
If the store_dir is removed, it is created again by calling prepare.
>>> import shutil
>>> shutil.rmtree(store_dir)
>>> os.listdir(temp_dir)
[]
>>> filestore.prepare()
>>> os.listdir(temp_dir)
['store1']
>>> sorted(os.listdir(store_dir))
['cur', 'new', 'tmp']
Use a FileStore
===============
Adding files to the store works with the `create` method:
>>> f = filestore.create('a-file')
Files are created in the 'tmp' area with the 'w' mode (if not specified):
>>> f.name
'.../tmp/a-file'
>>> f.mode
'w'
We find the file in the tmp area. Note that `filestore.list` lists files with
their full path names, so we could feed the name directly to file/open:
>>> filestore.list('tmp')
['.../tmp/a-file']
We got a plain file back, so write to it:
>>> _ = f.write('Die Ente bleibt draussen!')
>>> f.close()
We have finished writing our file, so we can move it to the `new` space for
other applications to pick it up:
>>> filestore.move('a-file', 'tmp', 'new')
>>> filestore.list('tmp')
[]
>>> filestore.list('new')
['.../new/a-file']
The move operation uses os.move which is supposed to be atomic. When another
processes "sees" the file it can directly work with it and move it to 'cur':
>>> filestore.move('a-file', 'new', 'cur')
>>> filestore.list('new')
[]
>>> filestore.list('cur')
['.../cur/a-file']
Files can be copied, too:
>>> filestore.copy('a-file', 'cur', 'tmp')
>>> filestore.list('cur')
['.../cur/a-file']
>>> filestore.list('tmp')
['.../tmp/a-file']
Finally, files can be removed:
>>> filestore.remove('a-file', 'cur')
>>> filestore.list('cur')
[]
Cleanup
=======
Remove the temporary directory after testing:
>>> import shutil
>>> shutil.rmtree(store_dir)
=======
Changes
=======
1.0 (2023-07-14)
================
- Drop support for Python 2.7, 3.5, 3.6.
0.5 (2023-03-16)
================
- Add support for Python 3,9, 3.10, 3.11.
- Use GitHub actions as CI.
0.4 (2019-11-29)
================
- Migrate repository to Bitbucket.
- Migrate repository to GitHub.
- Made Python 3 compatible (tested with Python 2.7, 3.7 and 3.8).
- Replace bootstrap/buildout with tox.
- Increase test coverage to 100%.
0.3 (2009-10-08)
================
- Added copy() method.
0.2 (2007-08-30)
================
- Initial public release.
Raw data
{
"_id": null,
"home_page": "https://github.com/gocept/gocept.filestore",
"name": "gocept.filestore",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "filesystem consistency",
"author": "gocept gmbh & co. kg",
"author_email": "mail@gocept.com",
"download_url": "https://files.pythonhosted.org/packages/e5/a7/98c1abc3c87a7a7a3aade652b7fea069c6a49429f14e087451bdcf0e7f52/gocept.filestore-1.0.tar.gz",
"platform": null,
"description": ".. contents::\n\n=========\nFilestore\n=========\n\nThe filestore is an easy way to to process files with multiple processes\nwithout needing locks.\n\n\nInitialize a FileStore\n======================\n\nCreate a filestore in a temporary area:\n\n>>> import tempfile, os\n>>> temp_dir = tempfile.mkdtemp()\n>>> store_dir = os.path.join(temp_dir, 'store1')\n>>> os.mkdir(store_dir)\n>>> from gocept.filestore import FileStore\n>>> filestore = FileStore(store_dir)\n>>> filestore\n<gocept.filestore.filestore.FileStore object at 0x...>\n\nSo far nothing has happend:\n\n>>> import os\n>>> os.listdir(store_dir)\n[]\n\nBefore using the store we need to prepare it:\n\n>>> filestore.prepare()\n\nPrepare has created the tmp/new/cur directory structure:\n\n>>> sorted(os.listdir(store_dir))\n['cur', 'new', 'tmp']\n\nCalling prepare again does nothing:\n\n>>> filestore.prepare()\n>>> sorted(os.listdir(store_dir))\n['cur', 'new', 'tmp']\n\nIf the store_dir is removed, it is created again by calling prepare.\n\n>>> import shutil\n>>> shutil.rmtree(store_dir)\n>>> os.listdir(temp_dir)\n[]\n>>> filestore.prepare()\n>>> os.listdir(temp_dir)\n['store1']\n>>> sorted(os.listdir(store_dir))\n['cur', 'new', 'tmp']\n\n\nUse a FileStore\n===============\n\nAdding files to the store works with the `create` method:\n\n>>> f = filestore.create('a-file')\n\nFiles are created in the 'tmp' area with the 'w' mode (if not specified):\n\n>>> f.name\n'.../tmp/a-file'\n>>> f.mode\n'w'\n\nWe find the file in the tmp area. Note that `filestore.list` lists files with\ntheir full path names, so we could feed the name directly to file/open:\n\n>>> filestore.list('tmp')\n['.../tmp/a-file']\n\nWe got a plain file back, so write to it:\n\n>>> _ = f.write('Die Ente bleibt draussen!')\n>>> f.close()\n\nWe have finished writing our file, so we can move it to the `new` space for\nother applications to pick it up:\n\n>>> filestore.move('a-file', 'tmp', 'new')\n>>> filestore.list('tmp')\n[]\n>>> filestore.list('new')\n['.../new/a-file']\n\nThe move operation uses os.move which is supposed to be atomic. When another\nprocesses \"sees\" the file it can directly work with it and move it to 'cur':\n\n>>> filestore.move('a-file', 'new', 'cur')\n>>> filestore.list('new')\n[]\n>>> filestore.list('cur')\n['.../cur/a-file']\n\nFiles can be copied, too:\n\n>>> filestore.copy('a-file', 'cur', 'tmp')\n>>> filestore.list('cur')\n['.../cur/a-file']\n>>> filestore.list('tmp')\n['.../tmp/a-file']\n\nFinally, files can be removed:\n\n>>> filestore.remove('a-file', 'cur')\n>>> filestore.list('cur')\n[]\n\nCleanup\n=======\n\nRemove the temporary directory after testing:\n\n>>> import shutil\n>>> shutil.rmtree(store_dir)\n\n\n=======\nChanges\n=======\n\n1.0 (2023-07-14)\n================\n\n- Drop support for Python 2.7, 3.5, 3.6.\n\n\n0.5 (2023-03-16)\n================\n\n- Add support for Python 3,9, 3.10, 3.11.\n\n- Use GitHub actions as CI.\n\n\n0.4 (2019-11-29)\n================\n\n- Migrate repository to Bitbucket.\n\n- Migrate repository to GitHub.\n\n- Made Python 3 compatible (tested with Python 2.7, 3.7 and 3.8).\n\n- Replace bootstrap/buildout with tox.\n\n- Increase test coverage to 100%.\n\n\n0.3 (2009-10-08)\n================\n\n- Added copy() method.\n\n0.2 (2007-08-30)\n================\n\n- Initial public release.\n",
"bugtrack_url": null,
"license": "ZPL 2.1",
"summary": "Provides maildir like access to files",
"version": "1.0",
"project_urls": {
"Homepage": "https://github.com/gocept/gocept.filestore"
},
"split_keywords": [
"filesystem",
"consistency"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "8f25069ec6906b96827a4ccb9ed6af458964d7320c8e331a5d5de2da57a4451b",
"md5": "6dcd48c8a251f57ee3055d20cd145141",
"sha256": "f2488366f26e8c32e7c53fcb09e8d22c1918c4bcbbeba6a9afa1324c471bbcc6"
},
"downloads": -1,
"filename": "gocept.filestore-1.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "6dcd48c8a251f57ee3055d20cd145141",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.7",
"size": 7414,
"upload_time": "2023-07-14T12:53:06",
"upload_time_iso_8601": "2023-07-14T12:53:06.717537Z",
"url": "https://files.pythonhosted.org/packages/8f/25/069ec6906b96827a4ccb9ed6af458964d7320c8e331a5d5de2da57a4451b/gocept.filestore-1.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e5a798c1abc3c87a7a7a3aade652b7fea069c6a49429f14e087451bdcf0e7f52",
"md5": "e3016febe64d187f6c1719a35fe3dac3",
"sha256": "3fbec68b20cf2b2df54d046aafb6a382a83a772e1b5c59b459c44ec4dd843786"
},
"downloads": -1,
"filename": "gocept.filestore-1.0.tar.gz",
"has_sig": false,
"md5_digest": "e3016febe64d187f6c1719a35fe3dac3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 6308,
"upload_time": "2023-07-14T12:53:07",
"upload_time_iso_8601": "2023-07-14T12:53:07.923763Z",
"url": "https://files.pythonhosted.org/packages/e5/a7/98c1abc3c87a7a7a3aade652b7fea069c6a49429f14e087451bdcf0e7f52/gocept.filestore-1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-14 12:53:07",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "gocept",
"github_project": "gocept.filestore",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"tox": true,
"lcname": "gocept.filestore"
}
JSON
