super-csv
=============================
|pypi-badge| |CI| |codecov-badge| |doc-badge| |pyversions-badge|
|license-badge|
Generic CSV Processing for Django Apps
Overview
------------------------
This library provides a `CSVProcessor <https://github.com/openedx/super-csv/blob/master/super_csv/csv_processor.py>`_ class
which should be subclassed to implement your own per-row processing of CSV files.
At minimum, override `process_row(row)`.
The mixins support optional checksums of arbitrary columns, and asynchronous processing of files using Celery.
Documentation
-------------
Super CSV is a stand alone library that can be used for CSV management, both syncronous and async.
Testing
-------
::
make requirements
Will install all prerequisites. ::
make test
Runs the unit tests in local environment.
Using with Docker Devstack
--------------------------
Prerequisite: Have your Open edX https://github.com/openedx/devstack properly installed.
Note: When you see "from inside the lms" below, it means that you've run ``make lms-shell`` from your devstack directory
and are on a command prompt inside the LMS container.
#. Clone this repo into ``../src/`` directory (relative to your "devstack" repo location). This will mount the directory
in a way that is accessible to the lms container.
#. Clone inside the lms, uninstall super-csv and reinstall your local copy.
You can run the following line from inside the lms, or from the host machine run ``make install-local``.
This is necessary if one wants to use latest version for testing/development purposes::
pip uninstall super-csv -y; pip install -e /edx/src/super-csv
#. Setup dev environment (since virtual environments are recommended for python development, here is an example of using
virtualenv. Other tools would work as well). By default, edx containers come with virtualenv preinstalled ::
cd /edx/src/super-csv
virtualenv super-csv-env
source super-csv-env/bin/activate
make requirements
#. That's it.
License
-------
The code in this repository is licensed under the Apache v2 License unless
otherwise noted.
Please see ``LICENSE.txt`` for details.
How To Contribute
-----------------
Contributions are very welcome.
Please read `How To Contribute <https://github.com/openedx/.github/blob/master/CONTRIBUTING.md>`_ for details.
The pull request description template should be automatically applied if you are creating a pull request from GitHub. Otherwise you
can find it at `PULL_REQUEST_TEMPLATE.md <https://github.com/openedx/super-csv/blob/master/.github/PULL_REQUEST_TEMPLATE.md>`_.
The issue report template should be automatically applied if you are creating an issue on GitHub as well. Otherwise you
can find it at `ISSUE_TEMPLATE.md <https://github.com/openedx/super-csv/blob/master/.github/ISSUE_TEMPLATE.md>`_.
Reporting Security Issues
-------------------------
Please do not report security issues in public. Please email security@openedx.org.
Getting Help
------------
Have a question about this repository, or about Open edX in general? Please
refer to this `list of resources`_ if you need any assistance.
.. _list of resources: https://open.edx.org/getting-help
.. |pypi-badge| image:: https://img.shields.io/pypi/v/super-csv.svg
:target: https://pypi.python.org/pypi/super-csv/
:alt: PyPI
.. |CI| image:: https://github.com/openedx/super-csv/workflows/Python%20CI/badge.svg?branch=master
:target: https://github.com/openedx/super-csv/actions?query=workflow%3A%22Python+CI%22
:alt: CI
.. |codecov-badge| image:: http://codecov.io/github/edx/super-csv/coverage.svg?branch=master
:target: http://codecov.io/github/edx/super-csv?branch=master
:alt: Codecov
.. |doc-badge| image:: https://readthedocs.org/projects/super-csv/badge/?version=latest
:target: http://super-csv.readthedocs.io/en/latest/
:alt: Documentation
.. |pyversions-badge| image:: https://img.shields.io/pypi/pyversions/super-csv.svg
:target: https://pypi.python.org/pypi/super-csv/
:alt: Supported Python versions
.. |license-badge| image:: https://img.shields.io/github/license/edx/super-csv.svg
:target: https://github.com/openedx/super-csv/blob/master/LICENSE.txt
:alt: License
Change Log
----------
..
All enhancements and patches to super_csv will be documented
in this file. It adheres to the structure of http://keepachangelog.com/ ,
but in reStructuredText instead of Markdown (for ease of incorporation into
Sphinx documentation and the PyPI description).
This project adheres to Semantic Versioning (http://semver.org/).
.. There should always be an "Unreleased" section for changes pending release.
Unreleased
----------
[3.2.0] - 2023-07-31
~~~~~~~~~~~~~~~~~~~~
* Added support of python 3.11 and 3.12
* Dropped django32 support.
[3.1.0] - 2023-07-31
~~~~~~~~~~~~~~~~~~~~
* Switch from ``edx-sphinx-theme`` to ``sphinx-book-theme`` since the former is
deprecated. See https://github.com/openedx/edx-sphinx-theme/issues/184 for
more details.
* Added support of Django 4.2
[3.0.1] - 2022-02-17
~~~~~~~~~~~~~~~~~~~~
* fix: encode the csv data before save
[3.0.0] - 2022-02-17
~~~~~~~~~~~~~~~~~~~~
* Fix `dev.in` to pull from `ci.txt` rather than `travis.txt`.
* Dropped Django22, 30 and 31 support.
* Added Django40 support.
[2.1.1] - 2021-09-01
~~~~~~~~~~~~~~~~~~~~
* Fix missing file error for bulk grade. Some of the files were expired but the code still expect to read it without `try/catch`.
[2.1.0] - 2020-07-26
~~~~~~~~~~~~~~~~~~~~
* Added Django 3.0, 3.1 & 3.2 Support
[2.0.1] - 2020-03-12
~~~~~~~~~~~~~~~~~~~~
* Update write_file and get_iterator to allow row/column overrides
[2.0.0] - 2020-12-23
~~~~~~~~~~~~~~~~~~~~
* Removed python 3.5 support
* Adds support for celery 5
[1.1.0] - 2020-12-02
~~~~~~~~~~~~~~~~~~~~
* Add code_owner custom attribute for monitoring celery tasks.
[1.0.3] - 2020-11-20
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Updated travis badge in README.rst to point to travis-ci.com
[1.0.2] - 2020-09-14
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Move to Apache License
[1.0.1] - 2020-09-14
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Minor updates to attrs, code-annotations, and freezegun dependencies
[1.0.0] - 2020-09-02
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Upgrade celery to 4.2.2
[0.9.9] - 2020-05-28
~~~~~~~~~~~~~~~~~~~~
* Removed caniusepython3 & python_2_unicode_compatible packages
[0.9.8] - 2020-05-04
~~~~~~~~~~~~~~~~~~~~
* dropped support for Django version less than 2.2 and Added support for python 3.8
[0.9.7] - 2020-03-05
~~~~~~~~~~~~~~~~~~~~
* Remove django-celery dependency
[0.9.6] - 2019-11-20
~~~~~~~~~~~~~~~~~~~~
* Django 2.2 support
[0.9.5] - 2019-10-08
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Decode the input file before passing to DictReader if necessary
[0.9.4] - 2019-09-24
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Let the ``DeferrableMixin.save()`` method take an optional ``operating_user`` parameter.
[0.9.3] - 2019-09-20
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Commit after ``CSVOperation`` creation so that async celery tasks can find the operation record when they start.
[0.9.2] - 2019-09-17
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* If a class inheriting from DeferrableMixin has a field user_id, use that user for the CSVOperation
[0.9.1] - 2019-07-19
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Reverses the ordering of CSV operation history rows
[0.8] - 2019-07-22
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Changes checksum column to avoid CSV quoting issues
[0.7.1] - 2019-07-19
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Exposes additional fields in serialized history of operations re:degree of success of the operation
[0.5.0] - 2019-07-02
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* Exposes methods for serializing history of operations for particular processors.
[0.1.0] - 2019-05-15
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Added
_____
* First release on PyPI.
Raw data
{
"_id": null,
"home_page": "https://github.com/openedx/super-csv",
"name": "super-csv",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "Django edx",
"author": "edX",
"author_email": "oscm@edx.org",
"download_url": "https://files.pythonhosted.org/packages/ec/7c/1095bf8ec49c1b2e3caf175fe3343ea68480bb2276e9bd5775d62b411084/super-csv-3.2.0.tar.gz",
"platform": null,
"description": "super-csv\n=============================\n\n|pypi-badge| |CI| |codecov-badge| |doc-badge| |pyversions-badge|\n|license-badge|\n\nGeneric CSV Processing for Django Apps\n\nOverview\n------------------------\n\nThis library provides a `CSVProcessor <https://github.com/openedx/super-csv/blob/master/super_csv/csv_processor.py>`_ class\nwhich should be subclassed to implement your own per-row processing of CSV files.\nAt minimum, override `process_row(row)`.\n\nThe mixins support optional checksums of arbitrary columns, and asynchronous processing of files using Celery.\n\nDocumentation\n-------------\n\nSuper CSV is a stand alone library that can be used for CSV management, both syncronous and async.\n\nTesting\n-------\n::\n\n make requirements\n\nWill install all prerequisites. ::\n\n make test\n\nRuns the unit tests in local environment.\n\nUsing with Docker Devstack\n--------------------------\nPrerequisite: Have your Open edX https://github.com/openedx/devstack properly installed.\nNote: When you see \"from inside the lms\" below, it means that you've run ``make lms-shell`` from your devstack directory\nand are on a command prompt inside the LMS container.\n\n#. Clone this repo into ``../src/`` directory (relative to your \"devstack\" repo location). This will mount the directory\n in a way that is accessible to the lms container.\n\n#. Clone inside the lms, uninstall super-csv and reinstall your local copy.\n You can run the following line from inside the lms, or from the host machine run ``make install-local``.\n This is necessary if one wants to use latest version for testing/development purposes::\n\n pip uninstall super-csv -y; pip install -e /edx/src/super-csv\n\n#. Setup dev environment (since virtual environments are recommended for python development, here is an example of using\n virtualenv. Other tools would work as well). By default, edx containers come with virtualenv preinstalled ::\n\n cd /edx/src/super-csv\n virtualenv super-csv-env\n source super-csv-env/bin/activate\n make requirements\n\n#. That's it.\n\n\nLicense\n-------\n\nThe code in this repository is licensed under the Apache v2 License unless\notherwise noted.\n\nPlease see ``LICENSE.txt`` for details.\n\nHow To Contribute\n-----------------\n\nContributions are very welcome.\n\nPlease read `How To Contribute <https://github.com/openedx/.github/blob/master/CONTRIBUTING.md>`_ for details.\n\n\nThe pull request description template should be automatically applied if you are creating a pull request from GitHub. Otherwise you\ncan find it at `PULL_REQUEST_TEMPLATE.md <https://github.com/openedx/super-csv/blob/master/.github/PULL_REQUEST_TEMPLATE.md>`_.\n\nThe issue report template should be automatically applied if you are creating an issue on GitHub as well. Otherwise you\ncan find it at `ISSUE_TEMPLATE.md <https://github.com/openedx/super-csv/blob/master/.github/ISSUE_TEMPLATE.md>`_.\n\nReporting Security Issues\n-------------------------\n\nPlease do not report security issues in public. Please email security@openedx.org.\n\nGetting Help\n------------\n\nHave a question about this repository, or about Open edX in general? Please\nrefer to this `list of resources`_ if you need any assistance.\n\n.. _list of resources: https://open.edx.org/getting-help\n\n\n.. |pypi-badge| image:: https://img.shields.io/pypi/v/super-csv.svg\n :target: https://pypi.python.org/pypi/super-csv/\n :alt: PyPI\n\n.. |CI| image:: https://github.com/openedx/super-csv/workflows/Python%20CI/badge.svg?branch=master\n :target: https://github.com/openedx/super-csv/actions?query=workflow%3A%22Python+CI%22\n :alt: CI\n\n.. |codecov-badge| image:: http://codecov.io/github/edx/super-csv/coverage.svg?branch=master\n :target: http://codecov.io/github/edx/super-csv?branch=master\n :alt: Codecov\n\n.. |doc-badge| image:: https://readthedocs.org/projects/super-csv/badge/?version=latest\n :target: http://super-csv.readthedocs.io/en/latest/\n :alt: Documentation\n\n.. |pyversions-badge| image:: https://img.shields.io/pypi/pyversions/super-csv.svg\n :target: https://pypi.python.org/pypi/super-csv/\n :alt: Supported Python versions\n\n.. |license-badge| image:: https://img.shields.io/github/license/edx/super-csv.svg\n :target: https://github.com/openedx/super-csv/blob/master/LICENSE.txt\n :alt: License\n\n\nChange Log\n----------\n\n..\n All enhancements and patches to super_csv will be documented\n in this file. It adheres to the structure of http://keepachangelog.com/ ,\n but in reStructuredText instead of Markdown (for ease of incorporation into\n Sphinx documentation and the PyPI description).\n\n This project adheres to Semantic Versioning (http://semver.org/).\n\n.. There should always be an \"Unreleased\" section for changes pending release.\n\nUnreleased\n----------\n[3.2.0] - 2023-07-31\n~~~~~~~~~~~~~~~~~~~~\n\n* Added support of python 3.11 and 3.12\n* Dropped django32 support.\n\n\n[3.1.0] - 2023-07-31\n~~~~~~~~~~~~~~~~~~~~\n\n* Switch from ``edx-sphinx-theme`` to ``sphinx-book-theme`` since the former is\n deprecated. See https://github.com/openedx/edx-sphinx-theme/issues/184 for\n more details.\n* Added support of Django 4.2\n\n[3.0.1] - 2022-02-17\n~~~~~~~~~~~~~~~~~~~~\n\n* fix: encode the csv data before save\n\n[3.0.0] - 2022-02-17\n~~~~~~~~~~~~~~~~~~~~\n\n* Fix `dev.in` to pull from `ci.txt` rather than `travis.txt`.\n* Dropped Django22, 30 and 31 support.\n* Added Django40 support.\n\n[2.1.1] - 2021-09-01\n~~~~~~~~~~~~~~~~~~~~\n\n* Fix missing file error for bulk grade. Some of the files were expired but the code still expect to read it without `try/catch`.\n\n[2.1.0] - 2020-07-26\n~~~~~~~~~~~~~~~~~~~~\n\n* Added Django 3.0, 3.1 & 3.2 Support\n\n[2.0.1] - 2020-03-12\n~~~~~~~~~~~~~~~~~~~~\n\n* Update write_file and get_iterator to allow row/column overrides\n\n[2.0.0] - 2020-12-23\n~~~~~~~~~~~~~~~~~~~~\n\n* Removed python 3.5 support\n* Adds support for celery 5\n\n[1.1.0] - 2020-12-02\n~~~~~~~~~~~~~~~~~~~~\n\n* Add code_owner custom attribute for monitoring celery tasks.\n\n[1.0.3] - 2020-11-20\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n* Updated travis badge in README.rst to point to travis-ci.com\n\n[1.0.2] - 2020-09-14\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* Move to Apache License\n\n[1.0.1] - 2020-09-14\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* Minor updates to attrs, code-annotations, and freezegun dependencies\n\n[1.0.0] - 2020-09-02\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* Upgrade celery to 4.2.2\n\n[0.9.9] - 2020-05-28\n~~~~~~~~~~~~~~~~~~~~\n\n* Removed caniusepython3 & python_2_unicode_compatible packages\n\n[0.9.8] - 2020-05-04\n~~~~~~~~~~~~~~~~~~~~\n\n* dropped support for Django version less than 2.2 and Added support for python 3.8\n\n[0.9.7] - 2020-03-05\n~~~~~~~~~~~~~~~~~~~~\n\n* Remove django-celery dependency\n\n[0.9.6] - 2019-11-20\n~~~~~~~~~~~~~~~~~~~~\n\n* Django 2.2 support\n\n[0.9.5] - 2019-10-08\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* Decode the input file before passing to DictReader if necessary\n\n[0.9.4] - 2019-09-24\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* Let the ``DeferrableMixin.save()`` method take an optional ``operating_user`` parameter.\n\n[0.9.3] - 2019-09-20\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* Commit after ``CSVOperation`` creation so that async celery tasks can find the operation record when they start.\n\n[0.9.2] - 2019-09-17\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* If a class inheriting from DeferrableMixin has a field user_id, use that user for the CSVOperation\n\n[0.9.1] - 2019-07-19\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* Reverses the ordering of CSV operation history rows\n\n[0.8] - 2019-07-22\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* Changes checksum column to avoid CSV quoting issues\n\n[0.7.1] - 2019-07-19\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* Exposes additional fields in serialized history of operations re:degree of success of the operation\n\n[0.5.0] - 2019-07-02\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n* Exposes methods for serializing history of operations for particular processors.\n\n[0.1.0] - 2019-05-15\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nAdded\n_____\n\n* First release on PyPI.\n",
"bugtrack_url": null,
"license": "Apache 2.0",
"summary": "CSV Processor",
"version": "3.2.0",
"project_urls": {
"Homepage": "https://github.com/openedx/super-csv"
},
"split_keywords": [
"django",
"edx"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "2f6c07f6280e5e74a18d2262fb3c7465c04c1a9118777af5c9725856453a3bf2",
"md5": "22faeccd95a8669feace6ac54ae383d6",
"sha256": "63b96de9a035e27f45b3be98ed8bd6f20c94d46df3d0ac7fd44f73c4f3eb0fcc"
},
"downloads": -1,
"filename": "super_csv-3.2.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "22faeccd95a8669feace6ac54ae383d6",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 21539,
"upload_time": "2024-04-08T19:07:57",
"upload_time_iso_8601": "2024-04-08T19:07:57.258252Z",
"url": "https://files.pythonhosted.org/packages/2f/6c/07f6280e5e74a18d2262fb3c7465c04c1a9118777af5c9725856453a3bf2/super_csv-3.2.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ec7c1095bf8ec49c1b2e3caf175fe3343ea68480bb2276e9bd5775d62b411084",
"md5": "f8a5fe0458018f2a9e3dea407e6d06ef",
"sha256": "50074409bb3bac3a0a47e0aa421e97330afa3f824d688e1401a85797ff8e2ab1"
},
"downloads": -1,
"filename": "super-csv-3.2.0.tar.gz",
"has_sig": false,
"md5_digest": "f8a5fe0458018f2a9e3dea407e6d06ef",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 24527,
"upload_time": "2024-04-08T19:07:59",
"upload_time_iso_8601": "2024-04-08T19:07:59.070077Z",
"url": "https://files.pythonhosted.org/packages/ec/7c/1095bf8ec49c1b2e3caf175fe3343ea68480bb2276e9bd5775d62b411084/super-csv-3.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-08 19:07:59",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "openedx",
"github_project": "super-csv",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"tox": true,
"lcname": "super-csv"
}
JSON
