ricecooker
==========
[](https://pypi.python.org/pypi/ricecooker/)
[](https://github.com/learningequality/ricecooker/actions)
[](https://ricecooker.readthedocs.io/)
The `ricecooker` library is a framework for automating the conversion of educational content into
Kolibri content channels and uploading them to [Kolibri Studio](https://studio.learningequality.org/),
which is the central content server for [Kolibri](http://learningequality.org/kolibri/).
## [📚 Ricecooker docs](https://ricecooker.readthedocs.io/)
Visit the documentation site at [📚 ricecooker.readthedocs.io](https://ricecooker.readthedocs.io/)
for the full details about [installation](https://ricecooker.readthedocs.io/en/latest/installation.html),
[getting started](https://ricecooker.readthedocs.io/en/latest/tutorial/gettingstarted.html),
[API reference](https://ricecooker.readthedocs.io/en/latest/index_api_reference.html),
and [code examples](https://ricecooker.readthedocs.io/en/latest/examples/index.html).
## Overview of Kolibri content
`ricecooker` is used to take openly licensed educational content available on the
web and convert it into an offline-friendly package that can be imported into Kolibri.
The basic process of getting new content into Kolibri is as follows:
- **UPLOAD** your content to Kolibri Studio either manually through the Kolibri Studio
web interface or programmatically using a `ricecooker`-based content integration script.
- **PUBLISH** the channel on Kolibri Studio to make it accessible for use in Kolibri.
- **IMPORT** the the channel into Kolibri using the channel token displayed in
Kolibri Studio after the PUBLISH step is done.
The diagram below illustrates how content flows within the Kolibri ecosystem
and highlights the part which is covered by the `ricecooker` framework (bottom left).
*External content sources (left) are first uploaded to [Kolibri Studio](https://studio.learningequality.org/) (middle), so they can be used in the [Kolibri Learning Platform](http://learningequality.org/kolibri/) (right).*
##### Further reading
The [Ricecooker docs](https://ricecooker.readthedocs.io/) website is the best
place to learn about writing automated content integration scripts.
Here are some links to other documents and guides you can read to learn about
the other parts of the Kolibri content platform:
- The [Kolibri Content Integration Guide](https://learningequality.org/r/integration-guide)
is a comprehensive guide to the decisions, processes, and tools for integrating
external content sources for use in the Kolibri Learning Platform.
- Read the [Kolibri Studio docs](http://kolibri-studio.readthedocs.io/en/latest/)
to learn more about the Kolibri Studio features
- Read the [Kolibri docs](http://kolibri.readthedocs.io/en/latest/) to learn
how to install Kolibri on your machine (useful for testing channels)
=======
History
=======
0.6.46 (2020-09-21)
-------------------
* Add ``ricecooker.utils.youtube``, containing utilities for handling and caching YouTube videos and playlists.
* Improve validation of ``m`` and ``n`` mastery model values in ``ExerciseNode``.
0.6.45 (2020-07-25)
-------------------
* Remove SushiBar remote monitoring and remote command functionality
0.6.44 (2020-07-16)
-------------------
* Documentation overhaul and refresh, see `ricecooker.readthedocs.io <https://ricecooker.readthedocs.io/>`__
* Add support for specifying a Channel tagline
* Ensure we send ``extra_fields`` data for all node types
* Make ``--reset`` behavior the default
* Remove legacy code around ``compatibiliry_mode`` and BaseChef class
* Improved caching logic (no caching for thumbnails and local paths, always cache youtube downloads)
* Ensure chefs clean up after run (automatic temp files removal)
* Added ``save_channel_tree_as_json`` full channel metadata as part of every run.
* Added support for web content archiving
* Further improvements to logging
* Bugfix: deep deterministic cache keys for nested dicts values in ``ffmpeg_settings``
0.6.42 (2020-04-10)
-------------------
* Added ``--sample N`` command line option. Run script with ``--sample 10`` to
produce a test version of the channel with 10 randomly selected nodes from
the full channel. Use this to check transformations are working as expected.
* Added ``dryrun`` command. Use the command ``./sushichef.py dryrun`` to run the
chef as normal but skip the step where the files get uploaded to Studio.
* Added HTTP proxy functionality for YouTubeVideoFile and YouTubeSubtitleFile
Set the ``PROXY_LIST`` env variable to a ``;``-separated list of ``{ip}:{port}``.
Ricecooker will detect the presence of the ``PROXY_LIST`` and use it when
accessing resources via YoutubeDL. Alternatively, set ``USEPROXY`` env var
to use a list of free proxy servers, which are very slow and not reliable.
* Improved colored logging functionality and customizability of logging output.
0.6.40 (2020-02-07)
-------------------
* Changed default behaviour to upload the staging tree instead of the main tree
* Added ``--deploy`` flag to reproduce old bahavior (upload to main tree)
* Added thumbnail generating methods for audio, HTML5, PDF, and ePub nodes.
Set the ``derive_thumbnail=True`` when creating the Node instance, or pass the
command line argument ``--thumbnails`` to generate thumbnails for all nodes.
Note: automatic thumbnail generation will only work if ``thumbnail`` is None.
0.6.38 (2019-12-27)
-------------------
* Added support the ``h5p`` content kind and ``h5p`` file type
* Removed monkey-patching of ``localStorage`` and ``document.cookie``
in the helper method ``download_static_assets``
* Added validation logic for tags
* Improved error reporting
0.6.36 (2019-09-25)
-------------------
* Added support for tags using the ``JsonChef`` workflow
* Added validation step to ensure subtitles file are unique for each language code
* Document new ``SlidesShow`` content kind coming in Kolibri 0.13
* Added docs with detailed instruction for content upload and update workflows
* Bugfixes to file extension logic and improved error handling around subtitles
0.6.32 (2019-08-01)
-------------------
* Updated documentation to use top-level headings
* Removed support for Python 3.4
* Removed support for the "sous chef" workflow
0.6.31 (2019-07-01)
-------------------
* Handle more subtitle convertible formats: ``SRT``, ``TTML``, ``SCC``, ``DFXP``, and ``SAMI``
0.6.30 (2019-05-01)
-------------------
* Updated docs build scripts to make ricecooker docs available on read the docs
* Added ``corrections`` command line script for making bulk edits to content metadata
* Added ``StudioApi`` client to support CRUD (created, read, update, delete) Studio actions
* Added pdf-splitting helper methods (see ``ricecooker/utils/pdf.py``)
0.6.23 (2018-11-08)
-------------------
* Updated ``le-utils`` and ``pressurcooker`` dependencies to latest version
* Added support for ePub files (``EPubFile`` s can be added of ``DocumentNode`` s)
* Added tag support
* Changed default value for ``STUDIO_URL`` to ``api.studio.learningequality.org``
* Added ``aggregator`` and ``provider`` fields for content nodes
* Various bugfixes to image processing in exercises
* Changed validation logic to use ``self.filename`` to check file format is in ``self.allowed_formats``
* Added ``is_youtube_subtitle_file_supported_language`` helper function to support importing youtube subs
* Added ``srt2vtt`` subtitles conversion
* Added static assets downloader helper method in ``utils.downloader.download_static_assets``
* Added LineCook chef functions to ``--generate`` CSV from directory structure
* Fixed the always ``randomize=True`` bug
* Docs: general content node metadata guidelines
* Docs: video compression instructions and helper scripts ``convertvideo.bat`` and ``convertvideo.sh``
0.6.17 (2018-04-20)
-------------------
* Added support for ``role`` attribute on ConentNodes (currently ``coach`` || ``learner``)
* Update pressurecooker dependency (to catch compression errors)
* Docs improvements, see https://github.com/learningequality/ricecooker/tree/master/docs
0.6.15 (2018-03-06)
-------------------
* Added support for non-mp4 video files, with auto-conversion using ffmpeg. See ``git diff b1d15fa 87f2528``
* Added CSV exercises workflow support to ``LineCook`` chef class
* Added --nomonitor CLI argument to disable sushibar functionality
* Defined new ENV variables:
* PHANTOMJS_PATH: set this to a phantomjs binary (instead of assuming one in node_modules)
* STUDIO_URL (alias CONTENTWORKSHOP_URL): set to URL of Kolibri Studio server where to upload files
* Various fixes to support sushi chefs
* Removed ``minimize_html_css_js`` utility function from ``ricecooker/utils/html.py``
to remove dependency on ``css_html_js_minify`` and support Py3.4 fully.
0.6.9 (2017-11-14)
------------------
* Changed default logging level to --verbose
* Added support for cronjobs scripts via `--cmdsock` (see docs/daemonization.md)
* Added tools for creating HTML5Zip files in utils/html_writer.py
* Added utility for downloading HTML with optional js support in utils/downloader.py
* Added utils/path_builder.py and utils/data_writer.py for creating souschef archives
(zip archive that contains files in a folder hierarchy + Channel.csv + Content.csv)
0.6.7 (2017-10-04)
------------------
* Sibling content nodes are now required to have unique source_id
* The field `copyright_holder` is required for all licenses other than public domain
0.6.7 (2017-10-04)
------------------
* Sibling content nodes are now required to have unique source_id
* The field `copyright_holder` is required for all licenses other than public domain
0.6.6 (2017-09-29)
------------------
* Added `JsonTreeChef` class for creating channels from ricecooker json trees
* Added `LineCook` chef class to support souschef-based channel workflows
0.6.4 (2017-08-31)
------------------
* Added `language` attribute for `ContentNode` (string key in internal repr. defined in le-utils)
* Made `language` a required attribute for ChannelNode
* Enabled sushibar.learningequality.org progress monitoring by default
Set SUSHIBAR_URL env. var to control where progress is reported (e.g. http://localhost:8001)
* Updated le-utils and pressurecooker dependencies to latest
0.6.2 (2017-07-07)
------------------
* Clarify ricecooker is Python3 only (for now)
* Use https:// and wss:// for SuhiBar reporting
0.6.0 (2017-06-28)
------------------
* Remote progress reporting and logging to SushiBar (MVP version)
* New API based on the SuchiChef classes
* Support existing old-API chefs in compatibility mode
0.5.13 (2017-06-15)
-------------------
* Last stable release before SushiBar functionality was added
* Renamed --do-not-activate argument to --stage
0.1.0 (2016-09-30)
------------------
* First release on PyPI.
Raw data
{
"_id": null,
"home_page": "https://github.com/learningequality/ricecooker",
"name": "ricecooker",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7, <3.11",
"maintainer_email": "",
"keywords": "ricecooker",
"author": "Learning Equality",
"author_email": "dev@learningequality.org",
"download_url": "https://files.pythonhosted.org/packages/4d/ab/2069bfb1e32ac7bd72efe4535dc3d5ae7ad66a4fca9c9bd5f394ef8e0c6f/ricecooker-0.7.3.tar.gz",
"platform": null,
"description": "ricecooker\n==========\n[](https://pypi.python.org/pypi/ricecooker/)\n[](https://github.com/learningequality/ricecooker/actions)\n[](https://ricecooker.readthedocs.io/)\n\n\nThe `ricecooker` library is a framework for automating the conversion of educational content into\nKolibri content channels and uploading them to [Kolibri Studio](https://studio.learningequality.org/),\nwhich is the central content server for [Kolibri](http://learningequality.org/kolibri/).\n\n\n## [\ud83d\udcda Ricecooker docs](https://ricecooker.readthedocs.io/)\n\nVisit the documentation site at [\ud83d\udcda ricecooker.readthedocs.io](https://ricecooker.readthedocs.io/)\nfor the full details about [installation](https://ricecooker.readthedocs.io/en/latest/installation.html),\n[getting started](https://ricecooker.readthedocs.io/en/latest/tutorial/gettingstarted.html),\n[API reference](https://ricecooker.readthedocs.io/en/latest/index_api_reference.html),\nand [code examples](https://ricecooker.readthedocs.io/en/latest/examples/index.html).\n\n\n\n## Overview of Kolibri content\n\n`ricecooker` is used to take openly licensed educational content available on the\nweb and convert it into an offline-friendly package that can be imported into Kolibri.\n\nThe basic process of getting new content into Kolibri is as follows:\n\n - **UPLOAD** your content to Kolibri Studio either manually through the Kolibri Studio\n web interface or programmatically using a `ricecooker`-based content integration script.\n - **PUBLISH** the channel on Kolibri Studio to make it accessible for use in Kolibri.\n - **IMPORT** the the channel into Kolibri using the channel token displayed in\n Kolibri Studio after the PUBLISH step is done.\n\nThe diagram below illustrates how content flows within the Kolibri ecosystem\nand highlights the part which is covered by the `ricecooker` framework (bottom left).\n\n\n*External content sources (left) are first uploaded to [Kolibri Studio](https://studio.learningequality.org/) (middle), so they can be used in the [Kolibri Learning Platform](http://learningequality.org/kolibri/) (right).*\n\n\n\n\n##### Further reading\nThe [Ricecooker docs](https://ricecooker.readthedocs.io/) website is the best\nplace to learn about writing automated content integration scripts.\n\nHere are some links to other documents and guides you can read to learn about\nthe other parts of the Kolibri content platform:\n\n - The [Kolibri Content Integration Guide](https://learningequality.org/r/integration-guide)\n is a comprehensive guide to the decisions, processes, and tools for integrating\n external content sources for use in the Kolibri Learning Platform.\n - Read the [Kolibri Studio docs](http://kolibri-studio.readthedocs.io/en/latest/)\n to learn more about the Kolibri Studio features\n - Read the [Kolibri docs](http://kolibri.readthedocs.io/en/latest/) to learn\n how to install Kolibri on your machine (useful for testing channels)\n\n\n=======\nHistory\n=======\n\n0.6.46 (2020-09-21)\n-------------------\n* Add ``ricecooker.utils.youtube``, containing utilities for handling and caching YouTube videos and playlists.\n* Improve validation of ``m`` and ``n`` mastery model values in ``ExerciseNode``.\n\n\n0.6.45 (2020-07-25)\n-------------------\n* Remove SushiBar remote monitoring and remote command functionality\n\n\n0.6.44 (2020-07-16)\n-------------------\n* Documentation overhaul and refresh, see `ricecooker.readthedocs.io <https://ricecooker.readthedocs.io/>`__\n* Add support for specifying a Channel tagline\n* Ensure we send ``extra_fields`` data for all node types\n* Make ``--reset`` behavior the default\n* Remove legacy code around ``compatibiliry_mode`` and BaseChef class\n* Improved caching logic (no caching for thumbnails and local paths, always cache youtube downloads)\n* Ensure chefs clean up after run (automatic temp files removal)\n* Added ``save_channel_tree_as_json`` full channel metadata as part of every run.\n* Added support for web content archiving\n* Further improvements to logging\n* Bugfix: deep deterministic cache keys for nested dicts values in ``ffmpeg_settings``\n\n\n0.6.42 (2020-04-10)\n-------------------\n* Added ``--sample N`` command line option. Run script with ``--sample 10`` to\n produce a test version of the channel with 10 randomly selected nodes from\n the full channel. Use this to check transformations are working as expected.\n* Added ``dryrun`` command. Use the command ``./sushichef.py dryrun`` to run the\n chef as normal but skip the step where the files get uploaded to Studio.\n* Added HTTP proxy functionality for YouTubeVideoFile and YouTubeSubtitleFile\n Set the ``PROXY_LIST`` env variable to a ``;``-separated list of ``{ip}:{port}``.\n Ricecooker will detect the presence of the ``PROXY_LIST`` and use it when\n accessing resources via YoutubeDL. Alternatively, set ``USEPROXY`` env var\n to use a list of free proxy servers, which are very slow and not reliable.\n* Improved colored logging functionality and customizability of logging output.\n\n\n0.6.40 (2020-02-07)\n-------------------\n* Changed default behaviour to upload the staging tree instead of the main tree\n* Added ``--deploy`` flag to reproduce old bahavior (upload to main tree)\n* Added thumbnail generating methods for audio, HTML5, PDF, and ePub nodes.\n Set the ``derive_thumbnail=True`` when creating the Node instance, or pass the\n command line argument ``--thumbnails`` to generate thumbnails for all nodes.\n Note: automatic thumbnail generation will only work if ``thumbnail`` is None.\n\n\n0.6.38 (2019-12-27)\n-------------------\n* Added support the ``h5p`` content kind and ``h5p`` file type\n* Removed monkey-patching of ``localStorage`` and ``document.cookie``\n in the helper method ``download_static_assets``\n* Added validation logic for tags\n* Improved error reporting\n\n\n0.6.36 (2019-09-25)\n-------------------\n* Added support for tags using the ``JsonChef`` workflow\n* Added validation step to ensure subtitles file are unique for each language code\n* Document new ``SlidesShow`` content kind coming in Kolibri 0.13\n* Added docs with detailed instruction for content upload and update workflows\n* Bugfixes to file extension logic and improved error handling around subtitles\n\n\n0.6.32 (2019-08-01)\n-------------------\n* Updated documentation to use top-level headings\n* Removed support for Python 3.4\n* Removed support for the \"sous chef\" workflow\n\n\n0.6.31 (2019-07-01)\n-------------------\n* Handle more subtitle convertible formats: ``SRT``, ``TTML``, ``SCC``, ``DFXP``, and ``SAMI``\n\n\n0.6.30 (2019-05-01)\n-------------------\n* Updated docs build scripts to make ricecooker docs available on read the docs\n* Added ``corrections`` command line script for making bulk edits to content metadata\n* Added ``StudioApi`` client to support CRUD (created, read, update, delete) Studio actions\n* Added pdf-splitting helper methods (see ``ricecooker/utils/pdf.py``)\n\n\n0.6.23 (2018-11-08)\n-------------------\n* Updated ``le-utils`` and ``pressurcooker`` dependencies to latest version\n* Added support for ePub files (``EPubFile`` s can be added of ``DocumentNode`` s)\n* Added tag support\n* Changed default value for ``STUDIO_URL`` to ``api.studio.learningequality.org``\n* Added ``aggregator`` and ``provider`` fields for content nodes\n* Various bugfixes to image processing in exercises\n* Changed validation logic to use ``self.filename`` to check file format is in ``self.allowed_formats``\n* Added ``is_youtube_subtitle_file_supported_language`` helper function to support importing youtube subs\n* Added ``srt2vtt`` subtitles conversion\n* Added static assets downloader helper method in ``utils.downloader.download_static_assets``\n* Added LineCook chef functions to ``--generate`` CSV from directory structure\n* Fixed the always ``randomize=True`` bug\n* Docs: general content node metadata guidelines\n* Docs: video compression instructions and helper scripts ``convertvideo.bat`` and ``convertvideo.sh``\n\n\n0.6.17 (2018-04-20)\n-------------------\n* Added support for ``role`` attribute on ConentNodes (currently ``coach`` || ``learner``)\n* Update pressurecooker dependency (to catch compression errors)\n* Docs improvements, see https://github.com/learningequality/ricecooker/tree/master/docs\n\n\n0.6.15 (2018-03-06)\n-------------------\n* Added support for non-mp4 video files, with auto-conversion using ffmpeg. See ``git diff b1d15fa 87f2528``\n* Added CSV exercises workflow support to ``LineCook`` chef class\n* Added --nomonitor CLI argument to disable sushibar functionality\n* Defined new ENV variables:\n * PHANTOMJS_PATH: set this to a phantomjs binary (instead of assuming one in node_modules)\n * STUDIO_URL (alias CONTENTWORKSHOP_URL): set to URL of Kolibri Studio server where to upload files\n* Various fixes to support sushi chefs\n* Removed ``minimize_html_css_js`` utility function from ``ricecooker/utils/html.py``\n to remove dependency on ``css_html_js_minify`` and support Py3.4 fully.\n\n\n0.6.9 (2017-11-14)\n------------------\n* Changed default logging level to --verbose\n* Added support for cronjobs scripts via `--cmdsock` (see docs/daemonization.md)\n* Added tools for creating HTML5Zip files in utils/html_writer.py\n* Added utility for downloading HTML with optional js support in utils/downloader.py\n* Added utils/path_builder.py and utils/data_writer.py for creating souschef archives\n (zip archive that contains files in a folder hierarchy + Channel.csv + Content.csv)\n\n\n0.6.7 (2017-10-04)\n------------------\n* Sibling content nodes are now required to have unique source_id\n* The field `copyright_holder` is required for all licenses other than public domain\n\n\n0.6.7 (2017-10-04)\n------------------\n* Sibling content nodes are now required to have unique source_id\n* The field `copyright_holder` is required for all licenses other than public domain\n\n\n0.6.6 (2017-09-29)\n------------------\n* Added `JsonTreeChef` class for creating channels from ricecooker json trees\n* Added `LineCook` chef class to support souschef-based channel workflows\n\n\n0.6.4 (2017-08-31)\n------------------\n* Added `language` attribute for `ContentNode` (string key in internal repr. defined in le-utils)\n* Made `language` a required attribute for ChannelNode\n* Enabled sushibar.learningequality.org progress monitoring by default\n Set SUSHIBAR_URL env. var to control where progress is reported (e.g. http://localhost:8001)\n* Updated le-utils and pressurecooker dependencies to latest\n\n\n0.6.2 (2017-07-07)\n------------------\n* Clarify ricecooker is Python3 only (for now)\n* Use https:// and wss:// for SuhiBar reporting\n\n\n0.6.0 (2017-06-28)\n------------------\n* Remote progress reporting and logging to SushiBar (MVP version)\n* New API based on the SuchiChef classes\n* Support existing old-API chefs in compatibility mode\n\n\n\n0.5.13 (2017-06-15)\n-------------------\n* Last stable release before SushiBar functionality was added\n* Renamed --do-not-activate argument to --stage\n\n\n\n0.1.0 (2016-09-30)\n------------------\n* First release on PyPI.\n\n\n",
"bugtrack_url": null,
"license": "MIT license",
"summary": "API for adding content to the Kolibri content curation server",
"version": "0.7.3",
"project_urls": {
"Homepage": "https://github.com/learningequality/ricecooker"
},
"split_keywords": [
"ricecooker"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "94eebe83f7642dcb8c861500d1b31a2f6b0e72c557680e4be9796dfc1a51e179",
"md5": "e07f5180a5b4e2926a342e88f6495346",
"sha256": "a9862e6099a788c798fdd06eff1c34ce494084dd0ce52ac504cd8b2d51517025"
},
"downloads": -1,
"filename": "ricecooker-0.7.3-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "e07f5180a5b4e2926a342e88f6495346",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.7, <3.11",
"size": 139906,
"upload_time": "2023-07-21T21:19:33",
"upload_time_iso_8601": "2023-07-21T21:19:33.340486Z",
"url": "https://files.pythonhosted.org/packages/94/ee/be83f7642dcb8c861500d1b31a2f6b0e72c557680e4be9796dfc1a51e179/ricecooker-0.7.3-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4dab2069bfb1e32ac7bd72efe4535dc3d5ae7ad66a4fca9c9bd5f394ef8e0c6f",
"md5": "ec93b8a00e832b52f76bcc96141a9c0c",
"sha256": "0680a6478a89eb149ade0ebcf53b8ae7f545df91189051a8647cf4a11e63f250"
},
"downloads": -1,
"filename": "ricecooker-0.7.3.tar.gz",
"has_sig": false,
"md5_digest": "ec93b8a00e832b52f76bcc96141a9c0c",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7, <3.11",
"size": 7747093,
"upload_time": "2023-07-21T21:19:35",
"upload_time_iso_8601": "2023-07-21T21:19:35.168448Z",
"url": "https://files.pythonhosted.org/packages/4d/ab/2069bfb1e32ac7bd72efe4535dc3d5ae7ad66a4fca9c9bd5f394ef8e0c6f/ricecooker-0.7.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-07-21 21:19:35",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "learningequality",
"github_project": "ricecooker",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "ricecooker"
}
JSON
