==========
Blockstore
==========
|status-badge| |license-badge| |ci-badge|
-------
Purpose
-------
Blockstore is a system for storing versioned, reusable educational content for Open edX.
It is designed as a replacement for `modulestore <https://github.com/openedx/edx-platform/tree/master/xmodule/modulestore>`_. It is meant to be a lower-level service than the modulestore, and is designed around the concept of storing small, reusable pieces of content, rather than large, fixed content structures such as courses. For Open edX, Blockstore is designed to facilitate a much greater level of content re-use than is currently possible, enable new adaptive learning features, and enable delivery of learning content in new ways (not just large traditional courses).
.. list-table:: Comparison
:widths: 20 40 40
:header-rows: 1
* - System
- Modulestore
- Blockstore
* - Goal
- Designed to store courses consisting of a hierarchy of XModules (and later, XBlocks)
- Designed around the concept of storing small, reusable pieces of content which are simply files. Can be used for content libraries, courses, or any other purpose.
* - Stores data in
- MongoDB
- S3 (or similar)
* - Stores XBlock data as
- JSON field data, with "settings" and "content" fields separated in different documents
- `OLX <https://edx.readthedocs.io/projects/edx-open-learning-xml/en/latest/what-is-olx.html>`_
* - Content re-use
- Very limited support
- Built in support
* - Focus
- Includes deeply integrated XModule runtime, increasing complexity
- Not aware of XBlocks; XBlock runtime is implemented `separately <https://github.com/openedx/edx-platform/blob/master/openedx/core/djangoapps/xblock/runtime/blockstore_runtime.py>`_
* - Associated XBlock runtime (LMS)
- `LmsModuleSystem <https://github.com/openedx/edx-platform/blob/db32ff2cdf678fa8edd12c9da76a76eef0478614/lms/djangoapps/lms_xblock/runtime.py#L137>`_
- `BlockstoreXBlockRuntime <https://github.com/openedx/edx-platform/blob/7dc60db1d9832ae9382e08d2a686626995010338/openedx/core/djangoapps/xblock/runtime/blockstore_runtime.py#L28>`_
* - Image/PDF/other asset files used for each XBlock are stored
- In MongoDB at the course level (contentstore/GridFS)
- Alongside each XBlock's OLX file. Associated with the individual XBlock, not the course.
Blockstore was originally developed by Harvard's `LabXchange <https://www.labxchange.org/>`_ and the `Amgen Foundation <https://www.amgen.com/responsibility/amgen-foundation/>`_, along with `edX <https://www.edx.org>`_.
--------------
Current Status
--------------
As of **April 2022** ("master" version of Open edX) or the **Nutmeg** named release of Open edX, Blockstore is included as a django app within Open edX platform. See `moving blockstore into edx-platform <decisions/0002-app-not-service.rst>`_ for an explanation of this. Prior to those versions, Blockstore was only available as an independent application, and was not included in any Open edX installations by default.
Blockstore is used to power the **Content Libraries v2** feature of Open edX. It is *not* directly used for course functionality, nor for the "v1" Content Libraries feature.
--------------
Design Details
--------------
See `DESIGN <DESIGN.rst>`_ for an overview of Blockstore's design as it exists today. See `"Blockstore Design" <https://openedx.atlassian.net/wiki/spaces/AC/pages/737149430/Blockstore+Design>`_ on the wiki for historical context.
---------------
Getting Started
---------------
The easiest way to try out the "Content Libraries v2" feature along with Blockstore is to use the Tutor devstack and
`the library-authoring MFE Tutor plugin <https://github.com/openedx/frontend-app-library-authoring/tree/master/tutor-contrib-library-authoring-mfe#readme>`_. See that plugin's README for details.
-------------------------
Running Integration Tests
-------------------------
Open edX includes some integration tests for Blockstore, which run by default as part of the test suite. To run them manually, from a Studio/CMS shell, run these commands:
.. code::
python -Wd -m pytest --ds=cms.envs.test openedx/core/lib/blockstore_api/ openedx/core/djangolib/tests/test_blockstore_cache.py openedx/core/djangoapps/content_libraries/tests/
python -Wd -m pytest --ds=lms.envs.test openedx/core/lib/blockstore_api/ openedx/core/djangolib/tests/test_blockstore_cache.py openedx/core/djangoapps/content_libraries/tests/
To run these integration tests while using Elasticsearch, add ``EDXAPP_ENABLE_ELASTICSEARCH_FOR_TESTS=1`` to the beginning of the above commands. To run these integration tests while using a specific container's version of Elasticsearch, also add ``EDXAPP_TEST_ELASTICSEARCH_HOST`` with a specific container name on the above commands:
.. code::
EDXAPP_ENABLE_ELASTICSEARCH_FOR_TESTS=1 EDXAPP_TEST_ELASTICSEARCH_HOST=edx.devstack.elasticsearch710 python -Wd -m pytest ...
-------------------
Using in Production
-------------------
By default, blockstore is run as an app inside of Open edX. Enable it using the waffle switch `blockstore.use_blockstore_app_api <https://edx.readthedocs.io/projects/edx-platform-technical/en/latest/featuretoggles.html#featuretoggle-blockstore.use_blockstore_app_api>`_.
If you need to run blockstore as a separate service (e.g. for scalability or performance reasons), you can deploy blockstore in production using `the blockstore ansible role <https://github.com/openedx/configuration/tree/master/playbooks/roles/blockstore>`_.
-------------------------------------------------------
Development in Devstack
-------------------------------------------------------
If you want to develop blockstore as a local python package installed in edx-platform, use the following steps:
#. Prerequisite: Have an Open edX `Devstack <https://github.com/openedx/devstack>`_ properly installed and working. Your devstack must use the Nutmeg release of Open edX (or newer) or be tracking the ``master`` branch of ``edx-platform``.
#. To run blockstore as an app inside of Open edX, enable it by activating the waffle switch `blockstore.use_blockstore_app_api <https://edx.readthedocs.io/projects/edx-platform-technical/en/latest/featuretoggles.html#featuretoggle-blockstore.use_blockstore_app_api>`_ in LMS or Studio admin.
#. Clone this repo in the ``src/`` directory, a sibling directory of your devstack folder.
#. In your devstack directory, run the following commands in order:
.. code::
# bring up the lms +studio containers
make dev.up.large-and-slow
# Bring up the studio shell
make dev.shell.studio
# Unistall the pypi blockstore
pip uninstall openedx-blockstore #You will need to confirm the unistall with "Y".
#Install your local blockstore.
pip install -e /edx/src/blockstore/
exit
# do the same in the LMS
make dev.shell.lms
# Unistall the pypi blockstore
pip uninstall openedx-blockstore #You will need to confirm the unistall with "Y".
#Install your local blockstore.
pip install -e /edx/src/blockstore/
exit
#restart the servers to bring in changes.
make dev.restart-devserver.lms dev.restart-devserver.studio
# you can see that your env is now installed openedx-blockstore from a local directory ``/edx/src/blockstore`` by running:
make dev.shell.lms
pip list
Seeing your changes will sometimes require running make requirements and then restarting the container.
You're probably also going to want to create a collection and organization for using content libraries from the devstack command line, using:
.. code::
# Create a "Collection" that new content libraries / xblocks can be created within:
docker exec -t edx.devstack.lms bash -c "source ~/.bashrc && echo \"from blockstore.apps.bundles.models import Collection; coll, _ = Collection.objects.get_or_create(title='Devstack Content Collection', uuid='11111111-2111-4111-8111-111111111111')\" | ./manage.py lms shell"
# Create an "Organization":
docker exec -t edx.devstack.lms bash -c "source /edx/app/edxapp/edxapp_env && echo \"from organizations.models import Organization; Organization.objects.get_or_create(short_name='DeveloperInc', defaults={'name': 'DeveloperInc', 'active': True})\" | python /edx/app/edxapp/edx-platform/manage.py lms shell"
The Library Authoring MFE will want to know about that collection id `using a ENV variable. <https://github.com/openedx/frontend-app-library-authoring/blob/d590aa2eb54c94b39d94f2ba12a6b458082c2e5e/.env#L19>`_
Then restart Studio and the LMS (``make dev.restart-devserver.lms dev.restart-devserver.studio``).
You then can begin setting up the `library authoring MFE <https://github.com/openedx/frontend-app-library-authoring/>`_ to use the feature of blockstore for content libraries V2.
-------------------------------------------------------
Running and testing as a separate service
-------------------------------------------------------
Blockstore was initially developed as an independently deployed application, which runs in a separate container/proccess from the LMS. It is still possible to run blockstore that way, both in production and development.
To run it as an independent application in development:
#. Prerequisite: Have an Open edX `Devstack <https://github.com/openedx/devstack>`_ properly installed and working. Your devstack must use the Nutmeg release of Open edX (or newer) or be tracking the ``master`` branch of ``edx-platform``.
#. Clone this repo and ``cd`` into it.
#. To start the django development server inside a docker container, run this on
your host machine:
.. code::
make easyserver
Blockstore is now running at http://localhost:18250/ . Now we need to configure Studio/LMS to work with it.
#. Run these commands on your host computer:
.. code::
# Create a service user for the edx-platform to use when authenticating and making API calls
docker exec -t edx.devstack.blockstore bash -c "source ~/.bashrc && echo \"from django.contrib.auth import get_user_model; from rest_framework.authtoken.models import Token; User = get_user_model(); edxapp_user, _ = User.objects.get_or_create(username='edxapp'); Token.objects.get_or_create(user=edxapp_user, key='edxapp-insecure-devstack-key')\" | ./manage.py shell"
# Configure the LMS and Studio to use the key
docker exec -t edx.devstack.lms bash -c "grep BLOCKSTORE_API_AUTH_TOKEN /edx/app/edxapp/edx-platform/lms/envs/private.py || echo BLOCKSTORE_API_AUTH_TOKEN = \'edxapp-insecure-devstack-key\' >> /edx/app/edxapp/edx-platform/lms/envs/private.py"
docker exec -t edx.devstack.studio bash -c "grep BLOCKSTORE_API_AUTH_TOKEN /edx/app/edxapp/edx-platform/cms/envs/private.py || echo BLOCKSTORE_API_AUTH_TOKEN = \'edxapp-insecure-devstack-key\' >> /edx/app/edxapp/edx-platform/cms/envs/private.py"
# Create a "Collection" that new content libraries / xblocks can be created within:
docker exec -t edx.devstack.blockstore bash -c "source ~/.bashrc && echo \"from blockstore.apps.bundles.models import Collection; coll, _ = Collection.objects.get_or_create(title='Devstack Content Collection', uuid='11111111-2111-4111-8111-111111111111')\" | ./manage.py shell"
# Create an "Organization":
docker exec -t edx.devstack.lms bash -c "source /edx/app/edxapp/edxapp_env && echo \"from organizations.models import Organization; Organization.objects.get_or_create(short_name='DeveloperInc', defaults={'name': 'DeveloperInc', 'active': True})\" | python /edx/app/edxapp/edx-platform/manage.py lms shell"
Then restart Studio and the LMS (``make dev.restart-devserver.lms dev.restart-devserver.studio``).
#. Now you should be able to use Blockstore in Studio.
To edit Blockstore content libraries in Studio, you'll need to install either `the Content Libraries v2 Frontend <https://github.com/openedx/frontend-app-library-authoring/>`_ or `Ramshackle <https://github.com/open-craft/ramshackle/>`_. Alternatively, you can use the `Content Libraries v2 REST API <https://github.com/openedx/edx-platform/blob/master/openedx/core/djangoapps/content_libraries/urls.py>`_ to create content programmatically.
To use Blockstore library content in a course, open your course in Studio. In its advanced settings, add ``library_sourced`` to the list of "advanced block types". In the "Unit Edit View" in Studio, find the green "Add New Component" buttons. Click Advanced > Library Sourced Content. Edit the new Library Sourced Content XBlock to enter the XBlock ID of the library content that you'd like to use. It should be similar to ``lb:DeveloperInc:MyLibrary:1`` (note: ``lb:`` is short for "Library Block" and should not be confused with the ``lib:`` prefix used to identify a library).
#. Optional: To log in to Blockstore in your web browser directly, you'll need to configure SSO with your devstack. Most people won't need to do this, but it's helpful for debugging or development.
#. Go to http://localhost:18000/admin/oauth2_provider/application/ and add a new application
#. Set "Client id" to ``blockstore-sso-key``
#. Set "Redirect uris" to ``http://localhost:18250/complete/edx-oauth2/``
#. Set "Client type" to "Confidential"
#. Set "Authorization grant type" to "Authorization code"
#. Set "Name" to ``blockstore-sso``
#. Check "Skip authorization"
#. Press "Save and continue editing"
#. Go to http://localhost:18000/admin/oauth_dispatch/applicationaccess/
#. Click "Add Application Access +", choose Application: ``blockstore-sso`` and set Scopes to ``user_id``, then hit "Save"
#. Copy ``blockstore/settings/private.py.example`` to ``blockstore/settings/private.py``
#. In ``private.py``, set ``SOCIAL_AUTH_EDX_OAUTH2_SECRET`` to the random "Client secret" value.
#. Now you can login at http://localhost:18250/login/
#. Optional: If running an Open edX devstack under a project name different
than the default (support for which was introduced
[here](https://github.com/openedx/devstack/pull/532)), simply export
``OPENEDX_PROJECT_NAME`` and substitute the container names in the commands
above accordingly.
#. Optional: to run the unit tests in this mode:
#. Get into the blockstore container: ``make blockstore-shell``
#. And then run ``make test``
#. Optional: to run the integration tests in this mode:
Open edX includes some integration tests for Blockstore. To run them with a separate blockstore instance, first start an isolated test version of blockstore by running ``make testserver`` from the ``blockstore`` repo root directory on your host computer. Then, from ``make dev.shell.studio``, run these commands:
#. ``EDXAPP_RUN_BLOCKSTORE_TESTS=1 python -Wd -m pytest --ds=cms.envs.test openedx/core/lib/blockstore_api/ openedx/core/djangolib/tests/test_blockstore_cache.py openedx/core/djangoapps/content_libraries/tests/``
#. ``EDXAPP_RUN_BLOCKSTORE_TESTS=1 python -Wd -m pytest --ds=lms.envs.test openedx/core/lib/blockstore_api/ openedx/core/djangolib/tests/test_blockstore_cache.py openedx/core/djangoapps/content_libraries/tests/``
------------
Getting Help
------------
Ask questions and discuss this project on `Slack <https://openedx.slack.com/messages/general/>`_ or the `Open edX Community Discussion Forum <https://discuss.openedx.org/>`_.
------------
Contributing
------------
Contributions are welcome. Please read `How To Contribute <https://github.com/openedx/.github/blob/master/CONTRIBUTING.md>`_ for details.
----------------------------
The Open edX Code of Conduct
----------------------------
All community members are expected to follow the `Open edX Code of Conduct`_.
.. _Open edX Code of Conduct: https://openedx.org/code-of-conduct/
------
People
------
The assigned maintainers for this component and other project details may be
found in `Backstage`_. Backstage pulls this data from the ``catalog-info.yaml``
file in this repo.
.. _Backstage: https://backstage.openedx.org/catalog/default/component/blockstore
-------------------------
Reporting Security Issues
-------------------------
Please do not report security issues in public. Please email security@edx.org.
.. |ci-badge| image:: https://github.com/openedx/blockstore/workflows/CI/badge.svg?branch=master
:target: https://github.com/openedx/blockstore/actions
:alt: Test suite status
.. |status-badge| image:: https://img.shields.io/badge/Status-Maintained-brightgreen
:alt: Maintained
.. |license-badge| image:: https://img.shields.io/github/license/openedx/blockstore.svg
:target: https://github.com/openedx/blockstore/blob/master/LICENSE
:alt: License
Change Log
----------
..
All enhancements and patches to blockstore will be documented
in this file. It adheres to the structure of https://keepachangelog.com/ ,
but in reStructuredText instead of Markdown (for ease of incorporation into
Sphinx documentation and the PyPI description).
This project adheres to Semantic Versioning (https://semver.org/).
.. There should always be an "Unreleased" section for changes pending release.
Unreleased
~~~~~~~~~~
*
[1.4.0] - 2023-08-07
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Added
_______
* Added support for Django 4.2
[1.3.1] - 2023-03-06
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Changed
_______
* Adds migration file that was missing from v1.3.0.
[1.3.0] - 2023-02-06
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Added
_____
* Adds support for installing this package as a wheel (``pip install openedx-blockstore``) rather than having to use editable mode (``pip install -e ./blockstore``).
Changed
_______
* Fixes a bug where a new bundle's collection could not be set via Django admin.
* Various configuration and build tooling fixes. See commit log of 1.2.0...1.3.0 for full details.
[1.2.0] - 2021-01-25
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Added
_____
* Adds API unit tests to improve test coverage.
Changed
_______
* Updates the Python API to use the models directly.
[1.1.0] - 2021-10-25
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Added
_____
* Move apps/api to apps/rest_api.
* Copy edx-platform/openedx/core/lib/blockstore_api to blockstore/apps/api.
This code has been copied over so it is easier to review the Python API
implementation in development.
* Make code into an installable package.
Raw data
{
"_id": null,
"home_page": "https://github.com/openedx/blockstore",
"name": "openedx-blockstore",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "Python edx",
"author": "",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/62/cc/7441b42e0f7379a5f65d829117a1e7e151f792133185b1a583e839a7d6ba/openedx-blockstore-1.4.0.tar.gz",
"platform": null,
"description": "==========\nBlockstore\n==========\n\n|status-badge| |license-badge| |ci-badge|\n\n-------\nPurpose\n-------\n\nBlockstore is a system for storing versioned, reusable educational content for Open edX.\n\nIt is designed as a replacement for `modulestore <https://github.com/openedx/edx-platform/tree/master/xmodule/modulestore>`_. It is meant to be a lower-level service than the modulestore, and is designed around the concept of storing small, reusable pieces of content, rather than large, fixed content structures such as courses. For Open edX, Blockstore is designed to facilitate a much greater level of content re-use than is currently possible, enable new adaptive learning features, and enable delivery of learning content in new ways (not just large traditional courses).\n\n.. list-table:: Comparison\n :widths: 20 40 40\n :header-rows: 1\n\n * - System\n - Modulestore\n - Blockstore\n * - Goal\n - Designed to store courses consisting of a hierarchy of XModules (and later, XBlocks)\n - Designed around the concept of storing small, reusable pieces of content which are simply files. Can be used for content libraries, courses, or any other purpose.\n * - Stores data in\n - MongoDB\n - S3 (or similar)\n * - Stores XBlock data as\n - JSON field data, with \"settings\" and \"content\" fields separated in different documents\n - `OLX <https://edx.readthedocs.io/projects/edx-open-learning-xml/en/latest/what-is-olx.html>`_\n * - Content re-use\n - Very limited support\n - Built in support\n * - Focus\n - Includes deeply integrated XModule runtime, increasing complexity\n - Not aware of XBlocks; XBlock runtime is implemented `separately <https://github.com/openedx/edx-platform/blob/master/openedx/core/djangoapps/xblock/runtime/blockstore_runtime.py>`_\n * - Associated XBlock runtime (LMS)\n - `LmsModuleSystem <https://github.com/openedx/edx-platform/blob/db32ff2cdf678fa8edd12c9da76a76eef0478614/lms/djangoapps/lms_xblock/runtime.py#L137>`_\n - `BlockstoreXBlockRuntime <https://github.com/openedx/edx-platform/blob/7dc60db1d9832ae9382e08d2a686626995010338/openedx/core/djangoapps/xblock/runtime/blockstore_runtime.py#L28>`_\n * - Image/PDF/other asset files used for each XBlock are stored\n - In MongoDB at the course level (contentstore/GridFS)\n - Alongside each XBlock's OLX file. Associated with the individual XBlock, not the course.\n\nBlockstore was originally developed by Harvard's `LabXchange <https://www.labxchange.org/>`_ and the `Amgen Foundation <https://www.amgen.com/responsibility/amgen-foundation/>`_, along with `edX <https://www.edx.org>`_.\n\n--------------\nCurrent Status\n--------------\n\nAs of **April 2022** (\"master\" version of Open edX) or the **Nutmeg** named release of Open edX, Blockstore is included as a django app within Open edX platform. See `moving blockstore into edx-platform <decisions/0002-app-not-service.rst>`_ for an explanation of this. Prior to those versions, Blockstore was only available as an independent application, and was not included in any Open edX installations by default.\n\nBlockstore is used to power the **Content Libraries v2** feature of Open edX. It is *not* directly used for course functionality, nor for the \"v1\" Content Libraries feature.\n\n--------------\nDesign Details\n--------------\n\nSee `DESIGN <DESIGN.rst>`_ for an overview of Blockstore's design as it exists today. See `\"Blockstore Design\" <https://openedx.atlassian.net/wiki/spaces/AC/pages/737149430/Blockstore+Design>`_ on the wiki for historical context.\n\n---------------\nGetting Started\n---------------\n\nThe easiest way to try out the \"Content Libraries v2\" feature along with Blockstore is to use the Tutor devstack and\n`the library-authoring MFE Tutor plugin <https://github.com/openedx/frontend-app-library-authoring/tree/master/tutor-contrib-library-authoring-mfe#readme>`_. See that plugin's README for details.\n\n-------------------------\nRunning Integration Tests\n-------------------------\n\nOpen edX includes some integration tests for Blockstore, which run by default as part of the test suite. To run them manually, from a Studio/CMS shell, run these commands:\n\n.. code::\n\n python -Wd -m pytest --ds=cms.envs.test openedx/core/lib/blockstore_api/ openedx/core/djangolib/tests/test_blockstore_cache.py openedx/core/djangoapps/content_libraries/tests/\n python -Wd -m pytest --ds=lms.envs.test openedx/core/lib/blockstore_api/ openedx/core/djangolib/tests/test_blockstore_cache.py openedx/core/djangoapps/content_libraries/tests/\n\nTo run these integration tests while using Elasticsearch, add ``EDXAPP_ENABLE_ELASTICSEARCH_FOR_TESTS=1`` to the beginning of the above commands. To run these integration tests while using a specific container's version of Elasticsearch, also add ``EDXAPP_TEST_ELASTICSEARCH_HOST`` with a specific container name on the above commands:\n\n.. code::\n\n EDXAPP_ENABLE_ELASTICSEARCH_FOR_TESTS=1 EDXAPP_TEST_ELASTICSEARCH_HOST=edx.devstack.elasticsearch710 python -Wd -m pytest ...\n\n-------------------\nUsing in Production\n-------------------\n\nBy default, blockstore is run as an app inside of Open edX. Enable it using the waffle switch `blockstore.use_blockstore_app_api <https://edx.readthedocs.io/projects/edx-platform-technical/en/latest/featuretoggles.html#featuretoggle-blockstore.use_blockstore_app_api>`_.\n\nIf you need to run blockstore as a separate service (e.g. for scalability or performance reasons), you can deploy blockstore in production using `the blockstore ansible role <https://github.com/openedx/configuration/tree/master/playbooks/roles/blockstore>`_.\n\n-------------------------------------------------------\nDevelopment in Devstack\n-------------------------------------------------------\n\nIf you want to develop blockstore as a local python package installed in edx-platform, use the following steps:\n\n#. Prerequisite: Have an Open edX `Devstack <https://github.com/openedx/devstack>`_ properly installed and working. Your devstack must use the Nutmeg release of Open edX (or newer) or be tracking the ``master`` branch of ``edx-platform``.\n\n#. To run blockstore as an app inside of Open edX, enable it by activating the waffle switch `blockstore.use_blockstore_app_api <https://edx.readthedocs.io/projects/edx-platform-technical/en/latest/featuretoggles.html#featuretoggle-blockstore.use_blockstore_app_api>`_ in LMS or Studio admin.\n\n#. Clone this repo in the ``src/`` directory, a sibling directory of your devstack folder.\n\n#. In your devstack directory, run the following commands in order:\n\n .. code::\n\n # bring up the lms +studio containers\n make dev.up.large-and-slow\n # Bring up the studio shell\n make dev.shell.studio\n # Unistall the pypi blockstore\n pip uninstall openedx-blockstore #You will need to confirm the unistall with \"Y\".\n #Install your local blockstore.\n pip install -e /edx/src/blockstore/ \n exit\n\n # do the same in the LMS\n make dev.shell.lms\n # Unistall the pypi blockstore\n pip uninstall openedx-blockstore #You will need to confirm the unistall with \"Y\".\n #Install your local blockstore.\n pip install -e /edx/src/blockstore/ \n exit\n\n #restart the servers to bring in changes.\n make dev.restart-devserver.lms dev.restart-devserver.studio\n\n # you can see that your env is now installed openedx-blockstore from a local directory ``/edx/src/blockstore`` by running:\n make dev.shell.lms\n pip list\n\n\nSeeing your changes will sometimes require running make requirements and then restarting the container.\n\nYou're probably also going to want to create a collection and organization for using content libraries from the devstack command line, using:\n\n .. code::\n\n # Create a \"Collection\" that new content libraries / xblocks can be created within:\n docker exec -t edx.devstack.lms bash -c \"source ~/.bashrc && echo \\\"from blockstore.apps.bundles.models import Collection; coll, _ = Collection.objects.get_or_create(title='Devstack Content Collection', uuid='11111111-2111-4111-8111-111111111111')\\\" | ./manage.py lms shell\"\n # Create an \"Organization\":\n docker exec -t edx.devstack.lms bash -c \"source /edx/app/edxapp/edxapp_env && echo \\\"from organizations.models import Organization; Organization.objects.get_or_create(short_name='DeveloperInc', defaults={'name': 'DeveloperInc', 'active': True})\\\" | python /edx/app/edxapp/edx-platform/manage.py lms shell\"\n\nThe Library Authoring MFE will want to know about that collection id `using a ENV variable. <https://github.com/openedx/frontend-app-library-authoring/blob/d590aa2eb54c94b39d94f2ba12a6b458082c2e5e/.env#L19>`_\n\nThen restart Studio and the LMS (``make dev.restart-devserver.lms dev.restart-devserver.studio``).\n\nYou then can begin setting up the `library authoring MFE <https://github.com/openedx/frontend-app-library-authoring/>`_ to use the feature of blockstore for content libraries V2.\n\n-------------------------------------------------------\nRunning and testing as a separate service\n-------------------------------------------------------\n\nBlockstore was initially developed as an independently deployed application, which runs in a separate container/proccess from the LMS. It is still possible to run blockstore that way, both in production and development.\n\nTo run it as an independent application in development:\n\n#. Prerequisite: Have an Open edX `Devstack <https://github.com/openedx/devstack>`_ properly installed and working. Your devstack must use the Nutmeg release of Open edX (or newer) or be tracking the ``master`` branch of ``edx-platform``.\n\n#. Clone this repo and ``cd`` into it.\n\n#. To start the django development server inside a docker container, run this on\n your host machine:\n\n .. code::\n\n make easyserver\n\n Blockstore is now running at http://localhost:18250/ . Now we need to configure Studio/LMS to work with it.\n\n#. Run these commands on your host computer:\n\n .. code::\n\n # Create a service user for the edx-platform to use when authenticating and making API calls\n docker exec -t edx.devstack.blockstore bash -c \"source ~/.bashrc && echo \\\"from django.contrib.auth import get_user_model; from rest_framework.authtoken.models import Token; User = get_user_model(); edxapp_user, _ = User.objects.get_or_create(username='edxapp'); Token.objects.get_or_create(user=edxapp_user, key='edxapp-insecure-devstack-key')\\\" | ./manage.py shell\"\n # Configure the LMS and Studio to use the key\n docker exec -t edx.devstack.lms bash -c \"grep BLOCKSTORE_API_AUTH_TOKEN /edx/app/edxapp/edx-platform/lms/envs/private.py || echo BLOCKSTORE_API_AUTH_TOKEN = \\'edxapp-insecure-devstack-key\\' >> /edx/app/edxapp/edx-platform/lms/envs/private.py\"\n docker exec -t edx.devstack.studio bash -c \"grep BLOCKSTORE_API_AUTH_TOKEN /edx/app/edxapp/edx-platform/cms/envs/private.py || echo BLOCKSTORE_API_AUTH_TOKEN = \\'edxapp-insecure-devstack-key\\' >> /edx/app/edxapp/edx-platform/cms/envs/private.py\"\n # Create a \"Collection\" that new content libraries / xblocks can be created within:\n docker exec -t edx.devstack.blockstore bash -c \"source ~/.bashrc && echo \\\"from blockstore.apps.bundles.models import Collection; coll, _ = Collection.objects.get_or_create(title='Devstack Content Collection', uuid='11111111-2111-4111-8111-111111111111')\\\" | ./manage.py shell\"\n # Create an \"Organization\":\n docker exec -t edx.devstack.lms bash -c \"source /edx/app/edxapp/edxapp_env && echo \\\"from organizations.models import Organization; Organization.objects.get_or_create(short_name='DeveloperInc', defaults={'name': 'DeveloperInc', 'active': True})\\\" | python /edx/app/edxapp/edx-platform/manage.py lms shell\"\n\n Then restart Studio and the LMS (``make dev.restart-devserver.lms dev.restart-devserver.studio``).\n\n#. Now you should be able to use Blockstore in Studio.\n\n To edit Blockstore content libraries in Studio, you'll need to install either `the Content Libraries v2 Frontend <https://github.com/openedx/frontend-app-library-authoring/>`_ or `Ramshackle <https://github.com/open-craft/ramshackle/>`_. Alternatively, you can use the `Content Libraries v2 REST API <https://github.com/openedx/edx-platform/blob/master/openedx/core/djangoapps/content_libraries/urls.py>`_ to create content programmatically.\n\n To use Blockstore library content in a course, open your course in Studio. In its advanced settings, add ``library_sourced`` to the list of \"advanced block types\". In the \"Unit Edit View\" in Studio, find the green \"Add New Component\" buttons. Click Advanced > Library Sourced Content. Edit the new Library Sourced Content XBlock to enter the XBlock ID of the library content that you'd like to use. It should be similar to ``lb:DeveloperInc:MyLibrary:1`` (note: ``lb:`` is short for \"Library Block\" and should not be confused with the ``lib:`` prefix used to identify a library).\n\n#. Optional: To log in to Blockstore in your web browser directly, you'll need to configure SSO with your devstack. Most people won't need to do this, but it's helpful for debugging or development.\n\n #. Go to http://localhost:18000/admin/oauth2_provider/application/ and add a new application\n #. Set \"Client id\" to ``blockstore-sso-key``\n #. Set \"Redirect uris\" to ``http://localhost:18250/complete/edx-oauth2/``\n #. Set \"Client type\" to \"Confidential\"\n #. Set \"Authorization grant type\" to \"Authorization code\"\n #. Set \"Name\" to ``blockstore-sso``\n #. Check \"Skip authorization\"\n #. Press \"Save and continue editing\"\n #. Go to http://localhost:18000/admin/oauth_dispatch/applicationaccess/\n #. Click \"Add Application Access +\", choose Application: ``blockstore-sso`` and set Scopes to ``user_id``, then hit \"Save\"\n #. Copy ``blockstore/settings/private.py.example`` to ``blockstore/settings/private.py``\n #. In ``private.py``, set ``SOCIAL_AUTH_EDX_OAUTH2_SECRET`` to the random \"Client secret\" value.\n #. Now you can login at http://localhost:18250/login/\n\n#. Optional: If running an Open edX devstack under a project name different\n than the default (support for which was introduced\n [here](https://github.com/openedx/devstack/pull/532)), simply export\n ``OPENEDX_PROJECT_NAME`` and substitute the container names in the commands\n above accordingly.\n\n#. Optional: to run the unit tests in this mode:\n\n #. Get into the blockstore container: ``make blockstore-shell``\n #. And then run ``make test``\n\n#. Optional: to run the integration tests in this mode:\n\n Open edX includes some integration tests for Blockstore. To run them with a separate blockstore instance, first start an isolated test version of blockstore by running ``make testserver`` from the ``blockstore`` repo root directory on your host computer. Then, from ``make dev.shell.studio``, run these commands:\n\n #. ``EDXAPP_RUN_BLOCKSTORE_TESTS=1 python -Wd -m pytest --ds=cms.envs.test openedx/core/lib/blockstore_api/ openedx/core/djangolib/tests/test_blockstore_cache.py openedx/core/djangoapps/content_libraries/tests/``\n #. ``EDXAPP_RUN_BLOCKSTORE_TESTS=1 python -Wd -m pytest --ds=lms.envs.test openedx/core/lib/blockstore_api/ openedx/core/djangolib/tests/test_blockstore_cache.py openedx/core/djangoapps/content_libraries/tests/``\n\n------------\nGetting Help\n------------\n\nAsk questions and discuss this project on `Slack <https://openedx.slack.com/messages/general/>`_ or the `Open edX Community Discussion Forum <https://discuss.openedx.org/>`_.\n\n------------\nContributing\n------------\n\nContributions are welcome. Please read `How To Contribute <https://github.com/openedx/.github/blob/master/CONTRIBUTING.md>`_ for details.\n\n----------------------------\nThe Open edX Code of Conduct\n----------------------------\n\nAll community members are expected to follow the `Open edX Code of Conduct`_.\n\n.. _Open edX Code of Conduct: https://openedx.org/code-of-conduct/\n\n------\nPeople\n------\n\nThe assigned maintainers for this component and other project details may be\nfound in `Backstage`_. Backstage pulls this data from the ``catalog-info.yaml``\nfile in this repo.\n\n.. _Backstage: https://backstage.openedx.org/catalog/default/component/blockstore\n\n\n-------------------------\nReporting Security Issues\n-------------------------\n\nPlease do not report security issues in public. Please email security@edx.org.\n\n\n.. |ci-badge| image:: https://github.com/openedx/blockstore/workflows/CI/badge.svg?branch=master\n :target: https://github.com/openedx/blockstore/actions\n :alt: Test suite status\n\n.. |status-badge| image:: https://img.shields.io/badge/Status-Maintained-brightgreen\n :alt: Maintained\n\n.. |license-badge| image:: https://img.shields.io/github/license/openedx/blockstore.svg\n :target: https://github.com/openedx/blockstore/blob/master/LICENSE\n :alt: License\n\n\nChange Log\n----------\n\n..\n All enhancements and patches to blockstore will be documented\n in this file. It adheres to the structure of https://keepachangelog.com/ ,\n but in reStructuredText instead of Markdown (for ease of incorporation into\n Sphinx documentation and the PyPI description).\n\n This project adheres to Semantic Versioning (https://semver.org/).\n\n.. There should always be an \"Unreleased\" section for changes pending release.\n\nUnreleased\n~~~~~~~~~~\n\n*\n\n[1.4.0] - 2023-08-07\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nAdded\n_______\n\n* Added support for Django 4.2\n\n\n[1.3.1] - 2023-03-06\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nChanged\n_______\n\n* Adds migration file that was missing from v1.3.0.\n\n\n[1.3.0] - 2023-02-06\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nAdded\n_____\n\n* Adds support for installing this package as a wheel (``pip install openedx-blockstore``) rather than having to use editable mode (``pip install -e ./blockstore``).\n\nChanged\n_______\n\n* Fixes a bug where a new bundle's collection could not be set via Django admin.\n* Various configuration and build tooling fixes. See commit log of 1.2.0...1.3.0 for full details.\n\n\n\n[1.2.0] - 2021-01-25\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nAdded\n_____\n\n* Adds API unit tests to improve test coverage.\n\nChanged\n_______\n\n* Updates the Python API to use the models directly.\n\n[1.1.0] - 2021-10-25\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nAdded\n_____\n\n* Move apps/api to apps/rest_api.\n* Copy edx-platform/openedx/core/lib/blockstore_api to blockstore/apps/api.\n This code has been copied over so it is easier to review the Python API\n implementation in development.\n* Make code into an installable package.\n\n\n\n",
"bugtrack_url": null,
"license": "AGPL 3.0",
"summary": "Blockstore is a storage system for learning content in Open edX.",
"version": "1.4.0",
"project_urls": {
"Homepage": "https://github.com/openedx/blockstore"
},
"split_keywords": [
"python",
"edx"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e754aa13c0f774a844ea98828365b68cbe6d2a70a88b154421858333a5c6a115",
"md5": "67dbd8976bea4c8be6de486749ab7ea0",
"sha256": "18e473dc31d24a1367d1c94bc985fb49604a7ac61324b5252dd41c4e5a621456"
},
"downloads": -1,
"filename": "openedx_blockstore-1.4.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "67dbd8976bea4c8be6de486749ab7ea0",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.8",
"size": 83032,
"upload_time": "2023-08-10T10:33:07",
"upload_time_iso_8601": "2023-08-10T10:33:07.887399Z",
"url": "https://files.pythonhosted.org/packages/e7/54/aa13c0f774a844ea98828365b68cbe6d2a70a88b154421858333a5c6a115/openedx_blockstore-1.4.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "62cc7441b42e0f7379a5f65d829117a1e7e151f792133185b1a583e839a7d6ba",
"md5": "566a93585c13afa5b06e73000f29e1e7",
"sha256": "dd0c995a40c6505d1645b818cd81468ab87ff3e28103ac8d654e8412a0d93991"
},
"downloads": -1,
"filename": "openedx-blockstore-1.4.0.tar.gz",
"has_sig": false,
"md5_digest": "566a93585c13afa5b06e73000f29e1e7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 72103,
"upload_time": "2023-08-10T10:33:10",
"upload_time_iso_8601": "2023-08-10T10:33:10.151540Z",
"url": "https://files.pythonhosted.org/packages/62/cc/7441b42e0f7379a5f65d829117a1e7e151f792133185b1a583e839a7d6ba/openedx-blockstore-1.4.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-10 10:33:10",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "openedx",
"github_project": "blockstore",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"lcname": "openedx-blockstore"
}
JSON
