.. This README is meant for consumption by humans and pypi. Pypi can render rst files so please do not use Sphinx features.
If you want to learn more about writing documentation, please check out: http://docs.plone.org/about/documentation_styleguide.html
This text does not appear on pypi or github. It is a comment.
========================
collective.elastic.plone
========================
OpenSearch or ElasticSearch Integration for Plone content.
It consists of these parts:
- indexer passing content to a separate running `collective.elastic.ingest <https://github.com/collective/collective.elastic.ingest>`_ service.
- catalog index acting as a proxy to Open-/ElasticSearch, integrates with ZCatalog. I.e. use as drop-in replacement for ``SearchableText`` index.
- custom plugins for ``plone.restapi`` to provide structural information for the ingestion service
- REST API endpoint ``@kitsearch`` accepting Open-/ ElasticSearch query returning results with Plone permission check.
.. contents:: Table of Contents
Installation
============
-------------
Preconditions
-------------
You need a working ``collective.elastic.ingest`` (version 2.x) service running.
This implies a running Redis instance and a running Open- xor ElasticSearch instance.
------------
mxdev/mxmake
------------
Add ``collective.elastic.plone[redis,opensearch]>=2.0.0b11`` to your ``requirements.txt`` (alternatively use a ``constraints.txt`` for version pinning).
The extra requirements are needed for the queue server and index server used and may vary, see below.
Alternatively add it to your ``pyproject.toml`` as dependencies (or in case of legacy code to ``setup.[py|cfg]``).
Provide and *source* an environments variable file (i.e. `.env`) in your backend directory before Plone startup with::
export INDEX_SERVER=localhost:9200
export INDEX_USE_SSL=1
export INDEX_OPENSEARCH=1
export INDEX_LOGIN=admin
export INDEX_PASSWORD=admin
export INDEX_NAME=plone
export CELERY_BROKER=redis://localhost:6379/0
--------
Buildout
--------
Install ``collective.elastic.plone[redis,opensearch]`` by adding it to your buildout.
The extra requirements are needed for the queue server and index server used may vary, see below.
Environment may vary too.
Also, see below.
::
[buildout]
# ...
eggs =
# ...
collective.elastic.plone[redis,opensearch]
environment-vars +=
INDEX_SERVER=localhost:9200
INDEX_USE_SSL=1
INDEX_OPENSEARCH=1
INDEX_LOGIN=admin
INDEX_PASSWORD=admin
INDEX_NAME=plone
CELERY_BROKER=redis://localhost:6379/0
[versions]
collective.elastic.plone = 2.0.0
and run ``bin/buildout``
------------------
Extra requirements
------------------
Depending on the queue server and index server used, the extra requirements vary:
- queue server: ``redis`` or ``rabbitmq``.
- index server: ``opensearch`` or ``elasticsearch``.
-------------
After Startup
-------------
After startup you need to install the addon in Plone via the Addons control panel.
This replaces the SearchableText index with the proxy index and a minimal configuration.
Best is to alter the configuration to the projects needs.
To index all content in the catalog, append ``/@@update-elasticsearch`` to the URL of your Plone site.
This queues all content for indexing in ElasticSearch (but not in the ZCatalog).
Alternatively a reindex catalog (in ZMI under advanced tab) works too.
New or modified content is queued for indexing automatically.
--------------
Volto Frontend
--------------
The proxy index works out of the box in Volto.
However, in Volto a direct (and much faster) search is possible by using the ``@kitsearch`` endpoint, bypassing the catalog.
The endpoint takes a native Open-/ ElasticSearch query and returns the results with Plone permission check.
The Volto add-on `volto-searchkit-block <https://github.com/rohberg/volto-searchkit-block/>`_ (based on `react-searchkit <https://www.npmjs.com/package/react-searchkit>`_) provides a configurable block using this endpoint.
Remark:
For security reasons, in collective.elastic.plone 2.0.0 the ``@kitsearch`` endpoint always overrides any "API URL" and "API index" settings with the configured values from the environment.
Configuration
=============
Global configuration is done via environment variables.
Each catalog proxy-index has its distinct JSON configuration.
-----------
Environment
-----------
Environment variables are:
INDEX_SERVER
The URL of the ElasticSearch or OpenSearch server.
Default: localhost:9200
INDEX_NAME
The name of the index to use at the ElasticSearch or OpenSearch service.
Default: plone
INDEX_USE_SSL
Whether to use a secure connection or not.
Default: 0
INDEX_OPENSEARCH
Whether to use OpenSearch or ElasticSearch.
Default: 1
INDEX_LOGIN
Username for the ElasticSearch 8+ or OpenSearch 2 server.
For the Plone addon read access is enough.
Default: admin
INDEX_PASSWORD
Password of the above user
Default: admin
CELERY_BROKER
The broker URL for Celery.
See `docs.celeryq.dev <https://docs.celeryq.dev/>`_ for details.
Default: `redis://localhost:6379/0`
-----------
Proxy-index
-----------
Through-the-web, the proxy-index can be configured in the Zope Management Interface (ZMI) under ``portal_catalog``, then click on the ``ElasticSearchProxyIndex`` (i.e. ``SearchableText``).
In the file system it can be configured as any other index in the ``portal_catalog`` tool using a GenericSetup profile and placing a ``catalog.xml`` file in there.
The index configuration looks like so:
.. code-block:: xml
<index meta_type="ElasticSearchProxyIndex"
name="SearchableText"
>
<querytemplate>
{
"query": {
"multi_match": {
"query": "{{keys[0]}}",
"fields": [
"title*^1.9",
"description*^1.5",
"text.data*^1.2",
"blocks_plaintext*^1.2"
"file__extracted.content*"
],
"analyzer": "{{analyzer}}","operator": "or",
"fuzziness": "AUTO",
"prefix_length": 1,
"type": "most_fields",
"minimum_should_match": "75%"
}
}
}
</querytemplate>
</index>
It uses Jinja2 templates to inject the search term into the query.
Available variables are:
``keys``
a list of search terms, usually just one.
``language``
the `current language <https://6.docs.plone.org/plone.api/portal.html#get-current-language>`_ of the portal.
``analyzer``
the name of the analyzer for the query based on the language.
This is hardcoded by now. If there is no analyzer for the language, the ``standard`` analyzer is used.
The resulting query needs to be a valid `OpenSearch Query DSL <https://opensearch.org/docs/latest/query-dsl/index/>`_ or `ElasticSearch Query DSL <https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html>`_ text.
Source Code
===========
The sources are in a GIT DVCS with its main branches at `github <http://github.com/collective/collective.elastic.plone>`_.
There you can report issue too.
We'd be happy to see many forks and pull-requests to make this addon even better.
Maintainers are `Jens Klein <mailto:jk@kleinundpartner.at>`_, `Peter Holzer <mailto:peter.holzer@agitator.com>`_ and the BlueDynamics Alliance developer team.
We appreciate any contribution and if a release is needed to be done on PyPI, please just contact one of us.
We also offer commercial support if any training, coaching, integration or adaptions are needed.
Releases are done using the Github Release feature and PyPI trusted publishing.
Never use a different release process!
If in doubt ask Jens.
Contributions
=============
Idea and testing: Peter Holzer
Initial concept & code by Jens W. Klein (Github: @jensens)
Contributors:
- Katja Süss (Github: @ksuess)
License
=======
The project is licensed under the GPLv2.
Raw data
{
"_id": null,
"home_page": "",
"name": "collective.elastic.plone",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": "Jens Klein <jk@kleinundpartner.at>",
"keywords": "elasticsearch,opensearch,plone,celery,search,indexer",
"author": "Jens Klein, Katja S\u00fcss, Peter Holzer",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/d4/91/a669ca33d8dfc3aef1d5f91336867e64ef8d73228e070859094cdc3cd8de/collective.elastic.plone-2.0.1.tar.gz",
"platform": null,
"description": ".. This README is meant for consumption by humans and pypi. Pypi can render rst files so please do not use Sphinx features.\n If you want to learn more about writing documentation, please check out: http://docs.plone.org/about/documentation_styleguide.html\n This text does not appear on pypi or github. It is a comment.\n\n========================\ncollective.elastic.plone\n========================\n\nOpenSearch or ElasticSearch Integration for Plone content.\n\nIt consists of these parts:\n\n- indexer passing content to a separate running `collective.elastic.ingest <https://github.com/collective/collective.elastic.ingest>`_ service.\n- catalog index acting as a proxy to Open-/ElasticSearch, integrates with ZCatalog. I.e. use as drop-in replacement for ``SearchableText`` index.\n- custom plugins for ``plone.restapi`` to provide structural information for the ingestion service\n- REST API endpoint ``@kitsearch`` accepting Open-/ ElasticSearch query returning results with Plone permission check.\n\n\n.. contents:: Table of Contents\n\nInstallation\n============\n\n-------------\nPreconditions\n-------------\n\nYou need a working ``collective.elastic.ingest`` (version 2.x) service running.\nThis implies a running Redis instance and a running Open- xor ElasticSearch instance.\n\n------------\nmxdev/mxmake\n------------\n\nAdd ``collective.elastic.plone[redis,opensearch]>=2.0.0b11`` to your ``requirements.txt`` (alternatively use a ``constraints.txt`` for version pinning).\n\nThe extra requirements are needed for the queue server and index server used and may vary, see below.\nAlternatively add it to your ``pyproject.toml`` as dependencies (or in case of legacy code to ``setup.[py|cfg]``).\n\nProvide and *source* an environments variable file (i.e. `.env`) in your backend directory before Plone startup with::\n\n export INDEX_SERVER=localhost:9200\n export INDEX_USE_SSL=1\n export INDEX_OPENSEARCH=1\n export INDEX_LOGIN=admin\n export INDEX_PASSWORD=admin\n export INDEX_NAME=plone\n export CELERY_BROKER=redis://localhost:6379/0\n\n\n--------\nBuildout\n--------\n\nInstall ``collective.elastic.plone[redis,opensearch]`` by adding it to your buildout.\nThe extra requirements are needed for the queue server and index server used may vary, see below.\nEnvironment may vary too.\nAlso, see below.\n\n::\n\n [buildout]\n\n # ...\n\n eggs =\n # ...\n collective.elastic.plone[redis,opensearch]\n\n environment-vars +=\n INDEX_SERVER=localhost:9200\n INDEX_USE_SSL=1\n INDEX_OPENSEARCH=1\n INDEX_LOGIN=admin\n INDEX_PASSWORD=admin\n INDEX_NAME=plone\n CELERY_BROKER=redis://localhost:6379/0\n\n [versions]\n collective.elastic.plone = 2.0.0\n\n\nand run ``bin/buildout``\n\n------------------\nExtra requirements\n------------------\n\nDepending on the queue server and index server used, the extra requirements vary:\n\n- queue server: ``redis`` or ``rabbitmq``.\n- index server: ``opensearch`` or ``elasticsearch``.\n\n\n-------------\nAfter Startup\n-------------\n\nAfter startup you need to install the addon in Plone via the Addons control panel.\nThis replaces the SearchableText index with the proxy index and a minimal configuration.\nBest is to alter the configuration to the projects needs.\n\nTo index all content in the catalog, append ``/@@update-elasticsearch`` to the URL of your Plone site.\nThis queues all content for indexing in ElasticSearch (but not in the ZCatalog).\nAlternatively a reindex catalog (in ZMI under advanced tab) works too.\n\nNew or modified content is queued for indexing automatically.\n\n\n--------------\nVolto Frontend\n--------------\n\nThe proxy index works out of the box in Volto.\n\nHowever, in Volto a direct (and much faster) search is possible by using the ``@kitsearch`` endpoint, bypassing the catalog.\nThe endpoint takes a native Open-/ ElasticSearch query and returns the results with Plone permission check.\n\nThe Volto add-on `volto-searchkit-block <https://github.com/rohberg/volto-searchkit-block/>`_ (based on `react-searchkit <https://www.npmjs.com/package/react-searchkit>`_) provides a configurable block using this endpoint.\n\nRemark:\nFor security reasons, in collective.elastic.plone 2.0.0 the ``@kitsearch`` endpoint always overrides any \"API URL\" and \"API index\" settings with the configured values from the environment.\n\nConfiguration\n=============\n\nGlobal configuration is done via environment variables.\n\nEach catalog proxy-index has its distinct JSON configuration.\n\n-----------\nEnvironment\n-----------\n\nEnvironment variables are:\n\nINDEX_SERVER\n The URL of the ElasticSearch or OpenSearch server.\n\n Default: localhost:9200\n\nINDEX_NAME\n The name of the index to use at the ElasticSearch or OpenSearch service.\n\n Default: plone\n\nINDEX_USE_SSL\n Whether to use a secure connection or not.\n\n Default: 0\n\nINDEX_OPENSEARCH\n Whether to use OpenSearch or ElasticSearch.\n\n Default: 1\n\nINDEX_LOGIN\n Username for the ElasticSearch 8+ or OpenSearch 2 server.\n For the Plone addon read access is enough.\n\n Default: admin\n\nINDEX_PASSWORD\n Password of the above user\n\n Default: admin\n\nCELERY_BROKER\n The broker URL for Celery.\n See `docs.celeryq.dev <https://docs.celeryq.dev/>`_ for details.\n\n Default: `redis://localhost:6379/0`\n\n-----------\nProxy-index\n-----------\n\nThrough-the-web, the proxy-index can be configured in the Zope Management Interface (ZMI) under ``portal_catalog``, then click on the ``ElasticSearchProxyIndex`` (i.e. ``SearchableText``).\n\nIn the file system it can be configured as any other index in the ``portal_catalog`` tool using a GenericSetup profile and placing a ``catalog.xml`` file in there.\nThe index configuration looks like so:\n\n.. code-block:: xml\n\n <index meta_type=\"ElasticSearchProxyIndex\"\n name=\"SearchableText\"\n >\n <querytemplate>\n {\n \"query\": {\n \"multi_match\": {\n \"query\": \"{{keys[0]}}\",\n \"fields\": [\n \"title*^1.9\",\n \"description*^1.5\",\n \"text.data*^1.2\",\n \"blocks_plaintext*^1.2\"\n \"file__extracted.content*\"\n ],\n \"analyzer\": \"{{analyzer}}\",\"operator\": \"or\",\n \"fuzziness\": \"AUTO\",\n \"prefix_length\": 1,\n \"type\": \"most_fields\",\n \"minimum_should_match\": \"75%\"\n }\n }\n }\n </querytemplate>\n </index>\n\nIt uses Jinja2 templates to inject the search term into the query.\nAvailable variables are:\n\n``keys``\n a list of search terms, usually just one.\n\n``language``\n the `current language <https://6.docs.plone.org/plone.api/portal.html#get-current-language>`_ of the portal.\n\n``analyzer``\n the name of the analyzer for the query based on the language.\n This is hardcoded by now. If there is no analyzer for the language, the ``standard`` analyzer is used.\n\nThe resulting query needs to be a valid `OpenSearch Query DSL <https://opensearch.org/docs/latest/query-dsl/index/>`_ or `ElasticSearch Query DSL <https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html>`_ text.\n\n\nSource Code\n===========\n\nThe sources are in a GIT DVCS with its main branches at `github <http://github.com/collective/collective.elastic.plone>`_.\nThere you can report issue too.\n\nWe'd be happy to see many forks and pull-requests to make this addon even better.\n\nMaintainers are `Jens Klein <mailto:jk@kleinundpartner.at>`_, `Peter Holzer <mailto:peter.holzer@agitator.com>`_ and the BlueDynamics Alliance developer team.\nWe appreciate any contribution and if a release is needed to be done on PyPI, please just contact one of us.\nWe also offer commercial support if any training, coaching, integration or adaptions are needed.\n\nReleases are done using the Github Release feature and PyPI trusted publishing.\nNever use a different release process!\nIf in doubt ask Jens.\n\n\nContributions\n=============\n\nIdea and testing: Peter Holzer\n\nInitial concept & code by Jens W. Klein (Github: @jensens)\n\nContributors:\n\n- Katja S\u00fcss (Github: @ksuess)\n\n\nLicense\n=======\n\nThe project is licensed under the GPLv2.\n",
"bugtrack_url": null,
"license": "GPL 2.0",
"summary": "Plone Search Integration: Addon to index content in Open-/ElasticSearch",
"version": "2.0.1",
"project_urls": {
"Changelog": "https://github.com/collective/collective.elastic.plone/blob/main/CHANGES.rst",
"Issues": "https://github.com/collective/collective.elastic.plone/issues",
"PyPI": "https://pypi.python.org/pypi/collective.elastic.plone",
"Source": "https://github.com/collective/collective.elastic.plone"
},
"split_keywords": [
"elasticsearch",
"opensearch",
"plone",
"celery",
"search",
"indexer"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "c80b2aa564476a4de8918969dbb1d6470d375196df26f67535b23337c14611ac",
"md5": "c6794b9c1b1bc91a900b9422944455a1",
"sha256": "8b73dd2ebafc311048d3d8d054540e594935c29c2d84ec86fb4bb73c0d722837"
},
"downloads": -1,
"filename": "collective.elastic.plone-2.0.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c6794b9c1b1bc91a900b9422944455a1",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 41171,
"upload_time": "2024-03-05T10:46:45",
"upload_time_iso_8601": "2024-03-05T10:46:45.984871Z",
"url": "https://files.pythonhosted.org/packages/c8/0b/2aa564476a4de8918969dbb1d6470d375196df26f67535b23337c14611ac/collective.elastic.plone-2.0.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "d491a669ca33d8dfc3aef1d5f91336867e64ef8d73228e070859094cdc3cd8de",
"md5": "f27b1d3a4e1ab9b8e70a6a1855ec6f6a",
"sha256": "ac8dae60bb5a44cf470912e0f2df54a1636eec37c4cea6a4830a492ecf5c04c6"
},
"downloads": -1,
"filename": "collective.elastic.plone-2.0.1.tar.gz",
"has_sig": false,
"md5_digest": "f27b1d3a4e1ab9b8e70a6a1855ec6f6a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 39088,
"upload_time": "2024-03-05T10:46:47",
"upload_time_iso_8601": "2024-03-05T10:46:47.134811Z",
"url": "https://files.pythonhosted.org/packages/d4/91/a669ca33d8dfc3aef1d5f91336867e64ef8d73228e070859094cdc3cd8de/collective.elastic.plone-2.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-03-05 10:46:47",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "collective",
"github_project": "collective.elastic.plone",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"requirements": [],
"tox": true,
"lcname": "collective.elastic.plone"
}
JSON
