Introduction
============
``ftw.solr`` integrates the Solr search engine with Plone.
.. IMPORTANT::
Since version 2.0 ftw.solr no longer depends on collective.solr. Instead it
provides it's own Solr integration using Solr's REST API. Version 2.0 is a
complete rewrite and is not compatible with ftw.solr 1.x.
ftw.solr does not try to replace the portal catalog of Plone and does not hook
into the catalog's search function. Instead it provides a search utiltity that
must be used explicitly. It's meant to be used in search forms for fulltext
searches while the portal catalog is still in use for things like navigation
or folder contents. The goal is to get rid off all fulltext indexes
(e.g. ZCTextIndex) in the portal catalog.
ftw.solr requires Apache Solr 7.0 or higher.
Installation
============
Add as dependency
-----------------
Install ``ftw.solr`` by adding it to the list of eggs in your
buildout or by adding it as a dependency of your policy package.
.. code:: rst
[instance]
eggs +=
ftw.solr
Solr installation
-----------------
To install a Solr server with buildout you can use the ``ftw.recipe.solr`` recipe.
.. code::
[solr]
recipe = ftw.recipe.solr
cores = mycore
Configure the Solr connection
-----------------------------
The connections settings for Solr can be configured in ZCML and thus in
buildout. Example::
[instance]
zcml-additional =
<configure xmlns:solr="http://namespaces.plone.org/solr">
<solr:connection host="localhost" port="8983" base="/solr/mycore"/>
</configure>
By default, ``ftw.solr`` will do full text extraction by passing the blob's
filesystem path to the Solr Cell extract handler, assuming that Solr runs on
the same machine and has access to the blob storage.
For setups where this isn't desired, the connection option ``upload_blobs``
can be set to ``true`` in order to make ``ftw.solr`` upload the blobs directly
to the extract handler via HTTP POST::
[instance]
zcml-additional =
<configure xmlns:solr="http://namespaces.plone.org/solr">
<solr:connection host="localhost" port="8983" base="/solr/mycore" upload_blobs="true"/>
</configure>
Run buildout
------------
After running buildout and restarting your instance you can install the ftw.solr
addon in Plone.
Python 3.8 - Plone 5.2
----------------------
Install and run tests locally:
.. code::
python3 -m venv .
./bin/pip install -r requirements.txt
./bin/buildout -c test-plone-5.2.x-py38.cfg
./bin/test
Usage
=====
Get the ``ISolrSearch`` utility and call the search method to get search results
from Solr.
.. code:: python
from ftw.solr.interfaces import ISolrSearch
from zope.component import getUtility
solr = getUtility(ISolrSearch)
resp = solr.search(query=u'SearchableText:foo')
You can get a ``plone.app.contentlisting`` style result by adapting ``IContentListing``:
.. code:: python
from plone.app.contentlisting.interfaces import IContentListing
listing = IContentListing(resp)
Solr Index Maintenance
======================
For indexing Plone content and other maintenance work you can use the ``solr`` Zope command.
Run ``bin/instance solr -h`` for available options.
Clear the Solr index:
.. code::
bin/instance solr clear
Rebuild the complete Solr index:
.. code::
bin/instance solr reindex
Reindex specific indexes:
.. code::
bin/instance solr reindex -i modified created
Synchronize the Solr index with the portal catalog:
.. code::
bin/instance solr sync
Links
=====
- Github: https://github.com/4teamwork/ftw.solr
- Issues: https://github.com/4teamwork/ftw.solr/issues
- Pypi: http://pypi.python.org/pypi/ftw.solr
- Continuous integration: https://jenkins.4teamwork.ch/search?q=ftw.solr
Copyright
=========
This package is copyright by `4teamwork <http://www.4teamwork.ch/>`_.
``ftw.solr`` is licensed under GNU General Public License, version 2.
Changelog
=========
2.13.2 (2024-04-04)
-------------------
- Fix argument parsing of zopectl command if run with Zope2.Startup.zopectl.
[buchi]
2.13.1 (2023-06-20)
-------------------
- Log warning when a search is performed without specifying fl.
[njohner]
- Fix solr sync maintenance command not committing all batches.
[lgraf]
2.13.0 (2023-04-11)
-------------------
- Add support for Plone 5.2 and Python 3.
[buchi]
- Add Python 3.8 and up compatibility.
[mathias.leimgruber]
2.12.1 (2022-08-03)
-------------------
- Make updates_in_post_commit_enabled() helper more robust. [lgraf]
2.12.0 (2022-07-19)
-------------------
- Add option to also check allowedRolesAndUsers when determining items not up to date in Solr. [njohner]
- Add option to execute update commands in post commit hook. [njohner]
2.11.0 (2022-07-04)
-------------------
- Add support for language specific fields. [buchi]
2.10.0 (2021-09-21)
-------------------
- Add flag to force clearing solr even when solr is not enabled. [njohner]
2.9.5 (2021-08-17)
------------------
- Add unrestricted_search method. [njohner]
2.9.4 (2021-07-29)
------------------
- SolrDocument returns None for existing fields with missing value. [njohner]
2.9.3 (2021-04-22)
------------------
- Fix indexing of SearchableText only for objects with primary field. [njohner]
2.9.2 (2021-01-13)
------------------
- Do not ignore None values when indexing data in solr. [njohner]
2.9.1 (2020-12-15)
------------------
- Cleanup extract_commands before sending them to solr. [njohner]
2.9.0 (2020-08-04)
------------------
- Treat docs with no created field as out of sync in `diff`. [deiferni]
2.8.6 (2020-06-02)
------------------
- Log maintenance commands to a dedicated logfile. [lgraf]
- Add option to limit items logged in diff, switch default to only diff 5 items. [deiferni]
- Add helper for creating Solr filters from catalog queries.
[buchi]
2.8.5 (2020-05-06)
------------------
- Make sure maintenance scripts are run as system user. [njohner]
2.8.4 (2020-04-02)
------------------
- Fix update of the modified index. [njohner]
2.8.3 (2020-04-02)
------------------
- Update the modified index to match new rounding of DateTime. [njohner]
2.8.2 (2020-03-27)
------------------
- Fix to_iso8601 prevent rounding errors with DateTime milliseconds above 999500. [deiferni]
- Fix blob extraction for DX items. [mbaechtold]
2.8.1 (2020-02-05)
------------------
- Fix to_iso8601 for years before 1900.
[deiferni]
2.8.0 (2019-12-06)
------------------
- Add support for uploading blobs to extract handler.
[lgraf]
- Add support for Plone 5.0 and 5.1.
[buchi]
2.7.0 (2019-09-16)
------------------
- Add console script for maintenance tasks.
[buchi]
- Add registry setting for enabling/disabling indexing.
[buchi]
2.6.2 (2019-09-12)
------------------
- Avoid sending atomic update commands with null-documents. [lgraf]
2.6.1 (2019-08-26)
------------------
- Fix 'path_depth' indexing handling, when 'path_depth' is not part of the schema. [phgross]
2.6.0 (2019-08-19)
------------------
- Use Solr Cell only for text extraction. The SearchableText is now also indexed
using the update handler which allows atomic updates. Also fixes #137.
[buchi]
- Fix error when trying to delete an object without a unique key. [jone]
2.5.0 (2019-07-02)
------------------
- Make sure to always update 'path_depth' if 'path' is (re)indexed.
[lgraf]
- Handle facet_counts in solr response.
[njohner]
2.4.0 (2019-06-13)
------------------
- Perform soft commits and wait for new searcher by default to allow near real
time (NRT) searching.
[buchi]
- Additionaly index metadata of files using an update command to make them
visible as soon as possible.
[buchi]
- Flush optimize command.
[buchi]
- Add support for indexing path depth.
[buchi]
2.3.2 (2019-04-29)
------------------
- Fix partial reindexing (atomic updates) for fields without a value.
[buchi]
2.3.1 (2018-11-01)
------------------
- Fix reindexObjectSecurity patch to account for special handling of system
roles in allowedRolesAndUsers indexer.
[lgraf]
2.3.0 (2018-10-18)
------------------
- Patch reindexObjectSecurity to optimize indexing of large trees.
[lgraf]
- Fix diff for items with a modification date that has no fractional seconds.
[buchi]
- Provide an error message when raising an AttributeError in SolrDocument.
[buchi]
2.2.0 (2018-09-21)
------------------
- Add option to synchronize Solr with catalog in maintenance view.
[buchi]
2.1.2 (2018-05-15)
------------------
- Set Content-Type 'application/x-www-form-urlencoded' for requests to
/update/extract endpoint to ensure compatibility with Solr 7.3.
[lgraf]
- Maintenance view: Allow reindexing without dooming the transaction.
[lgraf]
2.1.1 (2018-02-20)
------------------
- Make sure we do not pass invalid dates to Solr.
[buchi]
2.1.0 (2018-02-20)
------------------
- Declare dependency on plone.namedfile[blobs].
[lgraf]
- Add diff function to maintenance view to compare UIDs in portal catalog with
UIDs in Solr.
[buchi]
- Fix security filter: Properly quote roles/users with spaces.
[buchi]
- Add option to reindex Solr from a catalog query.
[buchi]
2.0.0 (2018-02-15)
------------------
- Complete rewrite. Not compatible with previous versions.
[buchi]
Raw data
{
"_id": null,
"home_page": "https://pypi.python.org/pypi/ftw.solr",
"name": "ftw.solr",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "Plone Solr",
"author": "Thomas Buchberger",
"author_email": "t.buchberger@4teamwork.ch",
"download_url": "https://files.pythonhosted.org/packages/28/d4/4999864e64143e42a878f2f93b93d001fd1756c2cbeb23e09d11c6b31d26/ftw.solr-2.13.2.tar.gz",
"platform": null,
"description": "Introduction\n============\n\n``ftw.solr`` integrates the Solr search engine with Plone.\n\n.. IMPORTANT::\n Since version 2.0 ftw.solr no longer depends on collective.solr. Instead it\n provides it's own Solr integration using Solr's REST API. Version 2.0 is a\n complete rewrite and is not compatible with ftw.solr 1.x.\n\nftw.solr does not try to replace the portal catalog of Plone and does not hook\ninto the catalog's search function. Instead it provides a search utiltity that\nmust be used explicitly. It's meant to be used in search forms for fulltext\nsearches while the portal catalog is still in use for things like navigation\nor folder contents. The goal is to get rid off all fulltext indexes\n(e.g. ZCTextIndex) in the portal catalog.\n\nftw.solr requires Apache Solr 7.0 or higher.\n\n\nInstallation\n============\n\nAdd as dependency\n-----------------\n\nInstall ``ftw.solr`` by adding it to the list of eggs in your\nbuildout or by adding it as a dependency of your policy package.\n\n.. code:: rst\n\n [instance]\n eggs +=\n ftw.solr\n\n\nSolr installation\n-----------------\n\nTo install a Solr server with buildout you can use the ``ftw.recipe.solr`` recipe.\n\n.. code::\n\n [solr]\n recipe = ftw.recipe.solr\n cores = mycore\n\n\nConfigure the Solr connection\n-----------------------------\n\nThe connections settings for Solr can be configured in ZCML and thus in\nbuildout. Example::\n\n [instance]\n zcml-additional =\n <configure xmlns:solr=\"http://namespaces.plone.org/solr\">\n <solr:connection host=\"localhost\" port=\"8983\" base=\"/solr/mycore\"/>\n </configure>\n\nBy default, ``ftw.solr`` will do full text extraction by passing the blob's\nfilesystem path to the Solr Cell extract handler, assuming that Solr runs on\nthe same machine and has access to the blob storage.\n\nFor setups where this isn't desired, the connection option ``upload_blobs``\ncan be set to ``true`` in order to make ``ftw.solr`` upload the blobs directly\nto the extract handler via HTTP POST::\n\n [instance]\n zcml-additional =\n <configure xmlns:solr=\"http://namespaces.plone.org/solr\">\n <solr:connection host=\"localhost\" port=\"8983\" base=\"/solr/mycore\" upload_blobs=\"true\"/>\n </configure>\n\n\nRun buildout\n------------\n\nAfter running buildout and restarting your instance you can install the ftw.solr\naddon in Plone.\n\n\nPython 3.8 - Plone 5.2\n----------------------\n\nInstall and run tests locally:\n\n.. code::\n\n python3 -m venv .\n ./bin/pip install -r requirements.txt\n ./bin/buildout -c test-plone-5.2.x-py38.cfg\n ./bin/test\n\n\nUsage\n=====\n\nGet the ``ISolrSearch`` utility and call the search method to get search results\nfrom Solr.\n\n.. code:: python\n\n from ftw.solr.interfaces import ISolrSearch\n from zope.component import getUtility\n\n solr = getUtility(ISolrSearch)\n resp = solr.search(query=u'SearchableText:foo')\n\n\nYou can get a ``plone.app.contentlisting`` style result by adapting ``IContentListing``:\n\n.. code:: python\n\n from plone.app.contentlisting.interfaces import IContentListing\n listing = IContentListing(resp)\n\n\nSolr Index Maintenance\n======================\n\nFor indexing Plone content and other maintenance work you can use the ``solr`` Zope command.\nRun ``bin/instance solr -h`` for available options.\n\nClear the Solr index:\n\n.. code::\n\n bin/instance solr clear\n\nRebuild the complete Solr index:\n\n.. code::\n\n bin/instance solr reindex\n\nReindex specific indexes:\n\n.. code::\n\n bin/instance solr reindex -i modified created\n\nSynchronize the Solr index with the portal catalog:\n\n.. code::\n\n bin/instance solr sync\n\n\nLinks\n=====\n\n- Github: https://github.com/4teamwork/ftw.solr\n- Issues: https://github.com/4teamwork/ftw.solr/issues\n- Pypi: http://pypi.python.org/pypi/ftw.solr\n- Continuous integration: https://jenkins.4teamwork.ch/search?q=ftw.solr\n\n\nCopyright\n=========\n\nThis package is copyright by `4teamwork <http://www.4teamwork.ch/>`_.\n\n``ftw.solr`` is licensed under GNU General Public License, version 2.\n\n\nChangelog\n=========\n\n2.13.2 (2024-04-04)\n-------------------\n\n- Fix argument parsing of zopectl command if run with Zope2.Startup.zopectl.\n [buchi]\n\n\n2.13.1 (2023-06-20)\n-------------------\n\n- Log warning when a search is performed without specifying fl.\n [njohner]\n\n- Fix solr sync maintenance command not committing all batches.\n [lgraf]\n\n\n2.13.0 (2023-04-11)\n-------------------\n\n- Add support for Plone 5.2 and Python 3.\n [buchi]\n\n- Add Python 3.8 and up compatibility.\n [mathias.leimgruber]\n\n\n2.12.1 (2022-08-03)\n-------------------\n\n- Make updates_in_post_commit_enabled() helper more robust. [lgraf]\n\n\n2.12.0 (2022-07-19)\n-------------------\n\n- Add option to also check allowedRolesAndUsers when determining items not up to date in Solr. [njohner]\n- Add option to execute update commands in post commit hook. [njohner]\n\n\n2.11.0 (2022-07-04)\n-------------------\n\n- Add support for language specific fields. [buchi]\n\n\n2.10.0 (2021-09-21)\n-------------------\n\n- Add flag to force clearing solr even when solr is not enabled. [njohner]\n\n\n2.9.5 (2021-08-17)\n------------------\n\n- Add unrestricted_search method. [njohner]\n\n\n2.9.4 (2021-07-29)\n------------------\n\n- SolrDocument returns None for existing fields with missing value. [njohner]\n\n\n2.9.3 (2021-04-22)\n------------------\n\n- Fix indexing of SearchableText only for objects with primary field. [njohner]\n\n2.9.2 (2021-01-13)\n------------------\n\n- Do not ignore None values when indexing data in solr. [njohner]\n\n\n2.9.1 (2020-12-15)\n------------------\n\n- Cleanup extract_commands before sending them to solr. [njohner]\n\n\n2.9.0 (2020-08-04)\n------------------\n\n- Treat docs with no created field as out of sync in `diff`. [deiferni]\n\n\n2.8.6 (2020-06-02)\n------------------\n\n- Log maintenance commands to a dedicated logfile. [lgraf]\n- Add option to limit items logged in diff, switch default to only diff 5 items. [deiferni]\n- Add helper for creating Solr filters from catalog queries.\n [buchi]\n\n\n2.8.5 (2020-05-06)\n------------------\n\n- Make sure maintenance scripts are run as system user. [njohner]\n\n\n2.8.4 (2020-04-02)\n------------------\n\n- Fix update of the modified index. [njohner]\n\n\n2.8.3 (2020-04-02)\n------------------\n\n- Update the modified index to match new rounding of DateTime. [njohner]\n\n\n2.8.2 (2020-03-27)\n------------------\n\n- Fix to_iso8601 prevent rounding errors with DateTime milliseconds above 999500. [deiferni]\n- Fix blob extraction for DX items. [mbaechtold]\n\n\n2.8.1 (2020-02-05)\n------------------\n\n- Fix to_iso8601 for years before 1900.\n [deiferni]\n\n\n2.8.0 (2019-12-06)\n------------------\n\n- Add support for uploading blobs to extract handler.\n [lgraf]\n\n- Add support for Plone 5.0 and 5.1.\n [buchi]\n\n\n2.7.0 (2019-09-16)\n------------------\n\n- Add console script for maintenance tasks.\n [buchi]\n\n- Add registry setting for enabling/disabling indexing.\n [buchi]\n\n\n2.6.2 (2019-09-12)\n------------------\n\n- Avoid sending atomic update commands with null-documents. [lgraf]\n\n\n2.6.1 (2019-08-26)\n------------------\n\n- Fix 'path_depth' indexing handling, when 'path_depth' is not part of the schema. [phgross]\n\n\n2.6.0 (2019-08-19)\n------------------\n\n- Use Solr Cell only for text extraction. The SearchableText is now also indexed\n using the update handler which allows atomic updates. Also fixes #137.\n [buchi]\n\n- Fix error when trying to delete an object without a unique key. [jone]\n\n\n2.5.0 (2019-07-02)\n------------------\n\n- Make sure to always update 'path_depth' if 'path' is (re)indexed.\n [lgraf]\n\n- Handle facet_counts in solr response.\n [njohner]\n\n\n2.4.0 (2019-06-13)\n------------------\n\n- Perform soft commits and wait for new searcher by default to allow near real\n time (NRT) searching.\n [buchi]\n\n- Additionaly index metadata of files using an update command to make them\n visible as soon as possible.\n [buchi]\n\n- Flush optimize command.\n [buchi]\n\n- Add support for indexing path depth.\n [buchi]\n\n\n2.3.2 (2019-04-29)\n------------------\n\n- Fix partial reindexing (atomic updates) for fields without a value.\n [buchi]\n\n\n2.3.1 (2018-11-01)\n------------------\n\n- Fix reindexObjectSecurity patch to account for special handling of system\n roles in allowedRolesAndUsers indexer.\n [lgraf]\n\n\n2.3.0 (2018-10-18)\n------------------\n\n- Patch reindexObjectSecurity to optimize indexing of large trees.\n [lgraf]\n\n- Fix diff for items with a modification date that has no fractional seconds.\n [buchi]\n\n- Provide an error message when raising an AttributeError in SolrDocument.\n [buchi]\n\n\n2.2.0 (2018-09-21)\n------------------\n\n- Add option to synchronize Solr with catalog in maintenance view.\n [buchi]\n\n\n2.1.2 (2018-05-15)\n------------------\n\n- Set Content-Type 'application/x-www-form-urlencoded' for requests to\n /update/extract endpoint to ensure compatibility with Solr 7.3.\n [lgraf]\n\n- Maintenance view: Allow reindexing without dooming the transaction.\n [lgraf]\n\n\n2.1.1 (2018-02-20)\n------------------\n\n- Make sure we do not pass invalid dates to Solr.\n [buchi]\n\n\n2.1.0 (2018-02-20)\n------------------\n\n- Declare dependency on plone.namedfile[blobs].\n [lgraf]\n\n- Add diff function to maintenance view to compare UIDs in portal catalog with\n UIDs in Solr.\n [buchi]\n\n- Fix security filter: Properly quote roles/users with spaces.\n [buchi]\n\n- Add option to reindex Solr from a catalog query.\n [buchi]\n\n\n2.0.0 (2018-02-15)\n------------------\n\n- Complete rewrite. Not compatible with previous versions.\n [buchi]\n",
"bugtrack_url": null,
"license": "GPL version 2",
"summary": "Solr integration for Plone",
"version": "2.13.2",
"project_urls": {
"Homepage": "https://pypi.python.org/pypi/ftw.solr"
},
"split_keywords": [
"plone",
"solr"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "28d44999864e64143e42a878f2f93b93d001fd1756c2cbeb23e09d11c6b31d26",
"md5": "766e3f01879fe83f32f379b91fce0a6d",
"sha256": "74ef768ed18f4bdc94afd53cfff40d8469f831842deb236d65407f4831a65e9a"
},
"downloads": -1,
"filename": "ftw.solr-2.13.2.tar.gz",
"has_sig": false,
"md5_digest": "766e3f01879fe83f32f379b91fce0a6d",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 59639,
"upload_time": "2024-04-04T11:39:19",
"upload_time_iso_8601": "2024-04-04T11:39:19.174640Z",
"url": "https://files.pythonhosted.org/packages/28/d4/4999864e64143e42a878f2f93b93d001fd1756c2cbeb23e09d11c6b31d26/ftw.solr-2.13.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-04 11:39:19",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "ftw.solr"
}
JSON