======
pysolr
======
``pysolr`` is a lightweight Python client for `Apache Solr`_. It provides an
interface that queries the server and returns results based on the query.
.. _`Apache Solr`: https://solr.apache.org/
Status
======
`Changelog <https://github.com/django-haystack/pysolr/blob/master/CHANGELOG.rst>`_
Features
========
* Basic operations such as selecting, updating & deleting.
* Index optimization.
* `"More Like This" <http://wiki.apache.org/solr/MoreLikeThis>`_ support (if set up in Solr).
* `Spelling correction <http://wiki.apache.org/solr/SpellCheckComponent>`_ (if set up in Solr).
* Timeout support.
* SolrCloud awareness
Requirements
============
* Python 2.7 - 3.7
* Requests 2.9.1+
* **Optional** - ``simplejson``
* **Optional** - ``kazoo`` for SolrCloud mode
Installation
============
pysolr is on PyPI:
.. code-block:: console
$ pip install pysolr
Or if you want to install directly from the repository:
.. code-block:: console
$ python setup.py install
Usage
=====
Basic usage looks like:
.. code-block:: python
# If on Python 2.X
from __future__ import print_function
import pysolr
# Create a client instance. The timeout and authentication options are not required.
solr = pysolr.Solr('http://localhost:8983/solr/', always_commit=True, [timeout=10], [auth=<type of authentication>])
# Note that auto_commit defaults to False for performance. You can set
# `auto_commit=True` to have commands always update the index immediately, make
# an update call with `commit=True`, or use Solr's `autoCommit` / `commitWithin`
# to have your data be committed following a particular policy.
# Do a health check.
solr.ping()
# How you'd index data.
solr.add([
{
"id": "doc_1",
"title": "A test document",
},
{
"id": "doc_2",
"title": "The Banana: Tasty or Dangerous?",
"_doc": [
{ "id": "child_doc_1", "title": "peel" },
{ "id": "child_doc_2", "title": "seed" },
]
},
])
# You can index a parent/child document relationship by
# associating a list of child documents with the special key '_doc'. This
# is helpful for queries that join together conditions on children and parent
# documents.
# Later, searching is easy. In the simple case, just a plain Lucene-style
# query is fine.
results = solr.search('bananas')
# The ``Results`` object stores total results found, by default the top
# ten most relevant results and any additional data like
# facets/highlighting/spelling/etc.
print("Saw {0} result(s).".format(len(results)))
# Just loop over it to access the results.
for result in results:
print("The title is '{0}'.".format(result['title']))
# For a more advanced query, say involving highlighting, you can pass
# additional options to Solr.
results = solr.search('bananas', **{
'hl': 'true',
'hl.fragsize': 10,
})
# Traverse a cursor using its iterator:
for doc in solr.search('*:*',fl='id',sort='id ASC',cursorMark='*'):
print(doc['id'])
# You can also perform More Like This searches, if your Solr is configured
# correctly.
similar = solr.more_like_this(q='id:doc_2', mltfl='text')
# Finally, you can delete either individual documents,
solr.delete(id='doc_1')
# also in batches...
solr.delete(id=['doc_1', 'doc_2'])
# ...or all documents.
solr.delete(q='*:*')
.. code-block:: python
# For SolrCloud mode, initialize your Solr like this:
zookeeper = pysolr.ZooKeeper("zkhost1:2181,zkhost2:2181,zkhost3:2181")
solr = pysolr.SolrCloud(zookeeper, "collection1", auth=<type of authentication>)
Multicore Index
~~~~~~~~~~~~~~~
Simply point the URL to the index core:
.. code-block:: python
# Setup a Solr instance. The timeout is optional.
solr = pysolr.Solr('http://localhost:8983/solr/core_0/', timeout=10)
Custom Request Handlers
~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: python
# Setup a Solr instance. The trailing slash is optional.
solr = pysolr.Solr('http://localhost:8983/solr/core_0/', search_handler='/autocomplete', use_qt_param=False)
If ``use_qt_param`` is ``True`` it is essential that the name of the handler is
exactly what is configured in ``solrconfig.xml``, including the leading slash
if any. If ``use_qt_param`` is ``False`` (default), the leading and trailing
slashes can be omitted.
If ``search_handler`` is not specified, pysolr will default to ``/select``.
The handlers for MoreLikeThis, Update, Terms etc. all default to the values set
in the ``solrconfig.xml`` SOLR ships with: ``mlt``, ``update``, ``terms`` etc.
The specific methods of pysolr's ``Solr`` class (like ``more_like_this``,
``suggest_terms`` etc.) allow for a kwarg ``handler`` to override that value.
This includes the ``search`` method. Setting a handler in ``search`` explicitly
overrides the ``search_handler`` setting (if any).
Custom Authentication
~~~~~~~~~~~~~~~~~~~~~
.. code-block:: python
# Setup a Solr instance in a kerborized environment
from requests_kerberos import HTTPKerberosAuth, OPTIONAL
kerberos_auth = HTTPKerberosAuth(mutual_authentication=OPTIONAL, sanitize_mutual_error_response=False)
solr = pysolr.Solr('http://localhost:8983/solr/', auth=kerberos_auth)
.. code-block:: python
# Setup a CloudSolr instance in a kerborized environment
from requests_kerberos import HTTPKerberosAuth, OPTIONAL
kerberos_auth = HTTPKerberosAuth(mutual_authentication=OPTIONAL, sanitize_mutual_error_response=False)
zookeeper = pysolr.ZooKeeper("zkhost1:2181/solr, zkhost2:2181,...,zkhostN:2181")
solr = pysolr.SolrCloud(zookeeper, "collection", auth=kerberos_auth)
If your Solr servers run off https
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. code-block:: python
# Setup a Solr instance in an https environment
solr = pysolr.Solr('http://localhost:8983/solr/', verify=path/to/cert.pem)
.. code-block:: python
# Setup a CloudSolr instance in a kerborized environment
zookeeper = pysolr.ZooKeeper("zkhost1:2181/solr, zkhost2:2181,...,zkhostN:2181")
solr = pysolr.SolrCloud(zookeeper, "collection", verify=path/to/cert.perm)
Custom Commit Policy
~~~~~~~~~~~~~~~~~~~~
.. code-block:: python
# Setup a Solr instance. The trailing slash is optional.
# All requests to Solr will be immediately committed because `always_commit=True`:
solr = pysolr.Solr('http://localhost:8983/solr/core_0/', search_handler='/autocomplete', always_commit=True)
``always_commit`` signals to the Solr object to either commit or not commit by
default for any solr request. Be sure to change this to ``True`` if you are
upgrading from a version where the default policy was always commit by default.
Functions like ``add`` and ``delete`` also still provide a way to override the
default by passing the ``commit`` kwarg.
It is generally good practice to limit the amount of commits to Solr as
excessive commits risk opening too many searchers or excessive system
resource consumption. See the Solr documentation for more information and
details about the ``autoCommit`` and ``commitWithin`` options:
https://lucene.apache.org/solr/guide/7_7/updatehandlers-in-solrconfig.html#UpdateHandlersinSolrConfig-autoCommit
LICENSE
=======
``pysolr`` is licensed under the New BSD license.
Contributing to pysolr
======================
For consistency, this project uses `pre-commit <https://pre-commit.com/>`_ to manage Git commit hooks:
#. Install the `pre-commit` package: e.g. `brew install pre-commit`,
`pip install pre-commit`, etc.
#. Run `pre-commit install` each time you check out a new copy of this Git
repository to ensure that every subsequent commit will be processed by
running `pre-commit run`, which you may also do as desired. To test the
entire repository or in a CI scenario, you can check every file rather than
just the staged ones using `pre-commit run --all`.
Running Tests
=============
The ``run-tests.py`` script will automatically perform the steps below and is
recommended for testing by default unless you need more control.
Running a test Solr instance
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Downloading, configuring and running Solr 4 looks like this::
./start-solr-test-server.sh
Running the tests
~~~~~~~~~~~~~~~~~
.. code-block:: console
$ python -m unittest tests
Raw data
{
"_id": null,
"home_page": "https://github.com/django-haystack/pysolr/",
"name": "pysolr",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Daniel Lindsley",
"author_email": "daniel@toastdriven.com",
"download_url": "https://files.pythonhosted.org/packages/6d/f5/59f3375b12172651c02615ed54b9ca8e5ca7cbf6d89a8506a5daec4ed813/pysolr-3.10.0.tar.gz",
"platform": null,
"description": "======\npysolr\n======\n\n``pysolr`` is a lightweight Python client for `Apache Solr`_. It provides an\ninterface that queries the server and returns results based on the query.\n\n.. _`Apache Solr`: https://solr.apache.org/\n\nStatus\n======\n\n`Changelog <https://github.com/django-haystack/pysolr/blob/master/CHANGELOG.rst>`_\n\nFeatures\n========\n\n* Basic operations such as selecting, updating & deleting.\n* Index optimization.\n* `\"More Like This\" <http://wiki.apache.org/solr/MoreLikeThis>`_ support (if set up in Solr).\n* `Spelling correction <http://wiki.apache.org/solr/SpellCheckComponent>`_ (if set up in Solr).\n* Timeout support.\n* SolrCloud awareness\n\nRequirements\n============\n\n* Python 2.7 - 3.7\n* Requests 2.9.1+\n* **Optional** - ``simplejson``\n* **Optional** - ``kazoo`` for SolrCloud mode\n\nInstallation\n============\n\npysolr is on PyPI:\n\n.. code-block:: console\n\n $ pip install pysolr\n\nOr if you want to install directly from the repository:\n\n.. code-block:: console\n\n $ python setup.py install\n\nUsage\n=====\n\nBasic usage looks like:\n\n.. code-block:: python\n\n # If on Python 2.X\n from __future__ import print_function\n\n import pysolr\n\n # Create a client instance. The timeout and authentication options are not required.\n solr = pysolr.Solr('http://localhost:8983/solr/', always_commit=True, [timeout=10], [auth=<type of authentication>])\n\n # Note that auto_commit defaults to False for performance. You can set\n # `auto_commit=True` to have commands always update the index immediately, make\n # an update call with `commit=True`, or use Solr's `autoCommit` / `commitWithin`\n # to have your data be committed following a particular policy.\n\n # Do a health check.\n solr.ping()\n\n # How you'd index data.\n solr.add([\n {\n \"id\": \"doc_1\",\n \"title\": \"A test document\",\n },\n {\n \"id\": \"doc_2\",\n \"title\": \"The Banana: Tasty or Dangerous?\",\n \"_doc\": [\n { \"id\": \"child_doc_1\", \"title\": \"peel\" },\n { \"id\": \"child_doc_2\", \"title\": \"seed\" },\n ]\n },\n ])\n\n # You can index a parent/child document relationship by\n # associating a list of child documents with the special key '_doc'. This\n # is helpful for queries that join together conditions on children and parent\n # documents.\n\n # Later, searching is easy. In the simple case, just a plain Lucene-style\n # query is fine.\n results = solr.search('bananas')\n\n # The ``Results`` object stores total results found, by default the top\n # ten most relevant results and any additional data like\n # facets/highlighting/spelling/etc.\n print(\"Saw {0} result(s).\".format(len(results)))\n\n # Just loop over it to access the results.\n for result in results:\n print(\"The title is '{0}'.\".format(result['title']))\n\n # For a more advanced query, say involving highlighting, you can pass\n # additional options to Solr.\n results = solr.search('bananas', **{\n 'hl': 'true',\n 'hl.fragsize': 10,\n })\n\n # Traverse a cursor using its iterator:\n for doc in solr.search('*:*',fl='id',sort='id ASC',cursorMark='*'):\n print(doc['id'])\n\n # You can also perform More Like This searches, if your Solr is configured\n # correctly.\n similar = solr.more_like_this(q='id:doc_2', mltfl='text')\n\n # Finally, you can delete either individual documents,\n solr.delete(id='doc_1')\n\n # also in batches...\n solr.delete(id=['doc_1', 'doc_2'])\n\n # ...or all documents.\n solr.delete(q='*:*')\n\n.. code-block:: python\n\n # For SolrCloud mode, initialize your Solr like this:\n\n zookeeper = pysolr.ZooKeeper(\"zkhost1:2181,zkhost2:2181,zkhost3:2181\")\n solr = pysolr.SolrCloud(zookeeper, \"collection1\", auth=<type of authentication>)\n\n\nMulticore Index\n~~~~~~~~~~~~~~~\n\nSimply point the URL to the index core:\n\n.. code-block:: python\n\n # Setup a Solr instance. The timeout is optional.\n solr = pysolr.Solr('http://localhost:8983/solr/core_0/', timeout=10)\n\n\nCustom Request Handlers\n~~~~~~~~~~~~~~~~~~~~~~~\n\n.. code-block:: python\n\n # Setup a Solr instance. The trailing slash is optional.\n solr = pysolr.Solr('http://localhost:8983/solr/core_0/', search_handler='/autocomplete', use_qt_param=False)\n\n\nIf ``use_qt_param`` is ``True`` it is essential that the name of the handler is\nexactly what is configured in ``solrconfig.xml``, including the leading slash\nif any. If ``use_qt_param`` is ``False`` (default), the leading and trailing\nslashes can be omitted.\n\nIf ``search_handler`` is not specified, pysolr will default to ``/select``.\n\nThe handlers for MoreLikeThis, Update, Terms etc. all default to the values set\nin the ``solrconfig.xml`` SOLR ships with: ``mlt``, ``update``, ``terms`` etc.\nThe specific methods of pysolr's ``Solr`` class (like ``more_like_this``,\n``suggest_terms`` etc.) allow for a kwarg ``handler`` to override that value.\nThis includes the ``search`` method. Setting a handler in ``search`` explicitly\noverrides the ``search_handler`` setting (if any).\n\n\nCustom Authentication\n~~~~~~~~~~~~~~~~~~~~~\n\n.. code-block:: python\n\n # Setup a Solr instance in a kerborized environment\n from requests_kerberos import HTTPKerberosAuth, OPTIONAL\n kerberos_auth = HTTPKerberosAuth(mutual_authentication=OPTIONAL, sanitize_mutual_error_response=False)\n\n solr = pysolr.Solr('http://localhost:8983/solr/', auth=kerberos_auth)\n\n.. code-block:: python\n\n # Setup a CloudSolr instance in a kerborized environment\n from requests_kerberos import HTTPKerberosAuth, OPTIONAL\n kerberos_auth = HTTPKerberosAuth(mutual_authentication=OPTIONAL, sanitize_mutual_error_response=False)\n\n zookeeper = pysolr.ZooKeeper(\"zkhost1:2181/solr, zkhost2:2181,...,zkhostN:2181\")\n solr = pysolr.SolrCloud(zookeeper, \"collection\", auth=kerberos_auth)\n\n\nIf your Solr servers run off https\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n.. code-block:: python\n\n # Setup a Solr instance in an https environment\n solr = pysolr.Solr('http://localhost:8983/solr/', verify=path/to/cert.pem)\n\n.. code-block:: python\n\n # Setup a CloudSolr instance in a kerborized environment\n\n zookeeper = pysolr.ZooKeeper(\"zkhost1:2181/solr, zkhost2:2181,...,zkhostN:2181\")\n solr = pysolr.SolrCloud(zookeeper, \"collection\", verify=path/to/cert.perm)\n\n\nCustom Commit Policy\n~~~~~~~~~~~~~~~~~~~~\n\n.. code-block:: python\n\n # Setup a Solr instance. The trailing slash is optional.\n # All requests to Solr will be immediately committed because `always_commit=True`:\n solr = pysolr.Solr('http://localhost:8983/solr/core_0/', search_handler='/autocomplete', always_commit=True)\n\n``always_commit`` signals to the Solr object to either commit or not commit by\ndefault for any solr request. Be sure to change this to ``True`` if you are\nupgrading from a version where the default policy was always commit by default.\n\nFunctions like ``add`` and ``delete`` also still provide a way to override the\ndefault by passing the ``commit`` kwarg.\n\nIt is generally good practice to limit the amount of commits to Solr as\nexcessive commits risk opening too many searchers or excessive system\nresource consumption. See the Solr documentation for more information and\ndetails about the ``autoCommit`` and ``commitWithin`` options:\n\nhttps://lucene.apache.org/solr/guide/7_7/updatehandlers-in-solrconfig.html#UpdateHandlersinSolrConfig-autoCommit\n\n\nLICENSE\n=======\n\n``pysolr`` is licensed under the New BSD license.\n\nContributing to pysolr\n======================\n\nFor consistency, this project uses `pre-commit <https://pre-commit.com/>`_ to manage Git commit hooks:\n\n#. Install the `pre-commit` package: e.g. `brew install pre-commit`,\n `pip install pre-commit`, etc.\n#. Run `pre-commit install` each time you check out a new copy of this Git\n repository to ensure that every subsequent commit will be processed by\n running `pre-commit run`, which you may also do as desired. To test the\n entire repository or in a CI scenario, you can check every file rather than\n just the staged ones using `pre-commit run --all`.\n\n\nRunning Tests\n=============\n\nThe ``run-tests.py`` script will automatically perform the steps below and is\nrecommended for testing by default unless you need more control.\n\nRunning a test Solr instance\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nDownloading, configuring and running Solr 4 looks like this::\n\n ./start-solr-test-server.sh\n\nRunning the tests\n~~~~~~~~~~~~~~~~~\n\n.. code-block:: console\n\n $ python -m unittest tests\n",
"bugtrack_url": null,
"license": "BSD",
"summary": "Lightweight Python client for Apache Solr",
"version": "3.10.0",
"project_urls": {
"Homepage": "https://github.com/django-haystack/pysolr/"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6df559f3375b12172651c02615ed54b9ca8e5ca7cbf6d89a8506a5daec4ed813",
"md5": "a96dca3a3ea8e52b988876b2394f56c1",
"sha256": "127b4a2dd169234acb1586643a6cd1e3e94b917921e69bf569d7b2a2aa0ef409"
},
"downloads": -1,
"filename": "pysolr-3.10.0.tar.gz",
"has_sig": false,
"md5_digest": "a96dca3a3ea8e52b988876b2394f56c1",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 59110,
"upload_time": "2024-09-18T22:49:01",
"upload_time_iso_8601": "2024-09-18T22:49:01.395292Z",
"url": "https://files.pythonhosted.org/packages/6d/f5/59f3375b12172651c02615ed54b9ca8e5ca7cbf6d89a8506a5daec4ed813/pysolr-3.10.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-18 22:49:01",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "django-haystack",
"github_project": "pysolr",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "pysolr"
}
JSON
