ArXiv Update CLI
================
Fetch new articles on arXiv by keywords and authors.
CLI tool to fetch new articles on arXiv in selected categories (e.g. quant-ph) filtered by keywords or authors.
The results are printed in the command line but they can also be written into a file or be sent by email so that
the script can be run automatically with `cron <https://en.wikipedia.org/wiki/Cron>`_. There is also an interactive command line interface to browse the results.
The script will fetch the articles on arXiv that
+ were *submitted/updated* after the last update date (or the provided date, see ``--since`` option)
**AND**
+ belong to one of the *categories*
**AND**
+ (one of the *authors* is in the author list) **OR** (one of the *keywords* is in the title or abstract)
All the *options* are set in the configuration file. Note that keywords can contain spaces, e.g. *machine learning*.
The ``query`` subcommand allows for sending custom queries to the arXiv API, see section `Custom queries to the ArXiv API`_.
Thank you to arXiv for use of its open access interoperability.
.. contents:: Table of Contents
Dependencies
------------
- Python 3
- `feedparser <https://pypi.python.org/pypi/feedparser>`_
Optional dependenices:
- `keyring <https://pypi.org/project/keyring/>`_: Store password of email account in system keyring (for the ``--email`` option).
- `pyreadline3 <https://pypi.org/project/pyreadline3/>`_: Tab autocompletion in interactive mode ``--interactive`` (Windows only since the ``readline`` module is available on the other OSs).
- `pyzotero <https://pypi.org/project/pyzotero/>`_: Import articles in Zotero Library (interactive mode).
- `tqdm <https://pypi.org/project/tqdm/>`_: Progressbar for file downloads.
Running the tests in Windows also requires `pynput <https://pypi.org/project/pynput/>`_.
Install
-------
The script can be installed with pip
::
$ pip install arxiv_update_cli
or by downloading the `source code <https://gitlab.com/j_4321/arxivscript/-/tags>`_ and running
::
$ python setup.py install
from within the folder.
For Archlinux users, there is a ``PKGBUILD`` file in ``packaging_archlinux/``.
Quick start
-----------
Execute ``arxiv-update-cli``, or if the package is not installed, execute ``arxiv_update_cli.py`` with python.
Usage:
::
arxiv_update_cli.py [-h] [-e] [-s YYYY-MM-DD] [-c [FILE]] [-v] [-o FILE] [-f FORMAT][-d FOLDER] [-i] [--log] [--max-trials MAX_TRIALS] {query} ...
Options:
============================================= ========================================================================================
\-h, \-\-help show the help message and exit
\-e, \-\-email send result by email using SMTP (you will be prompted for missing settings)
\-s YYYY-MM-DD, \-\-since YYYY-MM-DD fetch update since YYYY-MM-DD 00:00
\-c [FILE], \-\-config [FILE] config file to use or print path to default one and exit if no argument is provided
\-v, \-\-version show version and exit
\-o FILE, \-\-output FILE write the output into FILE
\-d FOLDER, \-\-download FOLDER download all the articles as pdf fiĺes in FOLDER (folder must exist)
\-i, \-\-interactive display result in interactive CLI
\-f FORMAT, \-\-format FORMAT article formatting: "id" (arXiv id, "title" (title - date),
"condensed" (title, authors, date - url),
"full" (title, authors, date, abstract, url, ...), or "bibtex"
\-\-log show path to log file and exit
\-\-max-trials MAX_TRIALS maximum number of attempts to fetch the data when there is some connection issue
\-\-nb-results-per-query NB_RESULTS_PER_QUERY split queries in chunks of NB_RESULTS_PER_QUERY. Sometimes there are issues with the API
and changing the number of results queried in one call helps solve them.
query subcommand to send a custom query the arXiv API, see `Custom queries to the ArXiv API`_
============================================= ========================================================================================
If no configuration file exists, one will be created. Then, you can edit the
*categories*, *keywords* and *authors* fields and run the script again.
Configuration file
------------------
The location of the configuration file is given by the ``-c`` option. The file contains to sections:
- General: configuration of the filtering of the arXiv updates.
- Email: email settings to receive the updates by email. You will be prompted to fill in the missing email settings when you run the script.
- Zotero: settings to access Zotero API and be able to import articles in `Zotero <https://www.zotero.org>`_ from the interactive CLI (requires pyzotero).
There are comments in the file explaining how to fill in the options in each section.
Note that some options in the configuration file can be overriden by command line arguments.
Custom queries to the ArXiv API
-------------------------------
Usage:
::
arxiv_update_cli.py [-e/o/f/d/i] query [-h] [--api-query-help] [--id-list ID [ID ...]] [--sort-by SORT_BY] [--sort-order ORDER]
[--start-date YYYY-MM-DD] [--end-date YYYY-MM-DD] [--max-results N] [QUERY]
Options:
========================= ==============================================================================================
\-h, \-\-help show this help message and exit
\-\-api-query-help display API search query help and exit
\-\-id-list ID [ID ...] only results with given arXiv ids
\-\-sort-by SORT_BY sort results by "relevance", "lastUpdatedDate" [default], or "submittedDate"
\-\-sort-order ORDER sort results in "ascending" or "descending" [default] order
\-\-start-date YYYY-MM-DD minimum date
\-\-end-date YYYY-MM-DD maximum date
\-\-max-results N display only the N first results
========================= ==============================================================================================
Positional arguments:
========================= ==============================================================================================
QUERY API search query, typical form: ``<field>:<query>+<operator>+<field>:<query>...``
========================= ==============================================================================================
Fields
~~~~~~
====== ========================
prefix explanation
====== ========================
ti Title
au Author
abs Abstract
co Comment
jr Journal Reference
cat Subject Category
rn Report Number
all All of the above
====== ========================
**Example:**: ``ti:checkerboard`` to list the articles whose titles contain the word *checkerboard*.
Operators
~~~~~~~~~
Several fields can be combined using boolean operators:
- AND
- OR
- ANDNOT
**Example:**: ``au:del_maestro+ANDNOT+ti:checkerboard`` to list the articles of *Adrian DelMaestro* with titles that do not contain the word *checkerboard*
Grouping operators
~~~~~~~~~~~~~~~~~~
- More complex queries can be used by using parentheses for grouping the Boolean expressions.
- Entire phrases can be used in a search field by enclosing them in double quotes.
The grouping operators are encoded in the following way:
============== ======== ========================================================================
symbol encoding explanation
============== ======== ========================================================================
( ) %28 %29 Used to group Boolean expressions for Boolean operator precedence.
"" %22 %22 Used to group multiple words into phrases to search a particular field.
space \+ Used to extend a search_query to include multiple fields.
============== ======== ========================================================================
**Example**: ``ti:%22quantum+criticality%22`` to list the articles whose titles contain the words *quantum* and *criticality*.
More about the arXiv API: https://arxiv.org/help/api/user-manual
Automatic execution
-------------------
Once the email setttings have been configured and the password saved in the keyring, regular executions of ``arxiv_update_cli`` can be scheduled. For instance, one can receive daily email updates at 9:30 on working days with the following `crontab <https://en.wikipedia.org/wiki/Cron>`_ (Unix)
::
30 09 * * 1-5 arxiv-update-cli -e
Troubleshooting
---------------
Errors are logged in the file ``arxiv_update_cli.log`` in the system's temporary folder. Use the ``--log`` option to display the path.
License
-------
| Copyright (c) 2022-2025 Juliette Monsel
|
| Permission is hereby granted, free of charge, to any person obtaining a copy
| of this software and associated documentation files (the "Software"), to deal
| in the Software without restriction, including without limitation the rights
| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
| copies of the Software, and to permit persons to whom the Software is
| furnished to do so, subject to the following conditions:
|
| The above copyright notice and this permission notice shall be included in all
| copies or substantial portions of the Software.
|
| THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
| SOFTWARE.
Changelog
---------
+ arxiv-update-cli 1.2.1
* Fix recent issues related to the retrieval of the data from the API.
* Add option (in configuration file) to attach pdf file to zotero item.
* Add option (in configuration file) to choose collection in Zotero. The default collection is still determined by the `collection_key` option.
* Add option to change the number of results per query when fetching results (in configuration file and command line). Sometimes there are issues with the API and changing the number of results queried in one call helps solve them.
* Add option to change the maximum number of trials (in configuration file).
+ arxiv-update-cli 1.2.0
* Add interactive CLI to visualize results in the terminal
* Add possibility to import articles in Zotero
* Handle better failures to retrieve results from arXiv API (especially Connection reset by peer), trying several times and not saving the new latest update date when failing
+ arxiv-update-cli 1.1.0
* Add *format* option to choose how the articles are displayed
* Add *output* option to write the results in a text file
* Add direct queries to the arXiv API
* Make compatible with Windows
+ arxiv-update-cli 1.0.3
* Set default config path to local folder if the script is not installed
+ arxiv-update-cli 1.0.2
* Add comment field to the article summary
* Color the article URL in blue like the DOI link in the terminal
+ arxiv-update-cli 1.0.1
* Fix URL in PKGBUILD and setup.py
+ arxiv-update-cli 1.0.0
* First release
Raw data
{
"_id": null,
"home_page": "https://gitlab.com/j_4321/arxivscript",
"name": "arxiv-update-cli",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Juliette Monsel",
"author_email": "j_4321@protonmail.com",
"download_url": "https://files.pythonhosted.org/packages/7f/cc/c1cb85ca8e82d12f0694a502a8713fedcd783230d1a1227ef0672a8e738f/arxiv_update_cli-1.2.1.tar.gz",
"platform": null,
"description": "ArXiv Update CLI\n================\nFetch new articles on arXiv by keywords and authors.\n\nCLI tool to fetch new articles on arXiv in selected categories (e.g. quant-ph) filtered by keywords or authors.\nThe results are printed in the command line but they can also be written into a file or be sent by email so that\nthe script can be run automatically with `cron <https://en.wikipedia.org/wiki/Cron>`_. There is also an interactive command line interface to browse the results.\n\nThe script will fetch the articles on arXiv that\n\n + were *submitted/updated* after the last update date (or the provided date, see ``--since`` option)\n\n **AND**\n\n + belong to one of the *categories*\n\n **AND**\n\n + (one of the *authors* is in the author list) **OR** (one of the *keywords* is in the title or abstract)\n\nAll the *options* are set in the configuration file. Note that keywords can contain spaces, e.g. *machine learning*.\n\nThe ``query`` subcommand allows for sending custom queries to the arXiv API, see section `Custom queries to the ArXiv API`_.\n\nThank you to arXiv for use of its open access interoperability.\n\n\n.. contents:: Table of Contents\n\nDependencies\n------------\n\n- Python 3\n- `feedparser <https://pypi.python.org/pypi/feedparser>`_\n\nOptional dependenices:\n\n- `keyring <https://pypi.org/project/keyring/>`_: Store password of email account in system keyring (for the ``--email`` option).\n- `pyreadline3 <https://pypi.org/project/pyreadline3/>`_: Tab autocompletion in interactive mode ``--interactive`` (Windows only since the ``readline`` module is available on the other OSs).\n- `pyzotero <https://pypi.org/project/pyzotero/>`_: Import articles in Zotero Library (interactive mode).\n\n- `tqdm <https://pypi.org/project/tqdm/>`_: Progressbar for file downloads.\n\nRunning the tests in Windows also requires `pynput <https://pypi.org/project/pynput/>`_.\n\n\nInstall\n-------\n\nThe script can be installed with pip\n\n::\n\n $ pip install arxiv_update_cli\n\nor by downloading the `source code <https://gitlab.com/j_4321/arxivscript/-/tags>`_ and running\n\n::\n\n $ python setup.py install\n\nfrom within the folder.\n\n\nFor Archlinux users, there is a ``PKGBUILD`` file in ``packaging_archlinux/``.\n\n\nQuick start\n-----------\n\nExecute ``arxiv-update-cli``, or if the package is not installed, execute ``arxiv_update_cli.py`` with python.\n\nUsage:\n\n::\n\n arxiv_update_cli.py [-h] [-e] [-s YYYY-MM-DD] [-c [FILE]] [-v] [-o FILE] [-f FORMAT][-d FOLDER] [-i] [--log] [--max-trials MAX_TRIALS] {query} ...\n\n\nOptions:\n\n============================================= ========================================================================================\n\\-h, \\-\\-help show the help message and exit\n\n\\-e, \\-\\-email send result by email using SMTP (you will be prompted for missing settings)\n\n\\-s YYYY-MM-DD, \\-\\-since YYYY-MM-DD fetch update since YYYY-MM-DD 00:00\n\n\\-c [FILE], \\-\\-config [FILE] config file to use or print path to default one and exit if no argument is provided\n\n\\-v, \\-\\-version show version and exit\n\n\\-o FILE, \\-\\-output FILE write the output into FILE\n\n\\-d FOLDER, \\-\\-download FOLDER download all the articles as pdf fi\u013aes in FOLDER (folder must exist)\n\n\\-i, \\-\\-interactive display result in interactive CLI\n\n\\-f FORMAT, \\-\\-format FORMAT article formatting: \"id\" (arXiv id, \"title\" (title - date),\n \"condensed\" (title, authors, date - url),\n \"full\" (title, authors, date, abstract, url, ...), or \"bibtex\"\n\n\\-\\-log show path to log file and exit\n\n\\-\\-max-trials MAX_TRIALS maximum number of attempts to fetch the data when there is some connection issue\n\n\\-\\-nb-results-per-query NB_RESULTS_PER_QUERY split queries in chunks of NB_RESULTS_PER_QUERY. Sometimes there are issues with the API \n and changing the number of results queried in one call helps solve them.\n\nquery subcommand to send a custom query the arXiv API, see `Custom queries to the ArXiv API`_\n\n============================================= ========================================================================================\n\nIf no configuration file exists, one will be created. Then, you can edit the\n*categories*, *keywords* and *authors* fields and run the script again.\n\n\nConfiguration file\n------------------\n\nThe location of the configuration file is given by the ``-c`` option. The file contains to sections:\n\n- General: configuration of the filtering of the arXiv updates.\n- Email: email settings to receive the updates by email. You will be prompted to fill in the missing email settings when you run the script.\n- Zotero: settings to access Zotero API and be able to import articles in `Zotero <https://www.zotero.org>`_ from the interactive CLI (requires pyzotero).\n\nThere are comments in the file explaining how to fill in the options in each section.\nNote that some options in the configuration file can be overriden by command line arguments.\n\n\nCustom queries to the ArXiv API\n-------------------------------\n\nUsage:\n\n::\n\n arxiv_update_cli.py [-e/o/f/d/i] query [-h] [--api-query-help] [--id-list ID [ID ...]] [--sort-by SORT_BY] [--sort-order ORDER]\n [--start-date YYYY-MM-DD] [--end-date YYYY-MM-DD] [--max-results N] [QUERY]\n\n\nOptions:\n\n========================= ==============================================================================================\n\\-h, \\-\\-help show this help message and exit\n\n\\-\\-api-query-help display API search query help and exit\n\n\\-\\-id-list ID [ID ...] only results with given arXiv ids\n\n\\-\\-sort-by SORT_BY sort results by \"relevance\", \"lastUpdatedDate\" [default], or \"submittedDate\"\n\n\\-\\-sort-order ORDER sort results in \"ascending\" or \"descending\" [default] order\n\n\\-\\-start-date YYYY-MM-DD minimum date\n\n\\-\\-end-date YYYY-MM-DD maximum date\n\n\\-\\-max-results N display only the N first results\n========================= ==============================================================================================\n\n\nPositional arguments:\n\n========================= ==============================================================================================\nQUERY API search query, typical form: ``<field>:<query>+<operator>+<field>:<query>...``\n========================= ==============================================================================================\n\nFields\n~~~~~~\n\n====== ========================\nprefix explanation\n====== ========================\nti Title\nau Author\nabs Abstract\nco Comment\njr Journal Reference\ncat Subject Category\nrn Report Number\nall All of the above\n====== ========================\n\n**Example:**: ``ti:checkerboard`` to list the articles whose titles contain the word *checkerboard*.\n\n\nOperators\n~~~~~~~~~\n\nSeveral fields can be combined using boolean operators:\n\n- AND\n- OR\n- ANDNOT\n\n**Example:**: ``au:del_maestro+ANDNOT+ti:checkerboard`` to list the articles of *Adrian DelMaestro* with titles that do not contain the word *checkerboard*\n\n\nGrouping operators\n~~~~~~~~~~~~~~~~~~\n\n- More complex queries can be used by using parentheses for grouping the Boolean expressions.\n- Entire phrases can be used in a search field by enclosing them in double quotes.\n\nThe grouping operators are encoded in the following way:\n\n============== ======== ========================================================================\nsymbol encoding explanation\n============== ======== ========================================================================\n( ) %28 %29 Used to group Boolean expressions for Boolean operator precedence.\n\"\" %22 %22 Used to group multiple words into phrases to search a particular field.\nspace \\+ Used to extend a search_query to include multiple fields.\n============== ======== ========================================================================\n\n**Example**: ``ti:%22quantum+criticality%22`` to list the articles whose titles contain the words *quantum* and *criticality*.\n\n\nMore about the arXiv API: https://arxiv.org/help/api/user-manual\n\n\nAutomatic execution\n-------------------\n\nOnce the email setttings have been configured and the password saved in the keyring, regular executions of ``arxiv_update_cli`` can be scheduled. For instance, one can receive daily email updates at 9:30 on working days with the following `crontab <https://en.wikipedia.org/wiki/Cron>`_ (Unix)\n\n::\n\n 30 09 * * 1-5 arxiv-update-cli -e\n\n\nTroubleshooting\n---------------\n\nErrors are logged in the file ``arxiv_update_cli.log`` in the system's temporary folder. Use the ``--log`` option to display the path.\n\nLicense\n-------\n\n| Copyright (c) 2022-2025 Juliette Monsel\n|\n| Permission is hereby granted, free of charge, to any person obtaining a copy\n| of this software and associated documentation files (the \"Software\"), to deal\n| in the Software without restriction, including without limitation the rights\n| to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\n| copies of the Software, and to permit persons to whom the Software is\n| furnished to do so, subject to the following conditions:\n|\n| The above copyright notice and this permission notice shall be included in all\n| copies or substantial portions of the Software.\n|\n| THE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\n| IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\n| FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\n| AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\n| LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\n| OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\n| SOFTWARE.\n\n\nChangelog\n---------\n+ arxiv-update-cli 1.2.1\n * Fix recent issues related to the retrieval of the data from the API.\n * Add option (in configuration file) to attach pdf file to zotero item.\n * Add option (in configuration file) to choose collection in Zotero. The default collection is still determined by the `collection_key` option.\n * Add option to change the number of results per query when fetching results (in configuration file and command line). Sometimes there are issues with the API and changing the number of results queried in one call helps solve them.\n * Add option to change the maximum number of trials (in configuration file).\n\n+ arxiv-update-cli 1.2.0\n * Add interactive CLI to visualize results in the terminal\n * Add possibility to import articles in Zotero\n * Handle better failures to retrieve results from arXiv API (especially Connection reset by peer), trying several times and not saving the new latest update date when failing\n\n+ arxiv-update-cli 1.1.0\n * Add *format* option to choose how the articles are displayed\n * Add *output* option to write the results in a text file\n * Add direct queries to the arXiv API\n * Make compatible with Windows\n\n+ arxiv-update-cli 1.0.3\n * Set default config path to local folder if the script is not installed\n\n+ arxiv-update-cli 1.0.2\n * Add comment field to the article summary\n * Color the article URL in blue like the DOI link in the terminal\n\n+ arxiv-update-cli 1.0.1\n * Fix URL in PKGBUILD and setup.py\n\n+ arxiv-update-cli 1.0.0\n * First release\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Fetch new articles on arXiv by keywords",
"version": "1.2.1",
"project_urls": {
"Homepage": "https://gitlab.com/j_4321/arxivscript"
},
"split_keywords": [],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "83d5b44385f0e05bc20c09098ca883dfc8f725904784f6750008f87c43a5ccc6",
"md5": "5c049a673310f3d1e040eae6c368a62f",
"sha256": "b667403da97a4c32fc09ba1ea6aed32f31425b4cb8f57fa2dd2d645abe90f1fa"
},
"downloads": -1,
"filename": "arxiv_update_cli-1.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "5c049a673310f3d1e040eae6c368a62f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 20856,
"upload_time": "2025-08-28T06:47:27",
"upload_time_iso_8601": "2025-08-28T06:47:27.349189Z",
"url": "https://files.pythonhosted.org/packages/83/d5/b44385f0e05bc20c09098ca883dfc8f725904784f6750008f87c43a5ccc6/arxiv_update_cli-1.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "7fccc1cb85ca8e82d12f0694a502a8713fedcd783230d1a1227ef0672a8e738f",
"md5": "de85bd6648c2b1d6dea8bfa59e442930",
"sha256": "85702f30085e8c6cb8dfca39c3deb5cb0e38c65665a2c71a434c2fad3a0c4915"
},
"downloads": -1,
"filename": "arxiv_update_cli-1.2.1.tar.gz",
"has_sig": false,
"md5_digest": "de85bd6648c2b1d6dea8bfa59e442930",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 31435,
"upload_time": "2025-08-28T06:47:28",
"upload_time_iso_8601": "2025-08-28T06:47:28.988835Z",
"url": "https://files.pythonhosted.org/packages/7f/cc/c1cb85ca8e82d12f0694a502a8713fedcd783230d1a1227ef0672a8e738f/arxiv_update_cli-1.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-28 06:47:28",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "j_4321",
"gitlab_project": "arxivscript",
"lcname": "arxiv-update-cli"
}
JSON