# feed-filter
Filter for modifying feed data. By default, reads feed from stdin and writes a
modified feed to stdout in either Atom (default) or RSS format.
feed-filter can modify the titles of entries via a regular expression
(python re syntax) and also add the entry's date to the far end of the
title as an aid to sorting for feed readers that cannot have both a
primary sort and secondary sort fields.
feed-filter can also optionaly make some modification to the content
such as converting URLs into links.
## Options
### --config-file
Override the default location of the config file. Because the config file
if optional, if the specified files does not exits, it will be silently ignored.
### --name (-n)
Section name in config file to use. If not specified, only the [DEFAULT]
section will apply. See Configuration File.
### --title-re and --title-sub
The --title-re option specifies a regular expression. And the --title-sub option can use backrefferences to the RE in --title-re.
So for example, if you have the following options
> --title-re='([^•]+ • )?(Re: )?(.*)' --title-sub='\3'
It will make the following modification to the title
Original title:
> Tutorials and videos • Re: Part design Tutorials and much more ...
Modified title:
> Part design Tutorials and much more ...
That change did 2 things. It removed a common prefix (forum title) that all entries have, and also removed the 'Re: ' that is added to replies.
If you wanted to keep the prefix, but just remove the 'Re: ', then
change the second option like the example below:
> --title-re='([^•]+ • )?(Re: )?(.*)' --title-sub='\1\3'
And the modified title will now be
Modified title:
> Tutorials and videos • Part design Tutorials and much more ...
Either of the above two examples can be helpfull for modifying a feed
for a forum so that you can sort the entries by title (headline) so
all posts and their responses are groups together.
The regular expression systax used is that of python's regular expressions.
### --add-date-to-title (--no-add-date-to-title)
Just grouping all related posts together is helpfull, but you probably
want to display them in the order they were created. If you happen to
have a feed reader that can sort on titles with a secondary sort on
the date, then you are all set. But if you can only do one sort
(title), the posts may be in the wrong order. This is where the
--add-date-to-title option comes in.
It does pretty much what it says. It appends the posting's date to
the end of the title after a bunch of spaces. All the spaces are just
to hide the date string. The date aids in sorting. Now when you sort
on the title, the entries will implicitly have a secondary sort on the
date due to its inclusion in the titles.
### --add-posts (--no-add-posts)
For each entry, it attempts to download a topic-specific rss or atom
feed and adds each entry in place of the original entry. This is
usefull for sites whos forum feed only shows the topics (first post)
and not any replies. Note that this option won't work on many sites
due to having to parse web pages. Raise issue for any site that
doesn't seem to work. Titles on additional posts fetched will all
be taken from the original entry.
### --auto-links (--no-auto-links)
In the content sections, anything that looks like a URL but is not already
an HTML link, will be made into a link.
### --output-fmt
Value can be either 'atom' (default), 'rss', or 'summary'. The
'summary' options just prints out a few fields in plain text format.
Used primary for debugging.
### Others
Run feed-filter with the --help option to see what other options
are available.
## Configuration File
This program can optionally use a configuration file so you don't have
to specify lots of options on the command line. The default location
for the configuration files is _${XDG_CONFIG_HOME}/feed-filter.conf_.
If _${XDG_CONFIG_HOME}_ is not set, the default value as specified in
the [XDG Base Directory Specification will be used](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).
The configuration file is an INI style file. The section to use is
specified via the --name option. This can ge used to group options
for specific feeds or groups of feed (perhaps from a common site)
together. If there is a [DEFAULT] section, any value in that section
will be used as a default value for whatever named section is actually
specified (--name). If --name is not specified, only the [DEFAULT]
section will be used.
For supported options (see --help output) just use the long option
name without the '--' prefix, an '=' and then the value. For boolean
type options, use True or False as the options value. e.g.
```
[DEFAULT]
auto-links = True
[mysection]
date-spaces = 220
auto-links = False
```
Caution, there is only limited checking for valid values. Specifying
an invalid option will be silently ignored.
# Installation
This [package](https://pypi.org/project/feedfilter/) is on
[PyPI.org](https://pypi.org/), so just install with pip or pipx like
```
pipx install feedfilter
```
# Development setup
It is recommended that you do any development in a virtual
environment. If you use direnv, a .envrc file is provided. You
should always look it over before allowing it to be used.
poetry is required. You can install it in your virtual enviroment
for this project via ```pip install poetry``` or alternatively via
```
pipx install poetry
```
Once that is installed, just run
```
make install-requirements
or
poetry install
```
To run feed-filter in development, you should be able to just run
```
feed-filter <args>
```
## Building
To create a build (sdist, wheel) run the following
```
make build
```
Results will be in the dist/ directory.
# Licensing
This project is licensed under the GNU GPL version 3 or later. See the
LICENSE file in the top-level directory.
Raw data
{
"_id": null,
"home_page": "https://gitlab.com/jim_bauer/feed-filter",
"name": "feedfilter",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10,<4.0",
"maintainer_email": "",
"keywords": "rss,atom",
"author": "Jim Bauer",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/be/7e/875b2828b7140593730e9d7bfbfc27dd10889010e260dc721e2c3821761d/feedfilter-1.1.0.tar.gz",
"platform": null,
"description": "# feed-filter\n\nFilter for modifying feed data. By default, reads feed from stdin and writes a\nmodified feed to stdout in either Atom (default) or RSS format.\n\nfeed-filter can modify the titles of entries via a regular expression\n(python re syntax) and also add the entry's date to the far end of the\ntitle as an aid to sorting for feed readers that cannot have both a\nprimary sort and secondary sort fields.\n\nfeed-filter can also optionaly make some modification to the content\nsuch as converting URLs into links.\n\n## Options\n\n### --config-file\n\nOverride the default location of the config file. Because the config file\nif optional, if the specified files does not exits, it will be silently ignored.\n\n### --name (-n)\n\nSection name in config file to use. If not specified, only the [DEFAULT]\nsection will apply. See Configuration File.\n\n### --title-re and --title-sub\n\nThe --title-re option specifies a regular expression. And the --title-sub option can use backrefferences to the RE in --title-re.\n\nSo for example, if you have the following options\n> --title-re='([^\u2022]+ \u2022 )?(Re: )?(.*)' --title-sub='\\3'\n\nIt will make the following modification to the title\n\nOriginal title:\n> Tutorials and videos \u2022 Re: Part design Tutorials and much more ...\n\nModified title:\n> Part design Tutorials and much more ...\n\nThat change did 2 things. It removed a common prefix (forum title) that all entries have, and also removed the 'Re: ' that is added to replies.\n\nIf you wanted to keep the prefix, but just remove the 'Re: ', then\nchange the second option like the example below:\n\n> --title-re='([^\u2022]+ \u2022 )?(Re: )?(.*)' --title-sub='\\1\\3'\n\nAnd the modified title will now be\n\nModified title:\n> Tutorials and videos \u2022 Part design Tutorials and much more ...\n\nEither of the above two examples can be helpfull for modifying a feed\nfor a forum so that you can sort the entries by title (headline) so\nall posts and their responses are groups together.\n\nThe regular expression systax used is that of python's regular expressions.\n\n### --add-date-to-title (--no-add-date-to-title)\n\nJust grouping all related posts together is helpfull, but you probably\nwant to display them in the order they were created. If you happen to\nhave a feed reader that can sort on titles with a secondary sort on\nthe date, then you are all set. But if you can only do one sort\n(title), the posts may be in the wrong order. This is where the\n--add-date-to-title option comes in.\n\nIt does pretty much what it says. It appends the posting's date to\nthe end of the title after a bunch of spaces. All the spaces are just\nto hide the date string. The date aids in sorting. Now when you sort\non the title, the entries will implicitly have a secondary sort on the\ndate due to its inclusion in the titles.\n\n### --add-posts (--no-add-posts)\n\nFor each entry, it attempts to download a topic-specific rss or atom\nfeed and adds each entry in place of the original entry. This is\nusefull for sites whos forum feed only shows the topics (first post)\nand not any replies. Note that this option won't work on many sites\ndue to having to parse web pages. Raise issue for any site that\ndoesn't seem to work. Titles on additional posts fetched will all\nbe taken from the original entry.\n\n### --auto-links (--no-auto-links)\n\nIn the content sections, anything that looks like a URL but is not already\nan HTML link, will be made into a link.\n\n### --output-fmt\n\nValue can be either 'atom' (default), 'rss', or 'summary'. The\n'summary' options just prints out a few fields in plain text format.\nUsed primary for debugging.\n\n### Others\n\nRun feed-filter with the --help option to see what other options\nare available.\n\n\n## Configuration File\n\nThis program can optionally use a configuration file so you don't have\nto specify lots of options on the command line. The default location\nfor the configuration files is _${XDG_CONFIG_HOME}/feed-filter.conf_.\nIf _${XDG_CONFIG_HOME}_ is not set, the default value as specified in\nthe [XDG Base Directory Specification will be used](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html).\n\nThe configuration file is an INI style file. The section to use is\nspecified via the --name option. This can ge used to group options\nfor specific feeds or groups of feed (perhaps from a common site)\ntogether. If there is a [DEFAULT] section, any value in that section\nwill be used as a default value for whatever named section is actually\nspecified (--name). If --name is not specified, only the [DEFAULT]\nsection will be used.\n\nFor supported options (see --help output) just use the long option\nname without the '--' prefix, an '=' and then the value. For boolean\ntype options, use True or False as the options value. e.g.\n\n```\n [DEFAULT]\n auto-links = True\n\n [mysection]\n date-spaces = 220\n auto-links = False\n```\n\nCaution, there is only limited checking for valid values. Specifying\nan invalid option will be silently ignored.\n\n\n# Installation\n\nThis [package](https://pypi.org/project/feedfilter/) is on\n[PyPI.org](https://pypi.org/), so just install with pip or pipx like\n\n```\npipx install feedfilter\n```\n\n# Development setup\n\nIt is recommended that you do any development in a virtual\nenvironment. If you use direnv, a .envrc file is provided. You\nshould always look it over before allowing it to be used.\n\npoetry is required. You can install it in your virtual enviroment\nfor this project via ```pip install poetry``` or alternatively via\n\n```\npipx install poetry\n```\n\nOnce that is installed, just run\n```\nmake install-requirements\n or\npoetry install\n```\n\nTo run feed-filter in development, you should be able to just run\n\n```\nfeed-filter <args>\n```\n\n## Building\n\nTo create a build (sdist, wheel) run the following\n\n```\nmake build\n```\n\nResults will be in the dist/ directory.\n\n# Licensing\n\nThis project is licensed under the GNU GPL version 3 or later. See the\nLICENSE file in the top-level directory.\n",
"bugtrack_url": null,
"license": "GPL-3.0-or-later",
"summary": "Modify RSS/Atom feeds",
"version": "1.1.0",
"project_urls": {
"Homepage": "https://gitlab.com/jim_bauer/feed-filter",
"Repository": "https://gitlab.com/jim_bauer/feed-filter"
},
"split_keywords": [
"rss",
"atom"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "578d5d0fa57010cff8f71e61c78ed2f534ca73df64794ec984a5673075444a63",
"md5": "114739dca8a9d2d36fabd491f0987592",
"sha256": "e6d0bb0d6f697076c9a2c6a93f0c3064753f37fcc02434dd092271f5b9eed27f"
},
"downloads": -1,
"filename": "feedfilter-1.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "114739dca8a9d2d36fabd491f0987592",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10,<4.0",
"size": 23294,
"upload_time": "2023-05-19T13:19:32",
"upload_time_iso_8601": "2023-05-19T13:19:32.611565Z",
"url": "https://files.pythonhosted.org/packages/57/8d/5d0fa57010cff8f71e61c78ed2f534ca73df64794ec984a5673075444a63/feedfilter-1.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "be7e875b2828b7140593730e9d7bfbfc27dd10889010e260dc721e2c3821761d",
"md5": "529d51ae27f1ba2d582e3599eeaf3684",
"sha256": "5ca8895c0f94c9376c6f91e7132af8bc7e34536cf51ad6674678d1fe65c3417b"
},
"downloads": -1,
"filename": "feedfilter-1.1.0.tar.gz",
"has_sig": false,
"md5_digest": "529d51ae27f1ba2d582e3599eeaf3684",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<4.0",
"size": 23689,
"upload_time": "2023-05-19T13:19:34",
"upload_time_iso_8601": "2023-05-19T13:19:34.410665Z",
"url": "https://files.pythonhosted.org/packages/be/7e/875b2828b7140593730e9d7bfbfc27dd10889010e260dc721e2c3821761d/feedfilter-1.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-05-19 13:19:34",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "jim_bauer",
"gitlab_project": "feed-filter",
"lcname": "feedfilter"
}
JSON