# wikiget
[][python]
[][action]
[][codecov]
[][pypi]
[][license]
Something like [wget] for downloading a file from MediaWiki sites (like Wikipedia or Wikimedia Commons) using only the
file name or the URL of its description page.
## Installation
Requires Python 3.7+ and pip. Install the latest version with:
```bash
pip install wikiget
```
For the latest features, at the risk of bugs and undocumented behavior, you can install the development version directly
from [GitHub]:
```bash
pip install https://github.com/clpo13/wikiget/archive/refs/heads/master.zip
```
## Usage
`wikiget [-h] [-V] [-q | -v] [-f] [-s SITE] [-P PATH] [-u USERNAME] [-p PASSWORD] [-o OUTPUT | -a] [-l LOGFILE] [-j THREADS] FILE`
The only required parameter is `FILE`, which is the file you want to download. It can either be the name of the file on
the wiki, including the namespace prefix, or a link to the file description page. If `FILE` is in the form
`File:Example.jpg` or `Image:Example.jpg`, it will be fetched from the default site, which is "commons.wikimedia.org".
If it's the fully-qualified URL of a file description page, like `https://en.wikipedia.org/wiki/File:Example.jpg`, the
file is fetched from the site in the URL, in this case "en.wikipedia.org". Note: full URLs may contain characters your
shell interprets differently, so you can either escape those characters with a backslash `\` or surround the entire URL
with single `'` or double `"` quotes. Use of a fully-qualified URL like this may require setting the `--path` flag (see
next paragraph).
The site can also be specified with the `--site` flag, though this will not have any effect if the full URL is given.
Non-Wikimedia sites should work, but you may need to specify the wiki's script path with `--path` (where `index.php` and
`api.php` live; on Wikimedia sites it's `/w/`, but other sites may use `/` or something else entirely). Private wikis
(those requiring login even for read access) are also supported with the use of the `--username` and `--password` flags.
More detailed information, such as the site used and full URL of the file, can be displayed with `-v` or `--verbose`.
Use `-vv` to display even more detail, mainly debugging information or API messages. `-q` can be used to silence
warnings. A logfile can be specified with `-l` or `--logfile`. If this option is present, the logfile will contain the
same information as `-v` along with timestamps. New log entries will be appended to an existing logfile.
By default, the program won't overwrite existing files with the same name as the target, but this can be forced with
`-f` or `--force`. Additionally, the file can be downloaded to a different name with `-o`.
Files can be batch downloaded with the `-a` or `--batch` flag. In this mode, `FILE` will be treated as an input file
containing multiple files to download, one filename or URL per line. Blank lines and lines starting with "#" are
ignored. If an error is encountered, execution stops immediately and the offending filename is printed. For large
batches, the process can be sped up by downloading files in parallel. The number of parallel downloads can be set with
`-j`. For instance, with `-a -j4`, wikiget will download four files at once. Without `-j` or with `-j` by itself without
a number, wikiget will download the files one at a time.
### Example usage
```bash
wikiget File:Example.jpg
wikiget --site en.wikipedia.org File:Example.jpg
wikiget https://en.wikipedia.org/wiki/File:Example.jpg -o test.jpg
```
## Future plans
- optional machine-readable (JSON) log output
- batch download by (Commons) category or user uploads
- maybe: download Wikipedia articles, in plain text, wikitext, or other formats
## Contributing
Pull requests, bug reports, or feature requests are more than welcome.
It's recommended that you use a [virtual environment manager][venv] like venv or [virtualenv] to create an isolated
environment in which to install this package's dependencies as not to clutter your system Python environment:
```bash
# if you plan on submitting pull requests, fork the repo on GitHub and clone that instead
git clone https://github.com/clpo13/wikiget
cd wikiget
python3 -m venv venv
# or
virtualenv venv
```
To activate the virtual environment, use one of the following commands:
```bash
# Linux and macOS; activate.csh and activate.fish are also available
source venv/bin/activate
# Windows (Command Prompt or PowerShell)
.\venv\Scripts\activate
```
Then run `pip install -e .` to invoke an ["editable" install][editable], meaning any changes made to the source will be
reflected immediately in the executable script. Unit tests can be run with `pytest` (make sure to run
`pip install pytest` in the virtual environment first.)
Alternatively, using [Hatch], simply clone the repository and run `hatch run test` to create the environment and run
pytest all in one go. Wikiget can also be run directly in the Hatch environment with `hatch run wikiget [...]`.
## License
Copyright (C) 2018-2023 Cody Logan and contributors
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public
License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later
version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied
warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see
<https://www.gnu.org/licenses/>.
[wget]: https://www.gnu.org/software/wget/
[github]: https://github.com/clpo13/wikiget/
[python]: https://www.python.org/
[action]: https://github.com/clpo13/wikiget/actions/workflows/python.yml
[codecov]: https://app.codecov.io/gh/clpo13/wikiget/
[pypi]: https://pypi.org/project/wikiget/
[license]: https://github.com/clpo13/wikiget/blob/master/LICENSE
[venv]: https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/
[virtualenv]: https://virtualenv.pypa.io/en/latest/
[editable]: https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs
[hatch]: https://hatch.pypa.io/latest/
Raw data
{
"_id": null,
"home_page": "",
"name": "wikiget",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "",
"keywords": "commons,mediawiki,wikimedia,wikipedia",
"author": "",
"author_email": "Cody Logan <cody@lokken.dev>",
"download_url": "https://files.pythonhosted.org/packages/0b/bf/f67b11e5f32218c5793b4dbbb7ae9309f11db10c1f750a059bdc843f1de2/wikiget-0.8.0.tar.gz",
"platform": null,
"description": "# wikiget\n\n[][python]\n[][action]\n[][codecov]\n[][pypi]\n[][license]\n\nSomething like [wget] for downloading a file from MediaWiki sites (like Wikipedia or Wikimedia Commons) using only the\nfile name or the URL of its description page.\n\n## Installation\n\nRequires Python 3.7+ and pip. Install the latest version with:\n\n```bash\npip install wikiget\n```\n\nFor the latest features, at the risk of bugs and undocumented behavior, you can install the development version directly\nfrom [GitHub]:\n\n```bash\npip install https://github.com/clpo13/wikiget/archive/refs/heads/master.zip\n```\n\n## Usage\n\n`wikiget [-h] [-V] [-q | -v] [-f] [-s SITE] [-P PATH] [-u USERNAME] [-p PASSWORD] [-o OUTPUT | -a] [-l LOGFILE] [-j THREADS] FILE`\n\nThe only required parameter is `FILE`, which is the file you want to download. It can either be the name of the file on\nthe wiki, including the namespace prefix, or a link to the file description page. If `FILE` is in the form\n`File:Example.jpg` or `Image:Example.jpg`, it will be fetched from the default site, which is \"commons.wikimedia.org\".\nIf it's the fully-qualified URL of a file description page, like `https://en.wikipedia.org/wiki/File:Example.jpg`, the\nfile is fetched from the site in the URL, in this case \"en.wikipedia.org\". Note: full URLs may contain characters your\nshell interprets differently, so you can either escape those characters with a backslash `\\` or surround the entire URL\nwith single `'` or double `\"` quotes. Use of a fully-qualified URL like this may require setting the `--path` flag (see\nnext paragraph).\n\nThe site can also be specified with the `--site` flag, though this will not have any effect if the full URL is given.\nNon-Wikimedia sites should work, but you may need to specify the wiki's script path with `--path` (where `index.php` and\n`api.php` live; on Wikimedia sites it's `/w/`, but other sites may use `/` or something else entirely). Private wikis\n(those requiring login even for read access) are also supported with the use of the `--username` and `--password` flags.\n\nMore detailed information, such as the site used and full URL of the file, can be displayed with `-v` or `--verbose`.\nUse `-vv` to display even more detail, mainly debugging information or API messages. `-q` can be used to silence\nwarnings. A logfile can be specified with `-l` or `--logfile`. If this option is present, the logfile will contain the\nsame information as `-v` along with timestamps. New log entries will be appended to an existing logfile.\n\nBy default, the program won't overwrite existing files with the same name as the target, but this can be forced with\n`-f` or `--force`. Additionally, the file can be downloaded to a different name with `-o`.\n\nFiles can be batch downloaded with the `-a` or `--batch` flag. In this mode, `FILE` will be treated as an input file\ncontaining multiple files to download, one filename or URL per line. Blank lines and lines starting with \"#\" are\nignored. If an error is encountered, execution stops immediately and the offending filename is printed. For large\nbatches, the process can be sped up by downloading files in parallel. The number of parallel downloads can be set with\n`-j`. For instance, with `-a -j4`, wikiget will download four files at once. Without `-j` or with `-j` by itself without\na number, wikiget will download the files one at a time.\n\n### Example usage\n\n```bash\nwikiget File:Example.jpg\nwikiget --site en.wikipedia.org File:Example.jpg\nwikiget https://en.wikipedia.org/wiki/File:Example.jpg -o test.jpg\n```\n\n## Future plans\n\n- optional machine-readable (JSON) log output\n- batch download by (Commons) category or user uploads\n- maybe: download Wikipedia articles, in plain text, wikitext, or other formats\n\n## Contributing\n\nPull requests, bug reports, or feature requests are more than welcome.\n\nIt's recommended that you use a [virtual environment manager][venv] like venv or [virtualenv] to create an isolated\nenvironment in which to install this package's dependencies as not to clutter your system Python environment:\n\n```bash\n# if you plan on submitting pull requests, fork the repo on GitHub and clone that instead\ngit clone https://github.com/clpo13/wikiget\ncd wikiget\n\npython3 -m venv venv\n# or\nvirtualenv venv\n```\n\nTo activate the virtual environment, use one of the following commands:\n\n```bash\n# Linux and macOS; activate.csh and activate.fish are also available\nsource venv/bin/activate\n\n# Windows (Command Prompt or PowerShell)\n.\\venv\\Scripts\\activate\n```\n\nThen run `pip install -e .` to invoke an [\"editable\" install][editable], meaning any changes made to the source will be\nreflected immediately in the executable script. Unit tests can be run with `pytest` (make sure to run\n`pip install pytest` in the virtual environment first.)\n\nAlternatively, using [Hatch], simply clone the repository and run `hatch run test` to create the environment and run\npytest all in one go. Wikiget can also be run directly in the Hatch environment with `hatch run wikiget [...]`.\n\n## License\n\nCopyright (C) 2018-2023 Cody Logan and contributors\n\nThis program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public\nLicense as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later\nversion.\n\nThis program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied\nwarranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.\n\nYou should have received a copy of the GNU General Public License along with this program. If not, see\n<https://www.gnu.org/licenses/>.\n\n[wget]: https://www.gnu.org/software/wget/\n[github]: https://github.com/clpo13/wikiget/\n[python]: https://www.python.org/\n[action]: https://github.com/clpo13/wikiget/actions/workflows/python.yml\n[codecov]: https://app.codecov.io/gh/clpo13/wikiget/\n[pypi]: https://pypi.org/project/wikiget/\n[license]: https://github.com/clpo13/wikiget/blob/master/LICENSE\n[venv]: https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/\n[virtualenv]: https://virtualenv.pypa.io/en/latest/\n[editable]: https://pip.pypa.io/en/stable/topics/local-project-installs/#editable-installs\n[hatch]: https://hatch.pypa.io/latest/\n",
"bugtrack_url": null,
"license": "GPL-3.0-or-later",
"summary": "CLI tool for downloading files from MediaWiki sites",
"version": "0.8.0",
"project_urls": {
"Bug Reports": "https://github.com/clpo13/wikiget/issues",
"Repository": "https://github.com/clpo13/wikiget"
},
"split_keywords": [
"commons",
"mediawiki",
"wikimedia",
"wikipedia"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b47bdbe505a41e5e96c53617cb1fd5adcd174a0136e6a9000a59fc7a2be8ebcc",
"md5": "87186c7a70e73b8f23241dc51dae5aec",
"sha256": "08740eec627f0c14854200db4d2e2fc9177cb6139b674c4add6829a3347e27fe"
},
"downloads": -1,
"filename": "wikiget-0.8.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "87186c7a70e73b8f23241dc51dae5aec",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 49278,
"upload_time": "2023-11-16T19:43:52",
"upload_time_iso_8601": "2023-11-16T19:43:52.446436Z",
"url": "https://files.pythonhosted.org/packages/b4/7b/dbe505a41e5e96c53617cb1fd5adcd174a0136e6a9000a59fc7a2be8ebcc/wikiget-0.8.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0bbff67b11e5f32218c5793b4dbbb7ae9309f11db10c1f750a059bdc843f1de2",
"md5": "4888156fc3e7ad9684871a4a1577cb3a",
"sha256": "6ddf1d43c725c2e52fab5bd7e80f13155acbda2f873f197e3bfccb129f693649"
},
"downloads": -1,
"filename": "wikiget-0.8.0.tar.gz",
"has_sig": false,
"md5_digest": "4888156fc3e7ad9684871a4a1577cb3a",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 36718,
"upload_time": "2023-11-16T19:43:54",
"upload_time_iso_8601": "2023-11-16T19:43:54.270087Z",
"url": "https://files.pythonhosted.org/packages/0b/bf/f67b11e5f32218c5793b4dbbb7ae9309f11db10c1f750a059bdc843f1de2/wikiget-0.8.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-16 19:43:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "clpo13",
"github_project": "wikiget",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "wikiget"
}
JSON
