<!-- markdownlint-disable -->
<p align="center">
<a href="https://github.com/DevelopersToolbox/">
<img src="https://cdn.wolfsoftware.com/assets/images/github/organisations/developerstoolbox/black-and-white-circle-256.png" alt="DevelopersToolbox logo" />
</a>
<br />
<a href="https://github.com/DevelopersToolbox/pypi-extractor-package/actions/workflows/cicd.yml">
<img src="https://img.shields.io/github/actions/workflow/status/DevelopersToolbox/pypi-extractor-package/cicd.yml?branch=master&label=build%20status&style=for-the-badge" alt="Github Build Status" />
</a>
<a href="https://github.com/DevelopersToolbox/pypi-extractor-package/blob/master/LICENSE.md">
<img src="https://img.shields.io/github/license/DevelopersToolbox/pypi-extractor-package?color=blue&label=License&style=for-the-badge" alt="License">
</a>
<a href="https://github.com/DevelopersToolbox/pypi-extractor-package">
<img src="https://img.shields.io/github/created-at/DevelopersToolbox/pypi-extractor-package?color=blue&label=Created&style=for-the-badge" alt="Created">
</a>
<br />
<a href="https://github.com/DevelopersToolbox/pypi-extractor-package/releases/latest">
<img src="https://img.shields.io/github/v/release/DevelopersToolbox/pypi-extractor-package?color=blue&label=Latest%20Release&style=for-the-badge" alt="Release">
</a>
<a href="https://github.com/DevelopersToolbox/pypi-extractor-package/releases/latest">
<img src="https://img.shields.io/github/release-date/DevelopersToolbox/pypi-extractor-package?color=blue&label=Released&style=for-the-badge" alt="Released">
</a>
<a href="https://github.com/DevelopersToolbox/pypi-extractor-package/releases/latest">
<img src="https://img.shields.io/github/commits-since/DevelopersToolbox/pypi-extractor-package/latest.svg?color=blue&style=for-the-badge" alt="Commits since release">
</a>
<br />
<a href="https://github.com/DevelopersToolbox/pypi-extractor-package/blob/master/.github/CODE_OF_CONDUCT.md">
<img src="https://img.shields.io/badge/Code%20of%20Conduct-blue?style=for-the-badge" />
</a>
<a href="https://github.com/DevelopersToolbox/pypi-extractor-package/blob/master/.github/CONTRIBUTING.md">
<img src="https://img.shields.io/badge/Contributing-blue?style=for-the-badge" />
</a>
<a href="https://github.com/DevelopersToolbox/pypi-extractor-package/blob/master/.github/SECURITY.md">
<img src="https://img.shields.io/badge/Report%20Security%20Concern-blue?style=for-the-badge" />
</a>
<a href="https://github.com/DevelopersToolbox/pypi-extractor-package/issues">
<img src="https://img.shields.io/badge/Get%20Support-blue?style=for-the-badge" />
</a>
</p>
## Overview
PyPI Extractor is a Python package designed to fetch and process detailed information about packages hosted on the
Python Package Index (PyPI). This package is particularly useful for users who want to retrieve and analyze metadata for packages
maintained by a specific PyPI user.
## Significant Update From 0.1.3
pypi.org no longer allow you to scrap details using the requests package, or any package that does not support JavaScript. To resolve this we have
updated this package to utilise [PlayWright](https://pypi.org/project/playwright/) when retrieving a list of packages for a given user. While we have
attempted to automate as much as possible you might want to do some of the work manually.
Playwright needs two commands to be run in order for it to function correctly:
```
playwright install
playwright install-deps
```
We have added an `auto_install` option to the main class so that you can instruct the package to do the install for you, this helps when installing the
package in a fully automated way, e.g. Puppet or similar.
## Features
- Retrieve a list of packages maintained by a specific PyPI user.
- Fetch detailed metadata for each package, including versions, author information, dependencies, and more.
- Custom exceptions for handling errors gracefully.
- Option to set the PyPI username after initializing the class.
## Installation
You can install the package using pip:
```sh
pip install wolfsoftware.pypi-extractor
```
## Usage
### Basic Usage
Here's a basic example of how to use the PyPI Extractor:
```python
from wolfsoftware.pypi_extractor import PyPiExtractor
# Initialize without username
pypi_info = PyPiExtractor()
# Set username later
pypi_info.set_username("your_pypi_username")
# Get detailed information for all packages
try:
packages_details = pypi_info.get_all_packages_details()
print(packages_details)
except PyPiExtractorError as e:
print(f"An error occurred: {e.message}")
```
### Setting Username During Initialization
You can also set the username during initialization:
```python
pypi_info = PyPiExtractor("your_pypi_username")
```
### Retrieving User Packages
You can retrieve a list of packages maintained by a specific user:
```python
packages = pypi_info.get_user_packages()
print(packages)
```
### Retrieving Package Details
To get detailed information about a specific package:
```python
package_details = pypi_info.get_package_details("package_name")
print(package_details)
```
## API Reference
### Classes
#### `PyPiExtractor`
A class to fetch and process package details for a given PyPI user.
##### `__init__(self, username: str, verbose: bool, auto_install: bool)`
- Initializes the `PyPiExtractor` with a username.
- Parameters:
- `username` (str): The PyPI username.
- `verbose` (bool): Verbose output (Default: False)
- `auto_install` (bool): Auto install PlayWright dependencies (Default: False)
- Raises:
- `PyPiExtractorError`: If the username is not provided.
##### `set_username(self, username: str)`
- Sets the PyPI username.
- Parameters:
- `username` (str): The PyPI username.
- Raises:
- `PyPiExtractorError`: If the username is not provided.
##### `enable_verbose(self)`
- Enable verbose mode.
##### `enable_auto_install(self)`
- Enable auto install.
##### `get_user_packages(self) -> list`
- Fetches the list of packages for the given PyPI user.
- Returns:
- `list`: A list of dictionaries containing package names and summaries.
- Raises:
- `PyPiExtractorError`: If there is an error fetching or parsing the user profile.
##### `get_package_details(self, package_name: str) -> dict`
- Fetches detailed information for a specific package.
- Parameters:
- `package_name` (str): The name of the package.
- Returns:
- `dict`: A dictionary containing detailed information about the package.
- Raises:
- `PyPiExtractorError`: If there is an error fetching or parsing the package details.
##### `get_all_packages_details(self) -> list`
- Fetches detailed information for all packages of the given PyPI user.
- Returns:
- `list`: A list of dictionaries containing detailed information about each package.
- Raises:
- `PyPiExtractorError`: If there is an error fetching or processing the package details.
#### `PyPiExtractorError`
Custom exception class for `PyPiExtractor` errors.
<br />
<p align="right"><a href="https://wolfsoftware.com/"><img src="https://img.shields.io/badge/Created%20by%20Wolf%20on%20behalf%20of%20Wolf%20Software-blue?style=for-the-badge" /></a></p>
Raw data
{
"_id": null,
"home_page": "https://github.com/DevelopersToolbox/pypi-extractor-package",
"name": "wolfsoftware.pypi-extractor",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "python, pypi",
"author": "Wolf Software",
"author_email": "pypi@wolfsoftware.com",
"download_url": "https://files.pythonhosted.org/packages/4e/e1/34e07199143f26c64e21ff9902c390dd00242dfecd8fce90bee2355881ec/wolfsoftware_pypi_extractor-0.1.3.tar.gz",
"platform": null,
"description": "<!-- markdownlint-disable -->\n<p align=\"center\">\n <a href=\"https://github.com/DevelopersToolbox/\">\n <img src=\"https://cdn.wolfsoftware.com/assets/images/github/organisations/developerstoolbox/black-and-white-circle-256.png\" alt=\"DevelopersToolbox logo\" />\n </a>\n <br />\n <a href=\"https://github.com/DevelopersToolbox/pypi-extractor-package/actions/workflows/cicd.yml\">\n <img src=\"https://img.shields.io/github/actions/workflow/status/DevelopersToolbox/pypi-extractor-package/cicd.yml?branch=master&label=build%20status&style=for-the-badge\" alt=\"Github Build Status\" />\n </a>\n <a href=\"https://github.com/DevelopersToolbox/pypi-extractor-package/blob/master/LICENSE.md\">\n <img src=\"https://img.shields.io/github/license/DevelopersToolbox/pypi-extractor-package?color=blue&label=License&style=for-the-badge\" alt=\"License\">\n </a>\n <a href=\"https://github.com/DevelopersToolbox/pypi-extractor-package\">\n <img src=\"https://img.shields.io/github/created-at/DevelopersToolbox/pypi-extractor-package?color=blue&label=Created&style=for-the-badge\" alt=\"Created\">\n </a>\n <br />\n <a href=\"https://github.com/DevelopersToolbox/pypi-extractor-package/releases/latest\">\n <img src=\"https://img.shields.io/github/v/release/DevelopersToolbox/pypi-extractor-package?color=blue&label=Latest%20Release&style=for-the-badge\" alt=\"Release\">\n </a>\n <a href=\"https://github.com/DevelopersToolbox/pypi-extractor-package/releases/latest\">\n <img src=\"https://img.shields.io/github/release-date/DevelopersToolbox/pypi-extractor-package?color=blue&label=Released&style=for-the-badge\" alt=\"Released\">\n </a>\n <a href=\"https://github.com/DevelopersToolbox/pypi-extractor-package/releases/latest\">\n <img src=\"https://img.shields.io/github/commits-since/DevelopersToolbox/pypi-extractor-package/latest.svg?color=blue&style=for-the-badge\" alt=\"Commits since release\">\n </a>\n <br />\n <a href=\"https://github.com/DevelopersToolbox/pypi-extractor-package/blob/master/.github/CODE_OF_CONDUCT.md\">\n <img src=\"https://img.shields.io/badge/Code%20of%20Conduct-blue?style=for-the-badge\" />\n </a>\n <a href=\"https://github.com/DevelopersToolbox/pypi-extractor-package/blob/master/.github/CONTRIBUTING.md\">\n <img src=\"https://img.shields.io/badge/Contributing-blue?style=for-the-badge\" />\n </a>\n <a href=\"https://github.com/DevelopersToolbox/pypi-extractor-package/blob/master/.github/SECURITY.md\">\n <img src=\"https://img.shields.io/badge/Report%20Security%20Concern-blue?style=for-the-badge\" />\n </a>\n <a href=\"https://github.com/DevelopersToolbox/pypi-extractor-package/issues\">\n <img src=\"https://img.shields.io/badge/Get%20Support-blue?style=for-the-badge\" />\n </a>\n</p>\n\n## Overview\n\nPyPI Extractor is a Python package designed to fetch and process detailed information about packages hosted on the\nPython Package Index (PyPI). This package is particularly useful for users who want to retrieve and analyze metadata for packages\nmaintained by a specific PyPI user.\n\n## Significant Update From 0.1.3\n\npypi.org no longer allow you to scrap details using the requests package, or any package that does not support JavaScript. To resolve this we have\nupdated this package to utilise [PlayWright](https://pypi.org/project/playwright/) when retrieving a list of packages for a given user. While we have \nattempted to automate as much as possible you might want to do some of the work manually.\n\nPlaywright needs two commands to be run in order for it to function correctly:\n\n```\nplaywright install\nplaywright install-deps\n```\n\nWe have added an `auto_install` option to the main class so that you can instruct the package to do the install for you, this helps when installing the\npackage in a fully automated way, e.g. Puppet or similar.\n\n## Features\n\n- Retrieve a list of packages maintained by a specific PyPI user.\n- Fetch detailed metadata for each package, including versions, author information, dependencies, and more.\n- Custom exceptions for handling errors gracefully.\n- Option to set the PyPI username after initializing the class.\n\n## Installation\n\nYou can install the package using pip:\n\n```sh\npip install wolfsoftware.pypi-extractor\n```\n\n## Usage\n\n### Basic Usage\n\nHere's a basic example of how to use the PyPI Extractor:\n\n```python\nfrom wolfsoftware.pypi_extractor import PyPiExtractor\n\n# Initialize without username\npypi_info = PyPiExtractor()\n\n# Set username later\npypi_info.set_username(\"your_pypi_username\")\n\n# Get detailed information for all packages\ntry:\n packages_details = pypi_info.get_all_packages_details()\n print(packages_details)\nexcept PyPiExtractorError as e:\n print(f\"An error occurred: {e.message}\")\n```\n\n### Setting Username During Initialization\n\nYou can also set the username during initialization:\n\n```python\npypi_info = PyPiExtractor(\"your_pypi_username\")\n```\n\n### Retrieving User Packages\n\nYou can retrieve a list of packages maintained by a specific user:\n\n```python\npackages = pypi_info.get_user_packages()\nprint(packages)\n```\n\n### Retrieving Package Details\n\nTo get detailed information about a specific package:\n\n```python\npackage_details = pypi_info.get_package_details(\"package_name\")\nprint(package_details)\n```\n\n## API Reference\n\n### Classes\n\n#### `PyPiExtractor`\n\nA class to fetch and process package details for a given PyPI user.\n\n##### `__init__(self, username: str, verbose: bool, auto_install: bool)`\n\n- Initializes the `PyPiExtractor` with a username.\n- Parameters:\n - `username` (str): The PyPI username.\n - `verbose` (bool): Verbose output (Default: False)\n - `auto_install` (bool): Auto install PlayWright dependencies (Default: False)\n- Raises:\n - `PyPiExtractorError`: If the username is not provided.\n\n##### `set_username(self, username: str)`\n\n- Sets the PyPI username.\n- Parameters:\n - `username` (str): The PyPI username.\n- Raises:\n - `PyPiExtractorError`: If the username is not provided.\n\n##### `enable_verbose(self)`\n\n- Enable verbose mode.\n\n##### `enable_auto_install(self)`\n\n- Enable auto install.\n\n##### `get_user_packages(self) -> list`\n\n- Fetches the list of packages for the given PyPI user.\n- Returns:\n - `list`: A list of dictionaries containing package names and summaries.\n- Raises:\n - `PyPiExtractorError`: If there is an error fetching or parsing the user profile.\n\n##### `get_package_details(self, package_name: str) -> dict`\n\n- Fetches detailed information for a specific package.\n- Parameters:\n - `package_name` (str): The name of the package.\n- Returns:\n - `dict`: A dictionary containing detailed information about the package.\n- Raises:\n - `PyPiExtractorError`: If there is an error fetching or parsing the package details.\n\n##### `get_all_packages_details(self) -> list`\n\n- Fetches detailed information for all packages of the given PyPI user.\n- Returns:\n - `list`: A list of dictionaries containing detailed information about each package.\n- Raises:\n - `PyPiExtractorError`: If there is an error fetching or processing the package details.\n\n#### `PyPiExtractorError`\n\nCustom exception class for `PyPiExtractor` errors.\n\n<br />\n<p align=\"right\"><a href=\"https://wolfsoftware.com/\"><img src=\"https://img.shields.io/badge/Created%20by%20Wolf%20on%20behalf%20of%20Wolf%20Software-blue?style=for-the-badge\" /></a></p>\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Extract package information for a given user in PyPi.",
"version": "0.1.3",
"project_urls": {
"Documentation": "https://github.com/DevelopersToolbox/pypi-extractor-package",
"Homepage": "https://github.com/DevelopersToolbox/pypi-extractor-package",
"Source": "https://github.com/DevelopersToolbox/pypi-extractor-package",
"Sponsor": "https://github.com/sponsors/WolfSoftware",
"Tracker": "https://github.com/wDevelopersToolbox/pypi-extractor-package/issues/"
},
"split_keywords": [
"python",
" pypi"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "27aeaf620b45487cb24278b9ccf2cff1ab96936ee4e2f7f51aae1284161b7485",
"md5": "4dfa53a795345fc5adb130e38a068451",
"sha256": "ac24fee8802483136c60647695644ebddac622d4eda4847e12f82a4c81ba8651"
},
"downloads": -1,
"filename": "wolfsoftware.pypi_extractor-0.1.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "4dfa53a795345fc5adb130e38a068451",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 7831,
"upload_time": "2024-12-12T15:32:59",
"upload_time_iso_8601": "2024-12-12T15:32:59.395662Z",
"url": "https://files.pythonhosted.org/packages/27/ae/af620b45487cb24278b9ccf2cff1ab96936ee4e2f7f51aae1284161b7485/wolfsoftware.pypi_extractor-0.1.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4ee134e07199143f26c64e21ff9902c390dd00242dfecd8fce90bee2355881ec",
"md5": "cf9a23930816018627b702b2d7d771c5",
"sha256": "6d15cdd78f2a98f1ff9a3d3b51c03b096fb0159c1ea5ff281a19d19a43682928"
},
"downloads": -1,
"filename": "wolfsoftware_pypi_extractor-0.1.3.tar.gz",
"has_sig": false,
"md5_digest": "cf9a23930816018627b702b2d7d771c5",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 10428,
"upload_time": "2024-12-12T15:33:01",
"upload_time_iso_8601": "2024-12-12T15:33:01.780254Z",
"url": "https://files.pythonhosted.org/packages/4e/e1/34e07199143f26c64e21ff9902c390dd00242dfecd8fce90bee2355881ec/wolfsoftware_pypi_extractor-0.1.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-12 15:33:01",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "DevelopersToolbox",
"github_project": "pypi-extractor-package",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "requests",
"specs": [
[
"==",
"2.32.3"
]
]
},
{
"name": "beautifulsoup4",
"specs": [
[
"==",
"4.12.3"
]
]
},
{
"name": "playwright",
"specs": [
[
"==",
"1.49.1"
]
]
}
],
"lcname": "wolfsoftware.pypi-extractor"
}
JSON
