# Weibo Downloader
## Overview
This tool does not require login authentication.
This tool downloads texts, pictures (jpg), and videos (mp4) of weibo posts to local PC or Mac.
Please ensure that Google Chrome is installed on your PC or Mac.
As a tool designed to mimic front-end user interactions, in contrast to API-based alternatives that though are more susceptible to media download restrictions, expect a slower speed in data retrieval.
## Introduction
Weibo Downloader is a front-end-based tool specifically designed for downloading Weibo posts and media (texts, pictures and videos). It is created for content download for data analysis and record-keeping. This tool operates without the need for user login and is unaffected by changes to the Weibo API, thanks to its front-end-based approach. However, it is not intended for fast-response scraping, and users should expect slower speeds due to web interactions.
## Features
- No login required for accessing public Weibo posts.
- Unaffected by Weibo API changes due to front-end-based operation.
- Ability to customize characteristic class names for post parsing with `self.dinstict_class_names`.
- Download media content including images and videos.
- Save posts in JSON or CSV formats.
- Configurable for specific date ranges or page limits.
## Installation
```bash
pip install weibo-downloader
```
## Usage
Weibo Downloader can be used in two primary ways:
1. **Use get_weibo_posts_by_name Function Directly**:
Directly fetch posts for a specific username using the **get_weibo_posts_by_name** function. This method returns a generator to iterate over posts.
```python
from weibo_downloader import get_weibo_posts_by_name
for post in get_weibo_posts_by_name("your_username", pages=1):
# process post, such as printing
print(post)
```
2. **Instantiate WeiboDownloader and Run**:
Create an instance of **WeiboDownloader** with the desired configuration and call the run() method to start fetching posts.
```python
from weibo_downloader import WeiboDownloader
downloader = WeiboDownloader(username="your_username", pages=1)
downloader.run()
``````
## Input Parameters
- **username**: Weibo username (string).
- **uid**: Weibo user ID (string).
- **save_path_csv**: Path to save posts in CSV format (string).
- **save_path_json**: Path to save posts in JSON format (string).
- **save_media_directory**: Directory to save downloaded media (string).
- **enable_get_video_links**: Enable fetching video links (bool).
- **enable_get_urls**: Enable fetching URLs from the posts (bool).
- **enable_fill_truncated_texts**: Enable filling truncated texts (bool).
- **enable_download_media_all**: Download all media (bool).
- **enable_download_media_image_only**: Download only images (bool).
- **enable_download_media_video_only**: Download only videos (bool).
- **enable_download_media_overwrite**: Overwrite existing media files (bool).
- **enable_simplified_json**: Enable simplified JSON structure (bool).
- **date_from**: Start date for fetching posts (YYYY-MM-DD).
- **date_to**: End date for fetching posts (YYYY-MM-DD).
- **pages**: Number of pages to fetch (int).
- **weibo_timeline_url_prefix**: URL prefix for Weibo timeline (string).
## Customization
Users can customize characteristic class names used for parsing posts through the self.dinstict_class_names attribute, allowing for flexibility in case of changes in the Weibo front-end structure.
## Limitations
Slower speed due to web interactions.
Intended for data analysis and record-keeping, not suited for real-time scraping.
## License
MIT License. See LICENSE for more
information.
Raw data
{
"_id": null,
"home_page": "https://github.com/xuejianma/weibo-downloader",
"name": "weibo-downloader",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "weibo,downloader,scraper,crawler,video,image",
"author": "Xuejian Ma",
"author_email": "Xuejian.Ma@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/58/43/cd7102a313c627c95c04ff842a91a65931726b3468ea02aaa13d43a00532/weibo-downloader-0.1.0.tar.gz",
"platform": null,
"description": "# Weibo Downloader\r\n\r\n## Overview\r\nThis tool does not require login authentication.\r\n\r\nThis tool downloads texts, pictures (jpg), and videos (mp4) of weibo posts to local PC or Mac.\r\n\r\nPlease ensure that Google Chrome is installed on your PC or Mac.\r\n\r\nAs a tool designed to mimic front-end user interactions, in contrast to API-based alternatives that though are more susceptible to media download restrictions, expect a slower speed in data retrieval.\r\n\r\n## Introduction\r\nWeibo Downloader is a front-end-based tool specifically designed for downloading Weibo posts and media (texts, pictures and videos). It is created for content download for data analysis and record-keeping. This tool operates without the need for user login and is unaffected by changes to the Weibo API, thanks to its front-end-based approach. However, it is not intended for fast-response scraping, and users should expect slower speeds due to web interactions.\r\n\r\n## Features\r\n- No login required for accessing public Weibo posts.\r\n- Unaffected by Weibo API changes due to front-end-based operation.\r\n- Ability to customize characteristic class names for post parsing with `self.dinstict_class_names`.\r\n- Download media content including images and videos.\r\n- Save posts in JSON or CSV formats.\r\n- Configurable for specific date ranges or page limits.\r\n\r\n## Installation\r\n```bash\r\npip install weibo-downloader\r\n```\r\n\r\n## Usage\r\nWeibo Downloader can be used in two primary ways:\r\n\r\n1. **Use get_weibo_posts_by_name Function Directly**:\r\n\r\n Directly fetch posts for a specific username using the **get_weibo_posts_by_name** function. This method returns a generator to iterate over posts.\r\n ```python\r\n from weibo_downloader import get_weibo_posts_by_name\r\n for post in get_weibo_posts_by_name(\"your_username\", pages=1):\r\n # process post, such as printing\r\n print(post)\r\n ```\r\n\r\n2. **Instantiate WeiboDownloader and Run**:\r\n\r\n Create an instance of **WeiboDownloader** with the desired configuration and call the run() method to start fetching posts.\r\n\r\n ```python\r\n from weibo_downloader import WeiboDownloader\r\n downloader = WeiboDownloader(username=\"your_username\", pages=1)\r\n downloader.run()\r\n ``````\r\n\r\n## Input Parameters\r\n- **username**: Weibo username (string).\r\n- **uid**: Weibo user ID (string).\r\n- **save_path_csv**: Path to save posts in CSV format (string).\r\n- **save_path_json**: Path to save posts in JSON format (string).\r\n- **save_media_directory**: Directory to save downloaded media (string).\r\n- **enable_get_video_links**: Enable fetching video links (bool).\r\n- **enable_get_urls**: Enable fetching URLs from the posts (bool).\r\n- **enable_fill_truncated_texts**: Enable filling truncated texts (bool).\r\n- **enable_download_media_all**: Download all media (bool).\r\n- **enable_download_media_image_only**: Download only images (bool).\r\n- **enable_download_media_video_only**: Download only videos (bool).\r\n- **enable_download_media_overwrite**: Overwrite existing media files (bool).\r\n- **enable_simplified_json**: Enable simplified JSON structure (bool).\r\n- **date_from**: Start date for fetching posts (YYYY-MM-DD).\r\n- **date_to**: End date for fetching posts (YYYY-MM-DD).\r\n- **pages**: Number of pages to fetch (int).\r\n- **weibo_timeline_url_prefix**: URL prefix for Weibo timeline (string).\r\n\r\n## Customization\r\nUsers can customize characteristic class names used for parsing posts through the self.dinstict_class_names attribute, allowing for flexibility in case of changes in the Weibo front-end structure.\r\n\r\n## Limitations\r\nSlower speed due to web interactions.\r\nIntended for data analysis and record-keeping, not suited for real-time scraping.\r\n## License\r\nMIT License. See LICENSE for more\r\ninformation.\r\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Download weibo posts and media including texts, images and videos.",
"version": "0.1.0",
"project_urls": {
"Homepage": "https://github.com/xuejianma/weibo-downloader"
},
"split_keywords": [
"weibo",
"downloader",
"scraper",
"crawler",
"video",
"image"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "ae251147317af87caba0db71d75e5883b079989c3772cd435bbe9d9081050b04",
"md5": "0172f56e42309a9608f9a2071e826837",
"sha256": "59cc2ea31f76736852dc554c8221beb7877bedb306c5e61447aa7cd286f78c50"
},
"downloads": -1,
"filename": "weibo_downloader-0.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "0172f56e42309a9608f9a2071e826837",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 10096,
"upload_time": "2023-11-26T08:35:04",
"upload_time_iso_8601": "2023-11-26T08:35:04.819735Z",
"url": "https://files.pythonhosted.org/packages/ae/25/1147317af87caba0db71d75e5883b079989c3772cd435bbe9d9081050b04/weibo_downloader-0.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "5843cd7102a313c627c95c04ff842a91a65931726b3468ea02aaa13d43a00532",
"md5": "7ffd86991ef2235172bd93c7fbcdfd45",
"sha256": "70296fea529f807710ffe7dc6547d82ab1784ea3ca73466fcfa64809f0a523f8"
},
"downloads": -1,
"filename": "weibo-downloader-0.1.0.tar.gz",
"has_sig": false,
"md5_digest": "7ffd86991ef2235172bd93c7fbcdfd45",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 11593,
"upload_time": "2023-11-26T08:35:19",
"upload_time_iso_8601": "2023-11-26T08:35:19.007540Z",
"url": "https://files.pythonhosted.org/packages/58/43/cd7102a313c627c95c04ff842a91a65931726b3468ea02aaa13d43a00532/weibo-downloader-0.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-11-26 08:35:19",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "xuejianma",
"github_project": "weibo-downloader",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "weibo-downloader"
}
JSON
