psdelivery

Name	psdelivery JSON
Version	0.1.2 JSON
	download
home_page	https://github.com/team-angeline/psdelivery
Summary	The simple crawler about algorithm problem service website
upload_time	2023-04-24 17:46:09
maintainer
docs_url	None
author	recoma
requires_python	>=3.10,<4.0
license	GPL-3.0-only
keywords	crwaler problem-solving
VCS
bugtrack_url
requirements	async-generator attrs beautifulsoup4 certifi cffi charset-normalizer colorama exceptiongroup h11 idna outcome packaging pycparser pysocks python-dotenv requests selenium sniffio sortedcontainers soupsieve tqdm trio-websocket trio urllib3 urllib3 webdriver-manager wsproto
Travis-CI	No Travis.
coveralls test coverage

            # 📦 psdelivery

[![Ubuntu](https://github.com/team-angeline/psdelivery/actions/workflows/test-ubuntu.yml/badge.svg)](https://github.com/team-angeline/psdelivery/actions/workflows/test-ubuntu.yml)
[![Windows](https://github.com/team-angeline/psdelivery/actions/workflows/test-windows.yml/badge.svg)](https://github.com/team-angeline/psdelivery/actions/workflows/test-windows.yml)
[![CI](https://github.com/team-angeline/psdelivery/actions/workflows/ci.yml/badge.svg)](https://github.com/team-angeline/psdelivery/actions/workflows/ci.yml)
![GitHub](https://img.shields.io/github/license/team-angeline/psdelivery)
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/psdelivery?logo=python&logoColor=white)
[![PyPI](https://img.shields.io/pypi/v/psdelivery?label=pypi%20package&logo=pypi&logoColor=white)](https://pypi.org/project/psdelivery)
[![codecov](https://codecov.io/gh/team-angeline/psdelivery/branch/main/graph/badge.svg?token=LFC7Z4PGCT)](https://codecov.io/gh/team-angeline/psdelivery)

```shell
pip install psdelivery
```

## Introduction 
여러 코딩 사이트의 코딩 문제들을 크롤링하는 라이브러리 입니다. 초기 버전이라 기본 수준의 기능만 지원하지만. 차후 더 많은 기능들을 추가할 예정입니다. 왜냐면 재가 제 메인 프로젝트에 쓸 거니깐요.

## 🙋‍♂️ Functions

### Implemented
* 해당 사이트에서 코딩문제를 가져올 수 있습니다.
    * 백준 (solved.ac)
    * 리트코드 (leetcode)
* 파이썬 프로젝트에 모듈로 사용할 수 있고, CLI 환경에서도 바로 사용할 수 있습니다.
    * CLI 환경해서 명령어를 통해 실행할 경우, 결과 데이터는 JSON으로 저장됩니다.

### Will be Implemented
* 해당 사이트에도 코딩문제를 가져올 예정입니다.
    * 코드포스 (codeforces)
    * 해커랭크 (hackerrank)
* 여러 페이지 범위의 문제 리스트를 뽑아올 수 있습니다.
    * 해당 사이트의 모든 문제들도 하나의 명령어로 가져올 수 있습니다.
* 특정 문제에 대한 세부 정보를 가져올 수 있습니다.
    * 알고리즘 태그를 가져올 수 있습니다.

## ⭐ Required
* 반드시 크롬이 설치되어 있어야 합니다.
    * Debian계열의 경우, bin디렉토리에 크롬을 설치하는 Shell Script가 있습니다.
* 파이썬 버전은 반드시 ```3.10``` 이상이어야 합니다.

## 💽 How To Install For Developer
크롬과 파이썬이 설치되어있다는 가정 하에 설명합니다.

1. requirements.txt를 통해 패키지를 설치합니다.
2. 끗

## Usage
### As Command Line
✔️ **Version 확인하기**
```shell
python -m psdelivery version

# 0.1.0
```

📚 **문제 리스트 가져오기**
```
python -m psdelivery getlist -t <topic> -sp <page index> -o <output json file>
```
* **Options**
    * -t(--topic): 코딩 페이지 사이트를 명시합니다. 데이터를 가지고올 수 있는 사이트는 다음과 같습니다.
        * 백준: ```baekjoon``` 또는 ```solved.ac```
        * 리트코드: ```leetcode```
    * -sp(--single-page): 페이지 인덱스를 나타냅니다. 해당 옵션을 명시하지 않으면 1페이지의 문제 리스트를 가져옵니다.
    * -o(--output): 코딩문제 데이터를 저장할 파일 루트를 명시합니다.
* **example**
    ```
    python -m psdelivery getlist -t baekjoon -sp 3 -o output.json
    ```

### As Python Module

```python
from psdelivery import PsDelivery

"""
PsDelivery 객체 생성
topic은 크롤링할 사이트 이름을 입력합니다.

백준: baekjoon 또는 solved.ac
리트코드: leetcode
"""
crawler = PsDelivery(topic='leetcode')

"""
특정 페이지의 문제 리스트를 가져옵니다.
page:
    가져올 페이지 쪽수를 입력합니다. 반드시 1 이상이어야 합니다.
serialize:
    일반적인 리턴된 리스트의 요소는 ProblemItem이라는 객체 입니다. 
    serialize=True로 설정하면 ProblemItem을 Dict 형태로 직렬화 합니다. 
    Default값은 False 입니다.
"""
result = crwaler.get_list_by_single_page(page=1)
"""
return: [<ProbmeItem>, <ProblemItem>, ...]
"""

result_as_json = crawler.get_list_by_single_page(page=2, serialize=True)
"""
return: [<Dict>, <Dict>, ...]
"""
```


            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/team-angeline/psdelivery",
    "name": "psdelivery",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.10,<4.0",
    "maintainer_email": "",
    "keywords": "crwaler,problem-solving",
    "author": "recoma",
    "author_email": "seokbong60@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/b6/9d/0235f6942aa61f1b80abf45fa975cc654af7a71d26bc51e770eed4d24857/psdelivery-0.1.2.tar.gz",
    "platform": null,
    "description": "# \ud83d\udce6 psdelivery\n\n[![Ubuntu](https://github.com/team-angeline/psdelivery/actions/workflows/test-ubuntu.yml/badge.svg)](https://github.com/team-angeline/psdelivery/actions/workflows/test-ubuntu.yml)\n[![Windows](https://github.com/team-angeline/psdelivery/actions/workflows/test-windows.yml/badge.svg)](https://github.com/team-angeline/psdelivery/actions/workflows/test-windows.yml)\n[![CI](https://github.com/team-angeline/psdelivery/actions/workflows/ci.yml/badge.svg)](https://github.com/team-angeline/psdelivery/actions/workflows/ci.yml)\n![GitHub](https://img.shields.io/github/license/team-angeline/psdelivery)\n![PyPI - Python Version](https://img.shields.io/pypi/pyversions/psdelivery?logo=python&logoColor=white)\n[![PyPI](https://img.shields.io/pypi/v/psdelivery?label=pypi%20package&logo=pypi&logoColor=white)](https://pypi.org/project/psdelivery)\n[![codecov](https://codecov.io/gh/team-angeline/psdelivery/branch/main/graph/badge.svg?token=LFC7Z4PGCT)](https://codecov.io/gh/team-angeline/psdelivery)\n\n```shell\npip install psdelivery\n```\n\n## Introduction \n\uc5ec\ub7ec \ucf54\ub529 \uc0ac\uc774\ud2b8\uc758 \ucf54\ub529 \ubb38\uc81c\ub4e4\uc744 \ud06c\ub864\ub9c1\ud558\ub294 \ub77c\uc774\ube0c\ub7ec\ub9ac \uc785\ub2c8\ub2e4. \ucd08\uae30 \ubc84\uc804\uc774\ub77c \uae30\ubcf8 \uc218\uc900\uc758 \uae30\ub2a5\ub9cc \uc9c0\uc6d0\ud558\uc9c0\ub9cc. \ucc28\ud6c4 \ub354 \ub9ce\uc740 \uae30\ub2a5\ub4e4\uc744 \ucd94\uac00\ud560 \uc608\uc815\uc785\ub2c8\ub2e4. \uc65c\ub0d0\uba74 \uc7ac\uac00 \uc81c \uba54\uc778 \ud504\ub85c\uc81d\ud2b8\uc5d0 \uc4f8 \uac70\ub2c8\uae50\uc694.\n\n## \ud83d\ude4b\u200d\u2642\ufe0f Functions\n\n### Implemented\n* \ud574\ub2f9 \uc0ac\uc774\ud2b8\uc5d0\uc11c \ucf54\ub529\ubb38\uc81c\ub97c \uac00\uc838\uc62c \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n    * \ubc31\uc900 (solved.ac)\n    * \ub9ac\ud2b8\ucf54\ub4dc (leetcode)\n* \ud30c\uc774\uc36c \ud504\ub85c\uc81d\ud2b8\uc5d0 \ubaa8\ub4c8\ub85c \uc0ac\uc6a9\ud560 \uc218 \uc788\uace0, CLI \ud658\uacbd\uc5d0\uc11c\ub3c4 \ubc14\ub85c \uc0ac\uc6a9\ud560 \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n    * CLI \ud658\uacbd\ud574\uc11c \uba85\ub839\uc5b4\ub97c \ud1b5\ud574 \uc2e4\ud589\ud560 \uacbd\uc6b0, \uacb0\uacfc \ub370\uc774\ud130\ub294 JSON\uc73c\ub85c \uc800\uc7a5\ub429\ub2c8\ub2e4.\n\n### Will be Implemented\n* \ud574\ub2f9 \uc0ac\uc774\ud2b8\uc5d0\ub3c4 \ucf54\ub529\ubb38\uc81c\ub97c \uac00\uc838\uc62c \uc608\uc815\uc785\ub2c8\ub2e4.\n    * \ucf54\ub4dc\ud3ec\uc2a4 (codeforces)\n    * \ud574\ucee4\ub7ad\ud06c (hackerrank)\n* \uc5ec\ub7ec \ud398\uc774\uc9c0 \ubc94\uc704\uc758 \ubb38\uc81c \ub9ac\uc2a4\ud2b8\ub97c \ubf51\uc544\uc62c \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n    * \ud574\ub2f9 \uc0ac\uc774\ud2b8\uc758 \ubaa8\ub4e0 \ubb38\uc81c\ub4e4\ub3c4 \ud558\ub098\uc758 \uba85\ub839\uc5b4\ub85c \uac00\uc838\uc62c \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n* \ud2b9\uc815 \ubb38\uc81c\uc5d0 \ub300\ud55c \uc138\ubd80 \uc815\ubcf4\ub97c \uac00\uc838\uc62c \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n    * \uc54c\uace0\ub9ac\uc998 \ud0dc\uadf8\ub97c \uac00\uc838\uc62c \uc218 \uc788\uc2b5\ub2c8\ub2e4.\n\n## \u2b50 Required\n* \ubc18\ub4dc\uc2dc \ud06c\ub86c\uc774 \uc124\uce58\ub418\uc5b4 \uc788\uc5b4\uc57c \ud569\ub2c8\ub2e4.\n    * Debian\uacc4\uc5f4\uc758 \uacbd\uc6b0, bin\ub514\ub809\ud1a0\ub9ac\uc5d0 \ud06c\ub86c\uc744 \uc124\uce58\ud558\ub294 Shell Script\uac00 \uc788\uc2b5\ub2c8\ub2e4.\n* \ud30c\uc774\uc36c \ubc84\uc804\uc740 \ubc18\ub4dc\uc2dc ```3.10``` \uc774\uc0c1\uc774\uc5b4\uc57c \ud569\ub2c8\ub2e4.\n\n## \ud83d\udcbd How To Install For Developer\n\ud06c\ub86c\uacfc \ud30c\uc774\uc36c\uc774 \uc124\uce58\ub418\uc5b4\uc788\ub2e4\ub294 \uac00\uc815 \ud558\uc5d0 \uc124\uba85\ud569\ub2c8\ub2e4.\n\n1. requirements.txt\ub97c \ud1b5\ud574 \ud328\ud0a4\uc9c0\ub97c \uc124\uce58\ud569\ub2c8\ub2e4.\n2. \ub057\n\n## Usage\n### As Command Line\n\u2714\ufe0f **Version \ud655\uc778\ud558\uae30**\n```shell\npython -m psdelivery version\n\n# 0.1.0\n```\n\n\ud83d\udcda **\ubb38\uc81c \ub9ac\uc2a4\ud2b8 \uac00\uc838\uc624\uae30**\n```\npython -m psdelivery getlist -t <topic> -sp <page index> -o <output json file>\n```\n* **Options**\n    * -t(--topic): \ucf54\ub529 \ud398\uc774\uc9c0 \uc0ac\uc774\ud2b8\ub97c \uba85\uc2dc\ud569\ub2c8\ub2e4. \ub370\uc774\ud130\ub97c \uac00\uc9c0\uace0\uc62c \uc218 \uc788\ub294 \uc0ac\uc774\ud2b8\ub294 \ub2e4\uc74c\uacfc \uac19\uc2b5\ub2c8\ub2e4.\n        * \ubc31\uc900: ```baekjoon``` \ub610\ub294 ```solved.ac```\n        * \ub9ac\ud2b8\ucf54\ub4dc: ```leetcode```\n    * -sp(--single-page): \ud398\uc774\uc9c0 \uc778\ub371\uc2a4\ub97c \ub098\ud0c0\ub0c5\ub2c8\ub2e4. \ud574\ub2f9 \uc635\uc158\uc744 \uba85\uc2dc\ud558\uc9c0 \uc54a\uc73c\uba74 1\ud398\uc774\uc9c0\uc758 \ubb38\uc81c \ub9ac\uc2a4\ud2b8\ub97c \uac00\uc838\uc635\ub2c8\ub2e4.\n    * -o(--output): \ucf54\ub529\ubb38\uc81c \ub370\uc774\ud130\ub97c \uc800\uc7a5\ud560 \ud30c\uc77c \ub8e8\ud2b8\ub97c \uba85\uc2dc\ud569\ub2c8\ub2e4.\n* **example**\n    ```\n    python -m psdelivery getlist -t baekjoon -sp 3 -o output.json\n    ```\n\n### As Python Module\n\n```python\nfrom psdelivery import PsDelivery\n\n\"\"\"\nPsDelivery \uac1d\uccb4 \uc0dd\uc131\ntopic\uc740 \ud06c\ub864\ub9c1\ud560 \uc0ac\uc774\ud2b8 \uc774\ub984\uc744 \uc785\ub825\ud569\ub2c8\ub2e4.\n\n\ubc31\uc900: baekjoon \ub610\ub294 solved.ac\n\ub9ac\ud2b8\ucf54\ub4dc: leetcode\n\"\"\"\ncrawler = PsDelivery(topic='leetcode')\n\n\"\"\"\n\ud2b9\uc815 \ud398\uc774\uc9c0\uc758 \ubb38\uc81c \ub9ac\uc2a4\ud2b8\ub97c \uac00\uc838\uc635\ub2c8\ub2e4.\npage:\n    \uac00\uc838\uc62c \ud398\uc774\uc9c0 \ucabd\uc218\ub97c \uc785\ub825\ud569\ub2c8\ub2e4. \ubc18\ub4dc\uc2dc 1 \uc774\uc0c1\uc774\uc5b4\uc57c \ud569\ub2c8\ub2e4.\nserialize:\n    \uc77c\ubc18\uc801\uc778 \ub9ac\ud134\ub41c \ub9ac\uc2a4\ud2b8\uc758 \uc694\uc18c\ub294 ProblemItem\uc774\ub77c\ub294 \uac1d\uccb4 \uc785\ub2c8\ub2e4. \n    serialize=True\ub85c \uc124\uc815\ud558\uba74 ProblemItem\uc744 Dict \ud615\ud0dc\ub85c \uc9c1\ub82c\ud654 \ud569\ub2c8\ub2e4. \n    Default\uac12\uc740 False \uc785\ub2c8\ub2e4.\n\"\"\"\nresult = crwaler.get_list_by_single_page(page=1)\n\"\"\"\nreturn: [<ProbmeItem>, <ProblemItem>, ...]\n\"\"\"\n\nresult_as_json = crawler.get_list_by_single_page(page=2, serialize=True)\n\"\"\"\nreturn: [<Dict>, <Dict>, ...]\n\"\"\"\n```\n\n",
    "bugtrack_url": null,
    "license": "GPL-3.0-only",
    "summary": "The simple crawler about algorithm problem service website",
    "version": "0.1.2",
    "split_keywords": [
        "crwaler",
        "problem-solving"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "e4e34f18e88ff3a2375eaa85f3d6f36a200648d57472ccfdf145a94680cdbed9",
                "md5": "92e7ef5abb66426790dd526164512493",
                "sha256": "61e1f3222bebb711db48310502feb80b424742ab54ff73bbbf98aaac4d6dced7"
            },
            "downloads": -1,
            "filename": "psdelivery-0.1.2-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "92e7ef5abb66426790dd526164512493",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.10,<4.0",
            "size": 26072,
            "upload_time": "2023-04-24T17:46:07",
            "upload_time_iso_8601": "2023-04-24T17:46:07.758670Z",
            "url": "https://files.pythonhosted.org/packages/e4/e3/4f18e88ff3a2375eaa85f3d6f36a200648d57472ccfdf145a94680cdbed9/psdelivery-0.1.2-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "b69d0235f6942aa61f1b80abf45fa975cc654af7a71d26bc51e770eed4d24857",
                "md5": "622656f20f93f8777d4efcfb5dc218f1",
                "sha256": "7d30a73ffa440b935cd2735a6677ab9edba6d745faa92fecb1ac0a7173b04121"
            },
            "downloads": -1,
            "filename": "psdelivery-0.1.2.tar.gz",
            "has_sig": false,
            "md5_digest": "622656f20f93f8777d4efcfb5dc218f1",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.10,<4.0",
            "size": 22794,
            "upload_time": "2023-04-24T17:46:09",
            "upload_time_iso_8601": "2023-04-24T17:46:09.566447Z",
            "url": "https://files.pythonhosted.org/packages/b6/9d/0235f6942aa61f1b80abf45fa975cc654af7a71d26bc51e770eed4d24857/psdelivery-0.1.2.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-04-24 17:46:09",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "github_user": "team-angeline",
    "github_project": "psdelivery",
    "travis_ci": false,
    "coveralls": true,
    "github_actions": true,
    "requirements": [
        {
            "name": "async-generator",
            "specs": [
                [
                    "==",
                    "1.10"
                ]
            ]
        },
        {
            "name": "attrs",
            "specs": [
                [
                    "==",
                    "23.1.0"
                ]
            ]
        },
        {
            "name": "beautifulsoup4",
            "specs": [
                [
                    "==",
                    "4.12.2"
                ]
            ]
        },
        {
            "name": "certifi",
            "specs": [
                [
                    "==",
                    "2022.12.7"
                ]
            ]
        },
        {
            "name": "cffi",
            "specs": [
                [
                    "==",
                    "1.15.1"
                ]
            ]
        },
        {
            "name": "charset-normalizer",
            "specs": [
                [
                    "==",
                    "3.1.0"
                ]
            ]
        },
        {
            "name": "colorama",
            "specs": [
                [
                    "==",
                    "0.4.6"
                ]
            ]
        },
        {
            "name": "exceptiongroup",
            "specs": [
                [
                    "==",
                    "1.1.1"
                ]
            ]
        },
        {
            "name": "h11",
            "specs": [
                [
                    "==",
                    "0.14.0"
                ]
            ]
        },
        {
            "name": "idna",
            "specs": [
                [
                    "==",
                    "3.4"
                ]
            ]
        },
        {
            "name": "outcome",
            "specs": [
                [
                    "==",
                    "1.2.0"
                ]
            ]
        },
        {
            "name": "packaging",
            "specs": [
                [
                    "==",
                    "23.1"
                ]
            ]
        },
        {
            "name": "pycparser",
            "specs": [
                [
                    "==",
                    "2.21"
                ]
            ]
        },
        {
            "name": "pysocks",
            "specs": [
                [
                    "==",
                    "1.7.1"
                ]
            ]
        },
        {
            "name": "python-dotenv",
            "specs": [
                [
                    "==",
                    "1.0.0"
                ]
            ]
        },
        {
            "name": "requests",
            "specs": [
                [
                    "==",
                    "2.28.2"
                ]
            ]
        },
        {
            "name": "selenium",
            "specs": [
                [
                    "==",
                    "4.9.0"
                ]
            ]
        },
        {
            "name": "sniffio",
            "specs": [
                [
                    "==",
                    "1.3.0"
                ]
            ]
        },
        {
            "name": "sortedcontainers",
            "specs": [
                [
                    "==",
                    "2.4.0"
                ]
            ]
        },
        {
            "name": "soupsieve",
            "specs": [
                [
                    "==",
                    "2.4.1"
                ]
            ]
        },
        {
            "name": "tqdm",
            "specs": [
                [
                    "==",
                    "4.65.0"
                ]
            ]
        },
        {
            "name": "trio-websocket",
            "specs": [
                [
                    "==",
                    "0.10.2"
                ]
            ]
        },
        {
            "name": "trio",
            "specs": [
                [
                    "==",
                    "0.22.0"
                ]
            ]
        },
        {
            "name": "urllib3",
            "specs": [
                [
                    "==",
                    "1.26.15"
                ]
            ]
        },
        {
            "name": "urllib3",
            "specs": [
                [
                    "==",
                    "1.26.15"
                ]
            ]
        },
        {
            "name": "webdriver-manager",
            "specs": [
                [
                    "==",
                    "3.8.6"
                ]
            ]
        },
        {
            "name": "wsproto",
            "specs": [
                [
                    "==",
                    "1.2.0"
                ]
            ]
        }
    ],
    "lcname": "psdelivery"
}