# Automatic text summarizer
[](https://github.com/miso-belica/sumy/actions/workflows/run-tests.yml)
[](https://gitpod.io/#https://github.com/miso-belica/sumy)
Simple library and command line utility for extracting summary from HTML
pages or plain texts. The package also contains simple evaluation
framework for text summaries. Implemented summarization methods are described in the [documentation](docs/summarizators.md). I also maintain a list of [alternative implementations](docs/alternatives.md) of the summarizers in various programming languages.
## Is my natural language supported?
There is a [good chance](docs/index.md#Tokenizer) it is. But if not it is [not too hard to add](docs/how-to-add-new-language.md) it.
## Installation
Make sure you have [Python](http://www.python.org/) 3.6+ and
[pip](https://crate.io/packages/pip/)
([Windows](http://docs.python-guide.org/en/latest/starting/install/win/),
[Linux](http://docs.python-guide.org/en/latest/starting/install/linux/))
installed. Run simply (preferred way):
```sh
$ [sudo] pip install sumy
$ [sudo] pip install git+git://github.com/miso-belica/sumy.git # for the fresh version
```
## Usage
Thanks to some good soul out there, the easiest way to try sumy is in your browser at https://huggingface.co/spaces/issam9/sumy_space
Sumy contains command line utility for quick summarization of documents.
```sh
$ sumy lex-rank --length=10 --url=https://en.wikipedia.org/wiki/Automatic_summarization # what's summarization?
$ sumy lex-rank --language=uk --length=30 --url=https://uk.wikipedia.org/wiki/Україна
$ sumy luhn --language=czech --url=https://www.zdrojak.cz/clanky/automaticke-zabezpeceni/
$ sumy edmundson --language=czech --length=3% --url=https://cs.wikipedia.org/wiki/Bitva_u_Lipan
$ sumy --help # for more info
```
Various evaluation methods for some summarization method can be executed
by commands below:
```sh
$ sumy_eval lex-rank reference_summary.txt --url=https://en.wikipedia.org/wiki/Automatic_summarization
$ sumy_eval lsa reference_summary.txt --language=czech --url=https://www.zdrojak.cz/clanky/automaticke-zabezpeceni/
$ sumy_eval edmundson reference_summary.txt --language=czech --url=https://cs.wikipedia.org/wiki/Bitva_u_Lipan
$ sumy_eval --help # for more info
```
If you don't want to bother by the installation, you can try it as a container.
```sh
$ docker run --rm misobelica/sumy lex-rank --length=10 --url=https://en.wikipedia.org/wiki/Automatic_summarization
```
## Python API
Or you can use sumy like a library in your project. Create file `sumy_example.py` ([don't name it `sumy.py`](https://stackoverflow.com/questions/41334622/python-sumy-no-module-named-sumy-parsers-html)) with the code below to test it.
```python
# -*- coding: utf-8 -*-
from __future__ import absolute_import
from __future__ import division, print_function, unicode_literals
from sumy.parsers.html import HtmlParser
from sumy.parsers.plaintext import PlaintextParser
from sumy.nlp.tokenizers import Tokenizer
from sumy.summarizers.lsa import LsaSummarizer as Summarizer
from sumy.nlp.stemmers import Stemmer
from sumy.utils import get_stop_words
LANGUAGE = "english"
SENTENCES_COUNT = 10
if __name__ == "__main__":
url = "https://en.wikipedia.org/wiki/Automatic_summarization"
parser = HtmlParser.from_url(url, Tokenizer(LANGUAGE))
# or for plain text files
# parser = PlaintextParser.from_file("document.txt", Tokenizer(LANGUAGE))
# parser = PlaintextParser.from_string("Check this out.", Tokenizer(LANGUAGE))
stemmer = Stemmer(LANGUAGE)
summarizer = Summarizer(stemmer)
summarizer.stop_words = get_stop_words(LANGUAGE)
for sentence in summarizer(parser.document, SENTENCES_COUNT):
print(sentence)
```
## Interesting projects using sumy
I found some interesting projects while browsing the internet or sometimes people wrote me an e-mail with questions, and I was curious how they use the sumy :)
* [Learning to generate questions from text](https://software.intel.com/en-us/articles/using-natural-language-processing-for-smart-question-generation) - https://github.com/adityasarvaiya/Automatic_Question_Generation
* Summarize your video to any duration - https://github.com/aswanthkoleri/VideoMash and similar https://github.com/OpenGenus/vidsum
* Tool for collectively summarizing large discussions - https://github.com/amyxzhang/wikum
Raw data
{
"_id": null,
"home_page": "https://github.com/miso-belica/sumy",
"name": "sumy",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "data mining,automatic summarization,data reduction,web-data extraction,NLP,natural language processing,latent semantic analysis,LSA,TextRank,LexRank",
"author": "Mi\u0161o Belica",
"author_email": "miso.belica@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/df/fd/59098f716c39422d5ca47caa97ef44cd2829384bfdf22e1420e839fde3c1/sumy-0.11.0.tar.gz",
"platform": null,
"description": "# Automatic text summarizer\n\n\n[](https://github.com/miso-belica/sumy/actions/workflows/run-tests.yml)\n[](https://gitpod.io/#https://github.com/miso-belica/sumy) \n\nSimple library and command line utility for extracting summary from HTML\npages or plain texts. The package also contains simple evaluation\nframework for text summaries. Implemented summarization methods are described in the [documentation](docs/summarizators.md). I also maintain a list of [alternative implementations](docs/alternatives.md) of the summarizers in various programming languages.\n\n## Is my natural language supported?\nThere is a [good chance](docs/index.md#Tokenizer) it is. But if not it is [not too hard to add](docs/how-to-add-new-language.md) it.\n\n## Installation\n\nMake sure you have [Python](http://www.python.org/) 3.6+ and\n[pip](https://crate.io/packages/pip/)\n([Windows](http://docs.python-guide.org/en/latest/starting/install/win/),\n[Linux](http://docs.python-guide.org/en/latest/starting/install/linux/))\ninstalled. Run simply (preferred way):\n\n```sh\n$ [sudo] pip install sumy\n$ [sudo] pip install git+git://github.com/miso-belica/sumy.git # for the fresh version\n```\n\n## Usage\n\nThanks to some good soul out there, the easiest way to try sumy is in your browser at https://huggingface.co/spaces/issam9/sumy_space\n\nSumy contains command line utility for quick summarization of documents.\n\n```sh\n$ sumy lex-rank --length=10 --url=https://en.wikipedia.org/wiki/Automatic_summarization # what's summarization?\n$ sumy lex-rank --language=uk --length=30 --url=https://uk.wikipedia.org/wiki/\u0423\u043a\u0440\u0430\u0457\u043d\u0430\n$ sumy luhn --language=czech --url=https://www.zdrojak.cz/clanky/automaticke-zabezpeceni/\n$ sumy edmundson --language=czech --length=3% --url=https://cs.wikipedia.org/wiki/Bitva_u_Lipan\n$ sumy --help # for more info\n```\n\nVarious evaluation methods for some summarization method can be executed\nby commands below:\n\n```sh\n$ sumy_eval lex-rank reference_summary.txt --url=https://en.wikipedia.org/wiki/Automatic_summarization\n$ sumy_eval lsa reference_summary.txt --language=czech --url=https://www.zdrojak.cz/clanky/automaticke-zabezpeceni/\n$ sumy_eval edmundson reference_summary.txt --language=czech --url=https://cs.wikipedia.org/wiki/Bitva_u_Lipan\n$ sumy_eval --help # for more info\n```\n\nIf you don't want to bother by the installation, you can try it as a container.\n\n```sh\n$ docker run --rm misobelica/sumy lex-rank --length=10 --url=https://en.wikipedia.org/wiki/Automatic_summarization\n```\n\n## Python API\n\nOr you can use sumy like a library in your project. Create file `sumy_example.py` ([don't name it `sumy.py`](https://stackoverflow.com/questions/41334622/python-sumy-no-module-named-sumy-parsers-html)) with the code below to test it.\n\n```python\n# -*- coding: utf-8 -*-\n\nfrom __future__ import absolute_import\nfrom __future__ import division, print_function, unicode_literals\n\nfrom sumy.parsers.html import HtmlParser\nfrom sumy.parsers.plaintext import PlaintextParser\nfrom sumy.nlp.tokenizers import Tokenizer\nfrom sumy.summarizers.lsa import LsaSummarizer as Summarizer\nfrom sumy.nlp.stemmers import Stemmer\nfrom sumy.utils import get_stop_words\n\n\nLANGUAGE = \"english\"\nSENTENCES_COUNT = 10\n\n\nif __name__ == \"__main__\":\n url = \"https://en.wikipedia.org/wiki/Automatic_summarization\"\n parser = HtmlParser.from_url(url, Tokenizer(LANGUAGE))\n # or for plain text files\n # parser = PlaintextParser.from_file(\"document.txt\", Tokenizer(LANGUAGE))\n # parser = PlaintextParser.from_string(\"Check this out.\", Tokenizer(LANGUAGE))\n stemmer = Stemmer(LANGUAGE)\n\n summarizer = Summarizer(stemmer)\n summarizer.stop_words = get_stop_words(LANGUAGE)\n\n for sentence in summarizer(parser.document, SENTENCES_COUNT):\n print(sentence)\n```\n\n## Interesting projects using sumy\n\nI found some interesting projects while browsing the internet or sometimes people wrote me an e-mail with questions, and I was curious how they use the sumy :)\n\n* [Learning to generate questions from text](https://software.intel.com/en-us/articles/using-natural-language-processing-for-smart-question-generation) - https://github.com/adityasarvaiya/Automatic_Question_Generation\n* Summarize your video to any duration - https://github.com/aswanthkoleri/VideoMash and similar https://github.com/OpenGenus/vidsum\n* Tool for collectively summarizing large discussions - https://github.com/amyxzhang/wikum\n",
"bugtrack_url": null,
"license": "Apache License, Version 2.0",
"summary": "Module for automatic summarization of text documents and HTML pages.",
"version": "0.11.0",
"project_urls": {
"Homepage": "https://github.com/miso-belica/sumy"
},
"split_keywords": [
"data mining",
"automatic summarization",
"data reduction",
"web-data extraction",
"nlp",
"natural language processing",
"latent semantic analysis",
"lsa",
"textrank",
"lexrank"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "194677859104e7c3e12dfa2e5c0e27b5dd1e14cb2409b50f9c936a48f29ceaee",
"md5": "7ad11711f53ae22432d9caa47e802f3a",
"sha256": "e74323ef95f42be8c76464a758a003597729b60cceb918f4806fdadad3eeb1a8"
},
"downloads": -1,
"filename": "sumy-0.11.0-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "7ad11711f53ae22432d9caa47e802f3a",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 97327,
"upload_time": "2022-10-23T16:40:35",
"upload_time_iso_8601": "2022-10-23T16:40:35.104787Z",
"url": "https://files.pythonhosted.org/packages/19/46/77859104e7c3e12dfa2e5c0e27b5dd1e14cb2409b50f9c936a48f29ceaee/sumy-0.11.0-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "dffd59098f716c39422d5ca47caa97ef44cd2829384bfdf22e1420e839fde3c1",
"md5": "2f565021806da2521cf0cf5da3872837",
"sha256": "1c52dcfda3072d87c7ae85a100c53a93f5b1f758466a1a1369f86aff1bb0129d"
},
"downloads": -1,
"filename": "sumy-0.11.0.tar.gz",
"has_sig": false,
"md5_digest": "2f565021806da2521cf0cf5da3872837",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 78223,
"upload_time": "2022-10-23T16:40:37",
"upload_time_iso_8601": "2022-10-23T16:40:37.540119Z",
"url": "https://files.pythonhosted.org/packages/df/fd/59098f716c39422d5ca47caa97ef44cd2829384bfdf22e1420e839fde3c1/sumy-0.11.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-10-23 16:40:37",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "miso-belica",
"github_project": "sumy",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "sumy"
}
JSON
