rbpy-rb


Namerbpy-rb JSON
Version 0.10.11 PyPI version JSON
download
home_pagehttps://git.readerbench.com/ReaderBench/Readerbench-python
SummaryReaderBench library written in python
upload_time2021-01-11 22:11:11
maintainer
docs_urlNone
authorWoodcarver
requires_python
license
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # ReaderBench Python

## Install
We recommend using virtual environments, as some packages require an exact version.   
If you only want to use the package do the following:  
1. `sudo apt-get install python3-pip, python3-venv, python3.6, python3-dev`    
2. `python3 -m venv rbenv` (create virutal environment named rbenv)
3. `source rbenv/bin/activate` (activate virtual env)
4. `pip3 uninstall setuptools && pip3 install setuptools && pip3 install --upgrade pip && pip3 install --no-cache-dir rbpy-rb`
5. Use it as in: https://git.readerbench.com/ReaderBench/readerbenchpy/blob/master/usage.py  

If you want to contribute to the code base of package:   
1. `sudo apt-get install python3-pip, python3-venv, python3.6, python3-dev`    
2. `git clone git@git.readerbench.com:ReaderBench/readerbenchpy.git && cd readerbenchpy/`  
3. `python3 -m venv rbenv` (create virutal environment named rbenv)
4. `source rbenv/bin/activate` (activate virtual env)
5. `pip3 uninstall setuptools && pip3 install setuptools && pip3 install --upgrade pip`
6. `pip3 install -r requirements.txt` 
7. `python3 -m spacy download xx_ent_wiki_sm`
8. `python3 nltk_download.py`  
Optional: prei-install model for en (otherwise most of the English processings would fail
    and ask to run this command):
9. `sudo python3 -m spacy download en_core_web_lg`


If you want to install spellchecking (hunspell) also you need this non-python libraries:
1. `sudo apt-get install libhunspell-1.6-0 libhunspell-dev hunspell-ro`
2. `pip3 install hunspell`

## Usage
For usage (parsing, lemmatization, NER, wordnet, content words, indices etc.)  see file `usage.py` from 
https://git.readerbench.com/ReaderBench/readerbenchpy    

Check main.py (`python3 main.py --help`) to see main processings available.

## Tips
You may also need some spacy models which are downloaded through spacy.     
You have to download these spacy models by yourself, using the command:    
`python3 -m spacy download name_of_the_model`   (do not install them with sudo if you are in a virtual environment)
The logger will also write instructions on which models you need, and how to download them.  
Be careful, you need to have spacy 2.1.3. 
If you change the version of spacy (you had a previous version) you need to reinstall xx_ent_wiki_sm model.

For neural coref errors install it as follows: https://github.com/huggingface/neuralcoref#spacystringsstringstore-size-changed-error

## Developer instructions

## How to use Bert
```
from rb.core.lang import Lang
from rb.processings.encoders.bert import BertWrapper
from tensorflow import keras

bert_wrapper = BertWrapper(Lang.RO, max_seq_len=128)
inputs, bert_layer = bert_wrapper.create_inputs_and_model()
cls_output = bert_wrapper.get_output(bert_layer, "cls") # or "pool"

# Add decision layer and compile model
# eg. 
# hidden = keras.layers.Dense(..)(cls_output)
# output = keras.layers.Dense(..)(hidden)
# model = keras.Model(inputs=inputs, outputs=[output])
# model.compile(..)

bert_wrapper.load_weights() #must be called after compile

# Process inputs for model
feed_inputs = bert_wrapper.process_input(["text1", "text2", "text3"])
# feed_output = ...
# model.fit(feed_inputs, feed_output, ...)
```

## How to use the logger
In each file you have to initialize the logger:  
```sh
from rb.utils.rblogger import Logger  
logger = Logger.get_logger() 
logger.info("info msg")
logger.warning("warning msg")  
logger.error()
```
## How to push the wheel on pip
1. `rm -r dist/`
2. `pip3 install twine wheel`
3. `./upload_to_pypi.sh`


## How to run rb/core/cscl/csv_parser.py
1. Do the installing steps from contribution
2. run `pip3 install xmltodict`
3. run `EXPORT PYTHONPATH=/add/path/to/repo/readerbenchpy/`
4. add json resources in a `jsons` directory in `readerbenchpy/rb/core/cscl/`
5. run `cd rb/core/cscl/ && python3 csv_parser.py`

## Supported Date Formats
ReaderBench is able to perform conversation analysis from chats and communities. Each utterance must have the time expressed in one of the following formats:
- %Y-%m-%d %H:%M:%S.%f %Z
- %Y-%m-%d %H:%M:%S %Z
- %Y-%m-%d %H:%M %Z
- %Y-%m-%d %H:%M:%S.%f
- %Y-%m-%d %H:%M:%S
- %Y-%m-%d %H:%M
where codifications are extracted from [Python date format codes](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes).
            

Raw data

            {
    "_id": null,
    "home_page": "https://git.readerbench.com/ReaderBench/Readerbench-python",
    "name": "rbpy-rb",
    "maintainer": "",
    "docs_url": null,
    "requires_python": "",
    "maintainer_email": "",
    "keywords": "",
    "author": "Woodcarver",
    "author_email": "batpepastrama@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/dc/d9/98ab1bce93e5e06269c539727e350b4aca764cbba06867bb01a235aa8011/rbpy-rb-0.10.11.tar.gz",
    "platform": "",
    "description": "# ReaderBench Python\n\n## Install\nWe recommend using virtual environments, as some packages require an exact version.   \nIf you only want to use the package do the following:  \n1. `sudo apt-get install python3-pip, python3-venv, python3.6, python3-dev`    \n2. `python3 -m venv rbenv` (create virutal environment named rbenv)\n3. `source rbenv/bin/activate` (activate virtual env)\n4. `pip3 uninstall setuptools && pip3 install setuptools && pip3 install --upgrade pip && pip3 install --no-cache-dir rbpy-rb`\n5. Use it as in: https://git.readerbench.com/ReaderBench/readerbenchpy/blob/master/usage.py  \n\nIf you want to contribute to the code base of package:   \n1. `sudo apt-get install python3-pip, python3-venv, python3.6, python3-dev`    \n2. `git clone git@git.readerbench.com:ReaderBench/readerbenchpy.git && cd readerbenchpy/`  \n3. `python3 -m venv rbenv` (create virutal environment named rbenv)\n4. `source rbenv/bin/activate` (activate virtual env)\n5. `pip3 uninstall setuptools && pip3 install setuptools && pip3 install --upgrade pip`\n6. `pip3 install -r requirements.txt` \n7. `python3 -m spacy download xx_ent_wiki_sm`\n8. `python3 nltk_download.py`  \nOptional: prei-install model for en (otherwise most of the English processings would fail\n    and ask to run this command):\n9. `sudo python3 -m spacy download en_core_web_lg`\n\n\nIf you want to install spellchecking (hunspell) also you need this non-python libraries:\n1. `sudo apt-get install libhunspell-1.6-0 libhunspell-dev hunspell-ro`\n2. `pip3 install hunspell`\n\n## Usage\nFor usage (parsing, lemmatization, NER, wordnet, content words, indices etc.)  see file `usage.py` from \nhttps://git.readerbench.com/ReaderBench/readerbenchpy    \n\nCheck main.py (`python3 main.py --help`) to see main processings available.\n\n## Tips\nYou may also need some spacy models which are downloaded through spacy.     \nYou have to download these spacy models by yourself, using the command:    \n`python3 -m spacy download name_of_the_model`   (do not install them with sudo if you are in a virtual environment)\nThe logger will also write instructions on which models you need, and how to download them.  \nBe careful, you need to have spacy 2.1.3. \nIf you change the version of spacy (you had a previous version) you need to reinstall xx_ent_wiki_sm model.\n\nFor neural coref errors install it as follows: https://github.com/huggingface/neuralcoref#spacystringsstringstore-size-changed-error\n\n## Developer instructions\n\n## How to use Bert\n```\nfrom rb.core.lang import Lang\nfrom rb.processings.encoders.bert import BertWrapper\nfrom tensorflow import keras\n\nbert_wrapper = BertWrapper(Lang.RO, max_seq_len=128)\ninputs, bert_layer = bert_wrapper.create_inputs_and_model()\ncls_output = bert_wrapper.get_output(bert_layer, \"cls\") # or \"pool\"\n\n# Add decision layer and compile model\n# eg. \n# hidden = keras.layers.Dense(..)(cls_output)\n# output = keras.layers.Dense(..)(hidden)\n# model = keras.Model(inputs=inputs, outputs=[output])\n# model.compile(..)\n\nbert_wrapper.load_weights() #must be called after compile\n\n# Process inputs for model\nfeed_inputs = bert_wrapper.process_input([\"text1\", \"text2\", \"text3\"])\n# feed_output = ...\n# model.fit(feed_inputs, feed_output, ...)\n```\n\n## How to use the logger\nIn each file you have to initialize the logger:  \n```sh\nfrom rb.utils.rblogger import Logger  \nlogger = Logger.get_logger() \nlogger.info(\"info msg\")\nlogger.warning(\"warning msg\")  \nlogger.error()\n```\n## How to push the wheel on pip\n1. `rm -r dist/`\n2. `pip3 install twine wheel`\n3. `./upload_to_pypi.sh`\n\n\n## How to run rb/core/cscl/csv_parser.py\n1. Do the installing steps from contribution\n2. run `pip3 install xmltodict`\n3. run `EXPORT PYTHONPATH=/add/path/to/repo/readerbenchpy/`\n4. add json resources in a `jsons` directory in `readerbenchpy/rb/core/cscl/`\n5. run `cd rb/core/cscl/ && python3 csv_parser.py`\n\n## Supported Date Formats\nReaderBench is able to perform conversation analysis from chats and communities. Each utterance must have the time expressed in one of the following formats:\n- %Y-%m-%d %H:%M:%S.%f %Z\n- %Y-%m-%d %H:%M:%S %Z\n- %Y-%m-%d %H:%M %Z\n- %Y-%m-%d %H:%M:%S.%f\n- %Y-%m-%d %H:%M:%S\n- %Y-%m-%d %H:%M\nwhere codifications are extracted from [Python date format codes](https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes).",
    "bugtrack_url": null,
    "license": "",
    "summary": "ReaderBench library written in python",
    "version": "0.10.11",
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "md5": "88be2d6e103fdebf494373af71627432",
                "sha256": "6774b30f629a1ac0cac9f105fd75590c8f307a66178f83a2d1fe15557edd44f6"
            },
            "downloads": -1,
            "filename": "rbpy-rb-0.10.11.tar.gz",
            "has_sig": false,
            "md5_digest": "88be2d6e103fdebf494373af71627432",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 1488376,
            "upload_time": "2021-01-11T22:11:11",
            "upload_time_iso_8601": "2021-01-11T22:11:11.710071Z",
            "url": "https://files.pythonhosted.org/packages/dc/d9/98ab1bce93e5e06269c539727e350b4aca764cbba06867bb01a235aa8011/rbpy-rb-0.10.11.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2021-01-11 22:11:11",
    "github": false,
    "gitlab": false,
    "bitbucket": false,
    "lcname": "rbpy-rb"
}
        
Elapsed time: 0.18193s