cappr


Namecappr JSON
Version 0.9.0 PyPI version JSON
download
home_pagehttps://github.com/kddubey/cappr/
SummaryCompletion After Prompt Probability. Make your LLM make a choice
upload_time2024-02-01 21:46:29
maintainer
docs_urlNone
author
requires_python>=3.8.0
licenseApache License 2.0
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # CAPPr: Completion After Prompt Probability

[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg?logo=python&style=for-the-badge)](https://www.python.org/downloads/release/python-380/)
[![tests](https://img.shields.io/github/actions/workflow/status/kddubey/cappr/test.yml?style=for-the-badge&logo=github&label=tests)](https://github.com/kddubey/cappr/actions/workflows/test.yml)
[![codecov](https://img.shields.io/codecov/c/github/kddubey/cappr?token=NYIL076PSM&style=for-the-badge&logo=codecov&color=%2309BC00)](https://codecov.io/gh/kddubey/cappr)
[![PyPI - Package Version](https://img.shields.io/pypi/v/cappr?logo=pypi&style=for-the-badge&color=orange)](https://pypi.org/project/cappr/)
[![License](https://img.shields.io/badge/License-Apache_2.0-purple.svg?logo=apache&style=for-the-badge)](https://opensource.org/licenses/Apache-2.0)

<!-- [![Documentation Status](https://readthedocs.org/projects/cappr/badge/?version=latest&style=for-the-badge)](https://cappr.readthedocs.io/en/latest/?badge=latest) -->


CAPPr performs text classification. No training. No post-processing. <br>
Just have your LLM pick from a list of choices. <br>
Or compute the probability of a completion given a prompt. <br>
Squeeze [more](https://cappr.readthedocs.io/en/latest/statistical_performance.html) out
of open source LLMs.


## Usage

<details>
<summary>Use a GGUF model</summary>

This model must be able to be loaded using
[`llama_cpp.Llama`](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama).

```python
from llama_cpp import Llama
from cappr.llama_cpp.classify import predict

# Load model
model = Llama("./TinyLLama-v0.Q8_0.gguf", verbose=False)

prompt = """Gary told Spongebob a story:
There once was a man from Peru; who dreamed he was eating his shoe. He
woke with a fright, in the middle of the night, to find that his dream
had come true.

The moral of the story is to"""

completions = (
  "look at the bright side",
  "use your imagination",
  "eat shoes",
)

pred = predict(prompt, completions, model)
print(pred)
# use your imagination
```

Notice that a completion can contain many tokens. CAPPr is **100% guaranteed** to return
an output from the list of possible answers.

See [this page of the
documentation](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#llama-cpp)
for more info on using GGUF models.
</details>


<details>
<summary>Use a HuggingFace AutoModelForCausalLM</summary>

This model must be able to be loaded using
[`transformers.AutoModelForCausalLM.from_pretrained`](https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoModelForCausalLM).

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from cappr.huggingface.classify import predict

# Load a model and its tokenizer
model_name = "gpt2"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompt = "Which planet is closer to the Sun: Mercury or Earth?"
completions = ("Mercury", "Earth")

pred = predict(prompt, completions, model_and_tokenizer=(model, tokenizer))
print(pred)
# Mercury
```

See [this page of the
documentation](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#huggingface)
for more info on using ``transformers`` models.
</details>


<details>
<summary>Use an AutoGPTQ model</summary>

[`cappr.huggingface`](https://cappr.readthedocs.io/en/latest/cappr.huggingface.html) is
compatible with models loaded via
[`auto_gptq.AutoGPTQForCausalLM.from_quantized`](https://github.com/PanQiWei/AutoGPTQ).
See [this
notebook](https://github.com/kddubey/cappr/blob/main/demos/huggingface/auto_gptq.ipynb)
for a minimal demo.

Note that for `transformers>=4.32.0`, you can load GPTQ models using
`transformers.AutoModelForCausalLM`.

See [this page of the
documentation](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#huggingface)
for more info on using these models.
</details>


<details>
<summary>Use an AutoAWQ model</summary>

[`cappr.huggingface.classify_no_cache`](https://cappr.readthedocs.io/en/latest/cappr.huggingface.html)
is compatible with models loaded via
[`awq.AutoAWQForCausalLM.from_quantized`](https://github.com/casper-hansen/AutoAWQ). See
[this
notebook](https://github.com/kddubey/cappr/blob/main/demos/huggingface/autoawq.ipynb)
for a minimal demo.

Note that for `transformers>=4.35.0`, you can load AWQ models using
`transformers.AutoModelForCausalLM`. AWQ models loaded this way are compatible with
[`cappr.huggingface.classify`](https://cappr.readthedocs.io/en/latest/cappr.huggingface.html),
which is usually faster.

See [this page of the
documentation](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#huggingface)
for more info on using these models.
</details>


<details>
<summary>Use a model from the OpenAI API</summary>

This model must be compatible with the
[/v1/completions](https://platform.openai.com/docs/models/model-endpoint-compatibility)
endpoint
([excluding](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#openai)
``gpt-3.5-turbo-instruct``).

```python
from cappr.openai.classify import predict

prompt = """
Tweet about a movie: "Oppenheimer was pretty good. But 3 hrs...cmon Nolan."
This tweet contains the following criticism:
""".strip("\n")

completions = ("bad message", "too long", "unfunny")

pred = predict(prompt, completions, model="text-ada-001")
print(pred)
# too long
```

See [this page of the
documentation](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#openai)
for more info on using OpenAI models.
</details>


<details>
<summary>Extract the final answer from a step-by-step completion</summary>

Step-by-step and chain-of-thought prompts are highly effective ways to get an LLM to
"reason" about more complex tasks. But if you need a structured output, a step-by-step
completion is unwieldy. Use CAPPr to extract the final answer from these types of
completions, given a list of possible answers.

See this idea in action [here in the
documentation](https://cappr.readthedocs.io/en/latest/select_a_prompt_completion_format.html#wrangle-step-by-step-completions).
</details>


<details>
<summary>Run in batches</summary>

Let's use a PyTorch ``transformers`` model. Also, let's predict probabilities instead of
the class.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from cappr.huggingface.classify import predict_proba

# Load a model and its tokenizer
model_name = "gpt2"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

prompts = [
    "Stephen Curry is a",
    "Martina Navratilova was a",
    "Dexter, from the TV Series Dexter's Laboratory, is a",
    "LeBron James is a",
]

# Each of the prompts could be completed with one of these:
class_names = ("basketball player", "tennis player", "scientist")
prior =       (      1/6,                1/6,            2/3    )
# Say I expect most of my data to have scientists

# Run CAPPr
pred_probs = predict_proba(
    prompts=prompts,
    completions=class_names,
    model_and_tokenizer=(model, tokenizer),
    batch_size=32,  # whatever fits on your CPU/GPU
    prior=prior,
)

# pred_probs[i,j] = probability that prompts[i] is classified as class_names[j]
print(pred_probs.round(1))
# [[0.5 0.3 0.2]
#  [0.3 0.6 0.2]
#  [0.1 0.1 0.8]
#  [0.8 0.2 0. ]]

# For each prompt, which completion is most likely?
pred_class_idxs = pred_probs.argmax(axis=-1)
preds = [class_names[pred_class_idx] for pred_class_idx in pred_class_idxs]
print(preds)
# ['basketball player',
#  'tennis player',
#  'scientist',
#  'basketball player']
```
</details>


<details>
<summary>Run in batches, where each prompt has a different set of possible completions
</summary>

Again, let's use a PyTorch ``transformers`` model to predict probabilities.

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from cappr.huggingface.classify import predict_proba_examples
from cappr import Example

# Load a model and its tokenizer
model_name = "gpt2"
model = AutoModelForCausalLM.from_pretrained(model_name)
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Create a sequence of Example objects representing your classification tasks
examples = [
    Example(
        prompt="Jodie Foster played",
        completions=("Clarice Starling", "Trinity in The Matrix"),
    ),
    Example(
        prompt="Batman, from Batman: The Animated Series, was played by",
        completions=("Pete Holmes", "Kevin Conroy", "Spongebob!"),
        prior=      (     1/3      ,      2/3     ,      0      ),
    ),
]

# Run CAPPr
pred_probs = predict_proba_examples(
    examples, model_and_tokenizer=(model, tokenizer)
)

# pred_probs[i][j] = probability that examples[i].prompt is classified as
# examples[i].completions[j]
print([example_pred_probs.round(2) for example_pred_probs in pred_probs])
# [array([0.7, 0.3]),
#  array([0.03, 0.97, 0.  ])]

# For each example, which completion is most likely?
pred_class_idxs = [
    example_pred_probs.argmax() for example_pred_probs in pred_probs
]
preds = [
    example.completions[pred_class_idx]
    for example, pred_class_idx in zip(examples, pred_class_idxs)
]
print(preds)
# ['Clarice Starling',
#  'Kevin Conroy']
```
</details>


See the [`demos`](https://github.com/kddubey/cappr/blob/main/demos/) for demonstrations
of slightly harder classification tasks.

For CAPPr, GPTQ models are the most computationally performant. These models are
compatible with `cappr.huggingface.classify`. See [this page of the
documentation](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#huggingface)
for more info on using these models.


## Documentation

https://cappr.readthedocs.io


## Installation

See [this page of the
documentation](https://cappr.readthedocs.io/en/latest/installation.html).


## Motivation

Reduce engineering complexity.

See [this page of the
documentation](https://cappr.readthedocs.io/en/latest/motivation.html) for more info.


## Performance

<details>
<summary>
Statistical performance
</summary>

For open source models, see

- the 4-bit 4 GB Llama 2 [COPA
  demo](https://github.com/kddubey/cappr/blob/main/demos/llama_cpp/superglue/copa.ipynb)
- the 4-bit 4 GB Llama 2 [AG News
  demo](https://github.com/kddubey/cappr/blob/main/demos/llama_cpp/ag_news.ipynb)
- the 4 GB Mistral [Craigslist Bargains
  demo](https://github.com/kddubey/cappr/blob/main/demos/huggingface/craigslist_bargains.ipynb)
- the 4 GB Mistral [Banking 77
  demo](https://github.com/kddubey/cappr/blob/main/demos/huggingface/banking_77_classes.ipynb),
  with 77 multi-token choices
- the 4 GB Mistral [SciQ
  demo](https://github.com/kddubey/cappr/blob/main/demos/huggingface/sciq.ipynb), with a
  multiple choice prompt
- the 4 GB Mistral [Tweet Emotions
  demo](https://github.com/kddubey/cappr/blob/main/demos/huggingface/tweet_emotion_multilabel.ipynb),
  which is a multilabel task—each tweet can be tagged with 1-12 emotions.

In general, you should expect similar or identical performance to text generation when
every completion is 1 token long.

See [this page of the
documentation](https://cappr.readthedocs.io/en/latest/statistical_performance.html)
for some discussion.

For OpenAI models (some deprecated), see

[2 SuperGLUE
datasets](https://github.com/kddubey/cappr/blob/main/demos/openai/superglue)

[RAFT zero-shot training
sets](https://github.com/kddubey/cappr/blob/main/demos/openai/raft)
</details>


<details>
<summary>
Computational performance
</summary>

See [this page of the
documentation](https://cappr.readthedocs.io/en/latest/computational_performance.html).
</details>


## How it works

You input a `prompt` string, a `end_of_prompt` string (a whitespace or empty) and a set
of candidate `completion` strings such that the string—

```python
{prompt}{end_of_prompt}{completion}
```

—is a naturally flowing thought. CAPPr picks the `completion` which is mostly likely to
follow `prompt` by computing the:

> **C**ompletion<br>
  **A**fter<br>
  **P**rompt<br>
  **Pr**obability<br>

The method is fleshed out in my [question on Cross
Validated](https://stats.stackexchange.com/q/601159/337906).


## Related work

See [this page of the
documentation](https://cappr.readthedocs.io/en/latest/related_work.html).


## Local development

See [this page of the documentation](https://cappr.readthedocs.io/en/latest/local.html).


## Todo

I'm dumping todos here:

[Code changes](https://github.com/users/kddubey/projects/1/views/1)

[Reseach experiments](https://github.com/users/kddubey/projects/2)

Feel free to raise issues ofc

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/kddubey/cappr/",
    "name": "cappr",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.8.0",
    "maintainer_email": "",
    "keywords": "",
    "author": "",
    "author_email": "kushdubey63@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/50/c0/4050ef56faffd5e0c70db91da3d771d07bc02f6f8916937f0984e3a5963a/cappr-0.9.0.tar.gz",
    "platform": null,
    "description": "# CAPPr: Completion After Prompt Probability\n\n[![Python 3.8+](https://img.shields.io/badge/python-3.8+-blue.svg?logo=python&style=for-the-badge)](https://www.python.org/downloads/release/python-380/)\n[![tests](https://img.shields.io/github/actions/workflow/status/kddubey/cappr/test.yml?style=for-the-badge&logo=github&label=tests)](https://github.com/kddubey/cappr/actions/workflows/test.yml)\n[![codecov](https://img.shields.io/codecov/c/github/kddubey/cappr?token=NYIL076PSM&style=for-the-badge&logo=codecov&color=%2309BC00)](https://codecov.io/gh/kddubey/cappr)\n[![PyPI - Package Version](https://img.shields.io/pypi/v/cappr?logo=pypi&style=for-the-badge&color=orange)](https://pypi.org/project/cappr/)\n[![License](https://img.shields.io/badge/License-Apache_2.0-purple.svg?logo=apache&style=for-the-badge)](https://opensource.org/licenses/Apache-2.0)\n\n<!-- [![Documentation Status](https://readthedocs.org/projects/cappr/badge/?version=latest&style=for-the-badge)](https://cappr.readthedocs.io/en/latest/?badge=latest) -->\n\n\nCAPPr performs text classification. No training. No post-processing. <br>\nJust have your LLM pick from a list of choices. <br>\nOr compute the probability of a completion given a prompt. <br>\nSqueeze [more](https://cappr.readthedocs.io/en/latest/statistical_performance.html) out\nof open source LLMs.\n\n\n## Usage\n\n<details>\n<summary>Use a GGUF model</summary>\n\nThis model must be able to be loaded using\n[`llama_cpp.Llama`](https://llama-cpp-python.readthedocs.io/en/latest/api-reference/#llama_cpp.Llama).\n\n```python\nfrom llama_cpp import Llama\nfrom cappr.llama_cpp.classify import predict\n\n# Load model\nmodel = Llama(\"./TinyLLama-v0.Q8_0.gguf\", verbose=False)\n\nprompt = \"\"\"Gary told Spongebob a story:\nThere once was a man from Peru; who dreamed he was eating his shoe. He\nwoke with a fright, in the middle of the night, to find that his dream\nhad come true.\n\nThe moral of the story is to\"\"\"\n\ncompletions = (\n  \"look at the bright side\",\n  \"use your imagination\",\n  \"eat shoes\",\n)\n\npred = predict(prompt, completions, model)\nprint(pred)\n# use your imagination\n```\n\nNotice that a completion can contain many tokens. CAPPr is **100% guaranteed** to return\nan output from the list of possible answers.\n\nSee [this page of the\ndocumentation](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#llama-cpp)\nfor more info on using GGUF models.\n</details>\n\n\n<details>\n<summary>Use a HuggingFace AutoModelForCausalLM</summary>\n\nThis model must be able to be loaded using\n[`transformers.AutoModelForCausalLM.from_pretrained`](https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoModelForCausalLM).\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nfrom cappr.huggingface.classify import predict\n\n# Load a model and its tokenizer\nmodel_name = \"gpt2\"\nmodel = AutoModelForCausalLM.from_pretrained(model_name)\ntokenizer = AutoTokenizer.from_pretrained(model_name)\n\nprompt = \"Which planet is closer to the Sun: Mercury or Earth?\"\ncompletions = (\"Mercury\", \"Earth\")\n\npred = predict(prompt, completions, model_and_tokenizer=(model, tokenizer))\nprint(pred)\n# Mercury\n```\n\nSee [this page of the\ndocumentation](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#huggingface)\nfor more info on using ``transformers`` models.\n</details>\n\n\n<details>\n<summary>Use an AutoGPTQ model</summary>\n\n[`cappr.huggingface`](https://cappr.readthedocs.io/en/latest/cappr.huggingface.html) is\ncompatible with models loaded via\n[`auto_gptq.AutoGPTQForCausalLM.from_quantized`](https://github.com/PanQiWei/AutoGPTQ).\nSee [this\nnotebook](https://github.com/kddubey/cappr/blob/main/demos/huggingface/auto_gptq.ipynb)\nfor a minimal demo.\n\nNote that for `transformers>=4.32.0`, you can load GPTQ models using\n`transformers.AutoModelForCausalLM`.\n\nSee [this page of the\ndocumentation](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#huggingface)\nfor more info on using these models.\n</details>\n\n\n<details>\n<summary>Use an AutoAWQ model</summary>\n\n[`cappr.huggingface.classify_no_cache`](https://cappr.readthedocs.io/en/latest/cappr.huggingface.html)\nis compatible with models loaded via\n[`awq.AutoAWQForCausalLM.from_quantized`](https://github.com/casper-hansen/AutoAWQ). See\n[this\nnotebook](https://github.com/kddubey/cappr/blob/main/demos/huggingface/autoawq.ipynb)\nfor a minimal demo.\n\nNote that for `transformers>=4.35.0`, you can load AWQ models using\n`transformers.AutoModelForCausalLM`. AWQ models loaded this way are compatible with\n[`cappr.huggingface.classify`](https://cappr.readthedocs.io/en/latest/cappr.huggingface.html),\nwhich is usually faster.\n\nSee [this page of the\ndocumentation](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#huggingface)\nfor more info on using these models.\n</details>\n\n\n<details>\n<summary>Use a model from the OpenAI API</summary>\n\nThis model must be compatible with the\n[/v1/completions](https://platform.openai.com/docs/models/model-endpoint-compatibility)\nendpoint\n([excluding](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#openai)\n``gpt-3.5-turbo-instruct``).\n\n```python\nfrom cappr.openai.classify import predict\n\nprompt = \"\"\"\nTweet about a movie: \"Oppenheimer was pretty good. But 3 hrs...cmon Nolan.\"\nThis tweet contains the following criticism:\n\"\"\".strip(\"\\n\")\n\ncompletions = (\"bad message\", \"too long\", \"unfunny\")\n\npred = predict(prompt, completions, model=\"text-ada-001\")\nprint(pred)\n# too long\n```\n\nSee [this page of the\ndocumentation](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#openai)\nfor more info on using OpenAI models.\n</details>\n\n\n<details>\n<summary>Extract the final answer from a step-by-step completion</summary>\n\nStep-by-step and chain-of-thought prompts are highly effective ways to get an LLM to\n\"reason\" about more complex tasks. But if you need a structured output, a step-by-step\ncompletion is unwieldy. Use CAPPr to extract the final answer from these types of\ncompletions, given a list of possible answers.\n\nSee this idea in action [here in the\ndocumentation](https://cappr.readthedocs.io/en/latest/select_a_prompt_completion_format.html#wrangle-step-by-step-completions).\n</details>\n\n\n<details>\n<summary>Run in batches</summary>\n\nLet's use a PyTorch ``transformers`` model. Also, let's predict probabilities instead of\nthe class.\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nfrom cappr.huggingface.classify import predict_proba\n\n# Load a model and its tokenizer\nmodel_name = \"gpt2\"\nmodel = AutoModelForCausalLM.from_pretrained(model_name)\ntokenizer = AutoTokenizer.from_pretrained(model_name)\n\nprompts = [\n    \"Stephen Curry is a\",\n    \"Martina Navratilova was a\",\n    \"Dexter, from the TV Series Dexter's Laboratory, is a\",\n    \"LeBron James is a\",\n]\n\n# Each of the prompts could be completed with one of these:\nclass_names = (\"basketball player\", \"tennis player\", \"scientist\")\nprior =       (      1/6,                1/6,            2/3    )\n# Say I expect most of my data to have scientists\n\n# Run CAPPr\npred_probs = predict_proba(\n    prompts=prompts,\n    completions=class_names,\n    model_and_tokenizer=(model, tokenizer),\n    batch_size=32,  # whatever fits on your CPU/GPU\n    prior=prior,\n)\n\n# pred_probs[i,j] = probability that prompts[i] is classified as class_names[j]\nprint(pred_probs.round(1))\n# [[0.5 0.3 0.2]\n#  [0.3 0.6 0.2]\n#  [0.1 0.1 0.8]\n#  [0.8 0.2 0. ]]\n\n# For each prompt, which completion is most likely?\npred_class_idxs = pred_probs.argmax(axis=-1)\npreds = [class_names[pred_class_idx] for pred_class_idx in pred_class_idxs]\nprint(preds)\n# ['basketball player',\n#  'tennis player',\n#  'scientist',\n#  'basketball player']\n```\n</details>\n\n\n<details>\n<summary>Run in batches, where each prompt has a different set of possible completions\n</summary>\n\nAgain, let's use a PyTorch ``transformers`` model to predict probabilities.\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\nfrom cappr.huggingface.classify import predict_proba_examples\nfrom cappr import Example\n\n# Load a model and its tokenizer\nmodel_name = \"gpt2\"\nmodel = AutoModelForCausalLM.from_pretrained(model_name)\ntokenizer = AutoTokenizer.from_pretrained(model_name)\n\n# Create a sequence of Example objects representing your classification tasks\nexamples = [\n    Example(\n        prompt=\"Jodie Foster played\",\n        completions=(\"Clarice Starling\", \"Trinity in The Matrix\"),\n    ),\n    Example(\n        prompt=\"Batman, from Batman: The Animated Series, was played by\",\n        completions=(\"Pete Holmes\", \"Kevin Conroy\", \"Spongebob!\"),\n        prior=      (     1/3      ,      2/3     ,      0      ),\n    ),\n]\n\n# Run CAPPr\npred_probs = predict_proba_examples(\n    examples, model_and_tokenizer=(model, tokenizer)\n)\n\n# pred_probs[i][j] = probability that examples[i].prompt is classified as\n# examples[i].completions[j]\nprint([example_pred_probs.round(2) for example_pred_probs in pred_probs])\n# [array([0.7, 0.3]),\n#  array([0.03, 0.97, 0.  ])]\n\n# For each example, which completion is most likely?\npred_class_idxs = [\n    example_pred_probs.argmax() for example_pred_probs in pred_probs\n]\npreds = [\n    example.completions[pred_class_idx]\n    for example, pred_class_idx in zip(examples, pred_class_idxs)\n]\nprint(preds)\n# ['Clarice Starling',\n#  'Kevin Conroy']\n```\n</details>\n\n\nSee the [`demos`](https://github.com/kddubey/cappr/blob/main/demos/) for demonstrations\nof slightly harder classification tasks.\n\nFor CAPPr, GPTQ models are the most computationally performant. These models are\ncompatible with `cappr.huggingface.classify`. See [this page of the\ndocumentation](https://cappr.readthedocs.io/en/latest/select_a_language_model.html#huggingface)\nfor more info on using these models.\n\n\n## Documentation\n\nhttps://cappr.readthedocs.io\n\n\n## Installation\n\nSee [this page of the\ndocumentation](https://cappr.readthedocs.io/en/latest/installation.html).\n\n\n## Motivation\n\nReduce engineering complexity.\n\nSee [this page of the\ndocumentation](https://cappr.readthedocs.io/en/latest/motivation.html) for more info.\n\n\n## Performance\n\n<details>\n<summary>\nStatistical performance\n</summary>\n\nFor open source models, see\n\n- the 4-bit 4 GB Llama 2 [COPA\n  demo](https://github.com/kddubey/cappr/blob/main/demos/llama_cpp/superglue/copa.ipynb)\n- the 4-bit 4 GB Llama 2 [AG News\n  demo](https://github.com/kddubey/cappr/blob/main/demos/llama_cpp/ag_news.ipynb)\n- the 4 GB Mistral [Craigslist Bargains\n  demo](https://github.com/kddubey/cappr/blob/main/demos/huggingface/craigslist_bargains.ipynb)\n- the 4 GB Mistral [Banking 77\n  demo](https://github.com/kddubey/cappr/blob/main/demos/huggingface/banking_77_classes.ipynb),\n  with 77 multi-token choices\n- the 4 GB Mistral [SciQ\n  demo](https://github.com/kddubey/cappr/blob/main/demos/huggingface/sciq.ipynb), with a\n  multiple choice prompt\n- the 4 GB Mistral [Tweet Emotions\n  demo](https://github.com/kddubey/cappr/blob/main/demos/huggingface/tweet_emotion_multilabel.ipynb),\n  which is a multilabel task\u2014each tweet can be tagged with 1-12 emotions.\n\nIn general, you should expect similar or identical performance to text generation when\nevery completion is 1 token long.\n\nSee [this page of the\ndocumentation](https://cappr.readthedocs.io/en/latest/statistical_performance.html)\nfor some discussion.\n\nFor OpenAI models (some deprecated), see\n\n[2 SuperGLUE\ndatasets](https://github.com/kddubey/cappr/blob/main/demos/openai/superglue)\n\n[RAFT zero-shot training\nsets](https://github.com/kddubey/cappr/blob/main/demos/openai/raft)\n</details>\n\n\n<details>\n<summary>\nComputational performance\n</summary>\n\nSee [this page of the\ndocumentation](https://cappr.readthedocs.io/en/latest/computational_performance.html).\n</details>\n\n\n## How it works\n\nYou input a `prompt` string, a `end_of_prompt` string (a whitespace or empty) and a set\nof candidate `completion` strings such that the string\u2014\n\n```python\n{prompt}{end_of_prompt}{completion}\n```\n\n\u2014is a naturally flowing thought. CAPPr picks the `completion` which is mostly likely to\nfollow `prompt` by computing the:\n\n> **C**ompletion<br>\n  **A**fter<br>\n  **P**rompt<br>\n  **Pr**obability<br>\n\nThe method is fleshed out in my [question on Cross\nValidated](https://stats.stackexchange.com/q/601159/337906).\n\n\n## Related work\n\nSee [this page of the\ndocumentation](https://cappr.readthedocs.io/en/latest/related_work.html).\n\n\n## Local development\n\nSee [this page of the documentation](https://cappr.readthedocs.io/en/latest/local.html).\n\n\n## Todo\n\nI'm dumping todos here:\n\n[Code changes](https://github.com/users/kddubey/projects/1/views/1)\n\n[Reseach experiments](https://github.com/users/kddubey/projects/2)\n\nFeel free to raise issues ofc\n",
    "bugtrack_url": null,
    "license": "Apache License 2.0",
    "summary": "Completion After Prompt Probability. Make your LLM make a choice",
    "version": "0.9.0",
    "project_urls": {
        "Homepage": "https://github.com/kddubey/cappr/"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "e04461980dd4325f9e098648e9e31e1d7da2b6183b6f07d0a6ad38f43ae19229",
                "md5": "67142fa8823d8264a65531fb1ffd7823",
                "sha256": "80fa8e18cb9a1fff58cbb241e7efba22f324ee90d006a791a473be81c648df23"
            },
            "downloads": -1,
            "filename": "cappr-0.9.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "67142fa8823d8264a65531fb1ffd7823",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8.0",
            "size": 59989,
            "upload_time": "2024-02-01T21:46:27",
            "upload_time_iso_8601": "2024-02-01T21:46:27.246781Z",
            "url": "https://files.pythonhosted.org/packages/e0/44/61980dd4325f9e098648e9e31e1d7da2b6183b6f07d0a6ad38f43ae19229/cappr-0.9.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "50c04050ef56faffd5e0c70db91da3d771d07bc02f6f8916937f0984e3a5963a",
                "md5": "f41a19339975d688fd13e10a80cc4a2b",
                "sha256": "df9e677f7aad64ffba3af0e2cc751a29d9d6f8a475874826b2608ae85ad465be"
            },
            "downloads": -1,
            "filename": "cappr-0.9.0.tar.gz",
            "has_sig": false,
            "md5_digest": "f41a19339975d688fd13e10a80cc4a2b",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8.0",
            "size": 1640195,
            "upload_time": "2024-02-01T21:46:29",
            "upload_time_iso_8601": "2024-02-01T21:46:29.237294Z",
            "url": "https://files.pythonhosted.org/packages/50/c0/4050ef56faffd5e0c70db91da3d771d07bc02f6f8916937f0984e3a5963a/cappr-0.9.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-02-01 21:46:29",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "kddubey",
    "github_project": "cappr",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "cappr"
}
        
Elapsed time: 0.23475s