# solar-as-judge
<img width="1422" alt="image" src="https://github.com/hunkim/solar-as-judge/assets/901975/738be829-1d20-4b9a-baad-e3f2c8d066c5">
## How to use
Set the UPSTAGE_API_KEY environment variable. Obtain your key from the Upstage console at <https://console.upstage.ai>.
```bash
pip install solar-as-judge
```
```python
import solar_as_judge as saj
# test prompt with an optional ground truth.
prompt = "Please extract one keyword from this text: I love you so much"
ground_truth = "love"
# The outcome of the A and B language models (AB testing).
A_answer = "love"
B_answer = "so much"
# Check the scores and the winner.
# If they are consistent, then determine the final score.
a_score, b_score = saj.judge(prompt, A_answer, B_answer, ground_truth)
print(a_score, b_score)
```
## Get detailed scores
```python
import solar_as_judge as saj
# test prompt with an optional ground truth.
prompt = "Please extract one keyword from this text: I love you so much"
ground_truth = "love"
# The outcome of the A and B language models (AB testing).
A_answer = "love"
B_answer = "so much"
# Get scores separately.
A_score = saj.get_judge_score(prompt, A_answer, ground_truth)
B_score = saj.get_judge_score(prompt, B_answer, ground_truth)
# Determine the winner.
winner = saj.get_winner(prompt, A_answer, B_answer, ground_truth)
print(A_score, B_score, winner)
```
Raw data
{
"_id": null,
"home_page": "https://github.com/hunkim/solar-as-judge",
"name": "solar-as-judge",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "solar judge coding competition ai",
"author": "Sung Kim",
"author_email": "hunkim@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/8a/ef/50eae28e3f5d764488aaf0c076c6b2ba640f625c13e592a5ebe5ea73262e/solar_as_judge-0.2.0.tar.gz",
"platform": null,
"description": "# solar-as-judge\n<img width=\"1422\" alt=\"image\" src=\"https://github.com/hunkim/solar-as-judge/assets/901975/738be829-1d20-4b9a-baad-e3f2c8d066c5\">\n\n\n## How to use\nSet the UPSTAGE_API_KEY environment variable. Obtain your key from the Upstage console at <https://console.upstage.ai>.\n\n```bash\npip install solar-as-judge\n```\n\n```python\nimport solar_as_judge as saj\n\n# test prompt with an optional ground truth.\nprompt = \"Please extract one keyword from this text: I love you so much\"\nground_truth = \"love\"\n\n# The outcome of the A and B language models (AB testing).\nA_answer = \"love\"\nB_answer = \"so much\"\n\n# Check the scores and the winner. \n# If they are consistent, then determine the final score.\na_score, b_score = saj.judge(prompt, A_answer, B_answer, ground_truth)\nprint(a_score, b_score)\n\n```\n\n## Get detailed scores\n\n```python\nimport solar_as_judge as saj\n\n# test prompt with an optional ground truth.\nprompt = \"Please extract one keyword from this text: I love you so much\"\nground_truth = \"love\"\n\n# The outcome of the A and B language models (AB testing).\nA_answer = \"love\"\nB_answer = \"so much\"\n\n\n# Get scores separately.\nA_score = saj.get_judge_score(prompt, A_answer, ground_truth)\nB_score = saj.get_judge_score(prompt, B_answer, ground_truth)\n\n# Determine the winner.\nwinner = saj.get_winner(prompt, A_answer, B_answer, ground_truth)\nprint(A_score, B_score, winner)\n```\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A tool for using SOLAR as a judge in coding competitions",
"version": "0.2.0",
"project_urls": {
"Homepage": "https://github.com/hunkim/solar-as-judge"
},
"split_keywords": [
"solar",
"judge",
"coding",
"competition",
"ai"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "66a4fb85aefe2e318bf225fe7955cf61231ce23b7044c68aae61844acb4fe41f",
"md5": "ecb3c2313a406a6e037f2651f2fa7bad",
"sha256": "df91f34133b9100fc1f881193bb72c5bfed477a66d0800b6a0b2615669cd1289"
},
"downloads": -1,
"filename": "solar_as_judge-0.2.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ecb3c2313a406a6e037f2651f2fa7bad",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 4351,
"upload_time": "2024-09-21T14:47:04",
"upload_time_iso_8601": "2024-09-21T14:47:04.548349Z",
"url": "https://files.pythonhosted.org/packages/66/a4/fb85aefe2e318bf225fe7955cf61231ce23b7044c68aae61844acb4fe41f/solar_as_judge-0.2.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "8aef50eae28e3f5d764488aaf0c076c6b2ba640f625c13e592a5ebe5ea73262e",
"md5": "f65fe9df0b3578fd6d515307f6426127",
"sha256": "878b7a0d90ce7fd2fdb6b131ba4a71255ed7f27d14564645b5841ca708e763d3"
},
"downloads": -1,
"filename": "solar_as_judge-0.2.0.tar.gz",
"has_sig": false,
"md5_digest": "f65fe9df0b3578fd6d515307f6426127",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 4227,
"upload_time": "2024-09-21T14:47:06",
"upload_time_iso_8601": "2024-09-21T14:47:06.249252Z",
"url": "https://files.pythonhosted.org/packages/8a/ef/50eae28e3f5d764488aaf0c076c6b2ba640f625c13e592a5ebe5ea73262e/solar_as_judge-0.2.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-09-21 14:47:06",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "hunkim",
"github_project": "solar-as-judge",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [],
"lcname": "solar-as-judge"
}
JSON
