# mdGPT - Markdown General Purpose Transformer
Translate markdown files using OpenAI ChatGPT, and generate localized copies of each file.
## Installation
### Using pip
```bash
pip install mdgpt
```
Set environment variable `OPENAI_API_KEY` or create it in a `.env` file
```bash
export OPENAI_API_KEY=YOUR_API_KEY
```
Download example prompts:
```bash
curl -o example.yaml https://raw.githubusercontent.com/Djarnis/mdGPT/main/prompts.yaml
```
Use the example `WEBSITE_BUILDER` option from the prompts to build some example files;
```bash
mdgpt build example
```
Translate these newly created markdown files into Finish (fi) versions:
```bash
mdgpt translate example --target fi
```
or Danish (da):
```bash
mdgpt translate example --target da
```
or German (de):
```bash
mdgpt translate example --target de
```
Or whatever. Just make sure it is an ISO 639-1 two-letter language code, and all should be fine.
Adjust the `example.yaml` prompts to suit your needs.
#### MODEL
You can change the `MODEL` to any engine supported by OpenAI, change the default temperature, and adjust max tokens.
Default values are:
```yaml
MODEL:
temperature: 0.2
engine: gpt-3.5-turbo
max_tokens: 2048
```
#### WEBSITE_BUILDER
This option is used for building mark down documents, given the example instructions below:
```yaml
WEBSITE_BUILDER:
variables:
title: the fictive product "AI Markdown Translator"
tone: sarcastic passive aggressive
year_founded: 2019
major_milestones: 3
system_prompt: |
Only reply in valid markdown with frontmatter.
No explanations. No notes.
Language: {lang[name]}
Markdown Document:
---
# Frontmatter attributes:
title: Title of webpage
description: Short meta description
---
<!-- markdown content -->
# This will be appended as a last line to all step prompts
user_suffix: |
Respond in valid markdown format including all provided frontmatter attributes.
It should be in a {tone} tone.
steps:
- prompt: |
Write the homepage content for {title} in {lang_name} ({lang_code}).
destination: index.md
- prompt: |
Write the "About Us" page content for a fictive team behind {title} in {lang_name} ({lang_code}).
destination: about.md
- prompt: |
Write the history for {title} in {lang_name} ({lang_code}), starting in {year_founded} with {major_milestones} major milestones.
destination: history.md
- prompt: |
Write a contact page for {title} in {lang_name} ({lang_code}). Make the headquarter be at some fancy addres in Silicon Valley.
destination: contact.md
```
#### URL_PROMPT
This prompt is used when translating file paths.
```yaml
URL_PROMPT:
- role: system
prompt: |
Only reply in valid json.
No explanations. No notes.
Language: {lang_name}
JSON document:
{content}
- role: user
prompt: |
Translate all keys in the JSON document.
The keys are in {lang_name} ({lang_code}).
They are url paths for a website in {lang_name}.
I need you to fill out the missing values with a translated {target_lang[name]} ({target_lang_code}) version.
Should contain no special characters (hyphens and slashes are ok, as are dots in file extensions (like .md) if any).
Respond in the same format as the JSON input given, with the `key` as the json key and the (translated) `value` as the value.
Example translating from English to Danish:
"about-us.md": "om-os.md"
```
#### MARKDOWN_PROMPT
This prompt is used when translating markdown files.
```yaml
MARKDOWN_PROMPT:
- role: system
prompt: |
Only reply in valid markdown with frontmatter.
No explanations. No notes.
Language: {lang_name}
Markdown Document:
---
# Frontmatter can be anything or nothing, but always make sure to include the frontmatter dashes.
{frontmatter}
---
{content}
- role: user
prompt: |
Translate all given values from {lang_name} ({lang_code}) to {target_lang[name]} ({target_lang_code}).
Respond in valid markdown format.
```
#### ONLY_INDEXES
Optional boolean value, if you only want `index.md` files translated when translating URL's.
```yaml
ONLY_INDEXES: True
```
#### FIELD_KEYS
Optional list of frontmatter keys you want to translate.
Per default, all keys will be translated, but you can define selected ones here.
```yaml
FIELD_KEYS:
- title
- description
- keywords
- heading
- teaser
```
#### TARGET_LANGUAGES
List of languages you want to target.
```yaml
TARGET_LANGUAGES:
- da
- de
- fr
- es
- it
- zh
```
You can run translations on all target languages by not specifying `--target`.
```bash
mdgpt translate example
```
---
### Using repo source and Poetry:
#### Step 1: Install Poetry
On Unix-based systems like Linux and MacOS, you can install Poetry by using the following command in your terminal:
```bash
curl -sSL https://install.python-poetry.org | python -
```
On Windows, you can use PowerShell to install Poetry with the following command:
```powershell
(Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python -
```
You can check if Poetry was installed correctly by running:
```bash
poetry --version
```
#### Step 2: Rename .env.tpl to .env
In your project directory, you have a file named .env.tpl which serves as a template for environment variables. To use this file, you should rename it to .env.
On Unix-based systems, use the following command:
```bash
mv .env.tpl .env
```
On Windows, use the following command:
```powershell
rename .env.tpl .env
```
#### Step 3: Add OPENAI_API_KEY value to .env
Open your .env file in a text editor. You should see a line that looks something like this:
```bash
OPENAI_API_KEY=
```
After the equal sign, add your OpenAI API key in quotes. It should look something like this:
```bash
OPENAI_API_KEY="your-api-key-goes-here"
```
Save the .env file and close it.
_Please note:_
- Make sure to replace "your-api-key-goes-here" with your actual OpenAI API key.
- Do not share your .env file or post it online, as it contains sensitive information.
#### Step 4: Install mdGPT
From the project directory, build and install mdGPT and its dependencies:
```bash
poetry build
```
```bash
poetry install
```
This installs mdGPT and all its dependencies, and you can now follow the example below.
## Example
### Build Markdown files
The example website ([./example/en](example/en)) was created using the `WEBSITE_BUILDER` option included in the [prompts.yaml](prompts.yaml) file.
```bash
poetry run mdgpt build example
```
Which will create these files in the ./example/en directory:
- index.md
- about.md
- contact.md
- history.md
## Translate website
To translate the markdown files into Finish (fi) versions, run this command:
```bash
poetry run mdgpt translate example --target fi
```
And you should get a `/fi` subdirectory ./example/fi/ containing these files, translated from their original English (en) source:
- index.md
- tietoja.md
- yhteystiedot.md
- historia.md
Raw data
{
"_id": null,
"home_page": "https://github.com/Djarnis/mdGPT",
"name": "mdgpt",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10,<4.0",
"maintainer_email": "",
"keywords": "markdown,translation,openai,chatgpt,gpt",
"author": "Jeppe B\u00e5rris",
"author_email": "jeppe@barris.dk",
"download_url": "https://files.pythonhosted.org/packages/45/b1/bda6db9fc04155ce62387bc809a62d293677a1f4c48e25815411d2e9f5f3/mdgpt-0.3.5.tar.gz",
"platform": null,
"description": "# mdGPT - Markdown General Purpose Transformer\n\nTranslate markdown files using OpenAI ChatGPT, and generate localized copies of each file.\n\n## Installation\n\n### Using pip\n\n```bash\npip install mdgpt\n```\n\nSet environment variable `OPENAI_API_KEY` or create it in a `.env` file\n\n```bash\nexport OPENAI_API_KEY=YOUR_API_KEY\n```\n\nDownload example prompts:\n\n```bash\ncurl -o example.yaml https://raw.githubusercontent.com/Djarnis/mdGPT/main/prompts.yaml\n```\n\nUse the example `WEBSITE_BUILDER` option from the prompts to build some example files;\n\n```bash\nmdgpt build example\n```\n\nTranslate these newly created markdown files into Finish (fi) versions:\n\n```bash\nmdgpt translate example --target fi\n```\n\nor Danish (da):\n\n```bash\nmdgpt translate example --target da\n```\n\nor German (de):\n\n```bash\nmdgpt translate example --target de\n```\n\nOr whatever. Just make sure it is an ISO 639-1 two-letter language code, and all should be fine.\n\nAdjust the `example.yaml` prompts to suit your needs.\n\n#### MODEL\n\nYou can change the `MODEL` to any engine supported by OpenAI, change the default temperature, and adjust max tokens.\n\nDefault values are:\n\n```yaml\nMODEL:\n temperature: 0.2\n engine: gpt-3.5-turbo\n max_tokens: 2048\n```\n\n#### WEBSITE_BUILDER\n\nThis option is used for building mark down documents, given the example instructions below:\n\n```yaml\nWEBSITE_BUILDER:\n variables:\n title: the fictive product \"AI Markdown Translator\"\n tone: sarcastic passive aggressive\n year_founded: 2019\n major_milestones: 3\n system_prompt: |\n Only reply in valid markdown with frontmatter.\n No explanations. No notes.\n Language: {lang[name]}\n Markdown Document:\n ---\n # Frontmatter attributes:\n title: Title of webpage\n description: Short meta description\n ---\n <!-- markdown content -->\n\n # This will be appended as a last line to all step prompts\n user_suffix: |\n Respond in valid markdown format including all provided frontmatter attributes.\n It should be in a {tone} tone.\n\n steps:\n - prompt: |\n Write the homepage content for {title} in {lang_name} ({lang_code}).\n destination: index.md\n - prompt: |\n Write the \"About Us\" page content for a fictive team behind {title} in {lang_name} ({lang_code}).\n destination: about.md\n - prompt: |\n Write the history for {title} in {lang_name} ({lang_code}), starting in {year_founded} with {major_milestones} major milestones.\n destination: history.md\n - prompt: |\n Write a contact page for {title} in {lang_name} ({lang_code}). Make the headquarter be at some fancy addres in Silicon Valley.\n destination: contact.md\n```\n\n#### URL_PROMPT\n\nThis prompt is used when translating file paths.\n\n```yaml\nURL_PROMPT:\n - role: system\n prompt: |\n Only reply in valid json.\n No explanations. No notes.\n Language: {lang_name}\n JSON document:\n {content}\n - role: user\n prompt: |\n Translate all keys in the JSON document.\n The keys are in {lang_name} ({lang_code}).\n They are url paths for a website in {lang_name}.\n I need you to fill out the missing values with a translated {target_lang[name]} ({target_lang_code}) version.\n Should contain no special characters (hyphens and slashes are ok, as are dots in file extensions (like .md) if any).\n Respond in the same format as the JSON input given, with the `key` as the json key and the (translated) `value` as the value.\n Example translating from English to Danish:\n \"about-us.md\": \"om-os.md\"\n```\n\n#### MARKDOWN_PROMPT\n\nThis prompt is used when translating markdown files.\n\n```yaml\nMARKDOWN_PROMPT:\n - role: system\n prompt: |\n Only reply in valid markdown with frontmatter.\n No explanations. No notes.\n Language: {lang_name}\n Markdown Document:\n ---\n # Frontmatter can be anything or nothing, but always make sure to include the frontmatter dashes.\n {frontmatter}\n ---\n {content}\n - role: user\n prompt: |\n Translate all given values from {lang_name} ({lang_code}) to {target_lang[name]} ({target_lang_code}).\n Respond in valid markdown format.\n```\n\n#### ONLY_INDEXES\n\nOptional boolean value, if you only want `index.md` files translated when translating URL's.\n\n```yaml\nONLY_INDEXES: True\n```\n\n#### FIELD_KEYS\n\nOptional list of frontmatter keys you want to translate.\n\nPer default, all keys will be translated, but you can define selected ones here.\n\n```yaml\nFIELD_KEYS:\n - title\n - description\n - keywords\n - heading\n - teaser\n```\n\n#### TARGET_LANGUAGES\n\nList of languages you want to target.\n\n```yaml\nTARGET_LANGUAGES:\n - da\n - de\n - fr\n - es\n - it\n - zh\n```\n\nYou can run translations on all target languages by not specifying `--target`.\n\n```bash\nmdgpt translate example\n```\n\n---\n\n### Using repo source and Poetry:\n\n#### Step 1: Install Poetry\n\nOn Unix-based systems like Linux and MacOS, you can install Poetry by using the following command in your terminal:\n\n```bash\ncurl -sSL https://install.python-poetry.org | python -\n```\n\nOn Windows, you can use PowerShell to install Poetry with the following command:\n\n```powershell\n(Invoke-WebRequest -Uri https://install.python-poetry.org -UseBasicParsing).Content | python -\n```\n\nYou can check if Poetry was installed correctly by running:\n\n```bash\npoetry --version\n```\n\n#### Step 2: Rename .env.tpl to .env\n\nIn your project directory, you have a file named .env.tpl which serves as a template for environment variables. To use this file, you should rename it to .env.\n\nOn Unix-based systems, use the following command:\n\n```bash\nmv .env.tpl .env\n```\n\nOn Windows, use the following command:\n\n```powershell\nrename .env.tpl .env\n```\n\n#### Step 3: Add OPENAI_API_KEY value to .env\n\nOpen your .env file in a text editor. You should see a line that looks something like this:\n\n```bash\nOPENAI_API_KEY=\n```\n\nAfter the equal sign, add your OpenAI API key in quotes. It should look something like this:\n\n```bash\nOPENAI_API_KEY=\"your-api-key-goes-here\"\n```\n\nSave the .env file and close it.\n\n_Please note:_\n\n- Make sure to replace \"your-api-key-goes-here\" with your actual OpenAI API key.\n- Do not share your .env file or post it online, as it contains sensitive information.\n\n#### Step 4: Install mdGPT\n\nFrom the project directory, build and install mdGPT and its dependencies:\n\n```bash\npoetry build\n```\n\n```bash\npoetry install\n```\n\nThis installs mdGPT and all its dependencies, and you can now follow the example below.\n\n## Example\n\n### Build Markdown files\n\nThe example website ([./example/en](example/en)) was created using the `WEBSITE_BUILDER` option included in the [prompts.yaml](prompts.yaml) file.\n\n```bash\npoetry run mdgpt build example\n```\n\nWhich will create these files in the ./example/en directory:\n\n- index.md\n- about.md\n- contact.md\n- history.md\n\n## Translate website\n\nTo translate the markdown files into Finish (fi) versions, run this command:\n\n```bash\npoetry run mdgpt translate example --target fi\n```\n\nAnd you should get a `/fi` subdirectory ./example/fi/ containing these files, translated from their original English (en) source:\n\n- index.md\n- tietoja.md\n- yhteystiedot.md\n- historia.md\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Translate markdown files using OpenAI ChatGPT, and generate localized copies of each file.",
"version": "0.3.5",
"project_urls": {
"Homepage": "https://github.com/Djarnis/mdGPT",
"Repository": "https://github.com/Djarnis/mdGPT"
},
"split_keywords": [
"markdown",
"translation",
"openai",
"chatgpt",
"gpt"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "93cf164f17a42a83445ba64e71d92d1335c37dfb6785ceb53acc49ecc8963256",
"md5": "de7fc139136ccb1385ec20fed35c4cdd",
"sha256": "069f58c3d40c6d0cb41cc8c2ef55d0180e34376328096f2ef42f9eb202a92ac8"
},
"downloads": -1,
"filename": "mdgpt-0.3.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "de7fc139136ccb1385ec20fed35c4cdd",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10,<4.0",
"size": 17504,
"upload_time": "2023-09-03T19:11:31",
"upload_time_iso_8601": "2023-09-03T19:11:31.306780Z",
"url": "https://files.pythonhosted.org/packages/93/cf/164f17a42a83445ba64e71d92d1335c37dfb6785ceb53acc49ecc8963256/mdgpt-0.3.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "45b1bda6db9fc04155ce62387bc809a62d293677a1f4c48e25815411d2e9f5f3",
"md5": "5564a8739aa72e1daaec39271d60d9e8",
"sha256": "5f87e7fa6a6df89761d0778c167a817aa392ff7f82df529f8b8d32891a22d84c"
},
"downloads": -1,
"filename": "mdgpt-0.3.5.tar.gz",
"has_sig": false,
"md5_digest": "5564a8739aa72e1daaec39271d60d9e8",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<4.0",
"size": 18070,
"upload_time": "2023-09-03T19:11:33",
"upload_time_iso_8601": "2023-09-03T19:11:33.063460Z",
"url": "https://files.pythonhosted.org/packages/45/b1/bda6db9fc04155ce62387bc809a62d293677a1f4c48e25815411d2e9f5f3/mdgpt-0.3.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-09-03 19:11:33",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "Djarnis",
"github_project": "mdGPT",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "mdgpt"
}
JSON
