convert-labse-tf-pt


Nameconvert-labse-tf-pt JSON
Version 3.0.0 PyPI version JSON
download
home_pagehttps://github.com/setu4993/convert-labse-tf-pt
SummaryConvert LaBSE model from TensorFlow to PyTorch.
upload_time2023-06-10 08:10:51
maintainer
docs_urlNone
authorSetu Shah
requires_python>=3.8,<3.11
licenseMIT
keywords transformers bert labse pytorch tensorflow
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # LaBSE

## Project

This project is an implementation to convert Google's [LaBSE](https://tfhub.dev/google/LaBSE/2) model from TensorFlow to PyTorch. It also offers extensions to convert the [smaller-LaBSE model](https://tfhub.dev/jeongukjae/smaller_LaBSE_15lang/1) from TensorFlow to PyTorch, and the [LEALLA family](https://tfhub.dev/google/collections/LEALLA/1) of models.

The models are uploaded to the [HuggingFace Model Hub](https://huggingface.co/setu4993/) in the PyTorch HF-compatible (original and `safetensors`), TensorFlow and Flax formats, alongwith a compatible tokenizer.

- [LaBSE](https://huggingface.co/setu4993/LaBSE)
- [smaller-LaBSE](https://huggingface.co/setu4993/smaller-LaBSE)
- [LEALLA-base](https://huggingface.co/setu4993/LEALLA-base)
- [LEALLA-small](https://huggingface.co/setu4993/LEALLA-small)
- [LEALLA-large](https://huggingface.co/setu4993/LEALLA-large)

## Export

To convert and export the models:

```shell
poetry install
poetry run convert_labse --output_path /path/to/models
```

To update the models on the [HuggingFace Model Hub](https://huggingface.co/setu4993/LaBSE):

```shell
# Clone the already uploaded models.
cd /path/to/model
git clone https://huggingface.co/setu4993/LaBSE.git

# Export models anew and update.
cd /path/to/repo
poetry install
poetry run convert_labse --output_path /path/to/models/LaBSE --huggingface_path
```

### Export Commands by Model

1. [LaBSE](https://huggingface.co/setu4993/LaBSE): `poetry run convert_labse --output_path /path/to/models/setu4993/LaBSE --huggingface_path`
2. [smaller-LaBSE](https://huggingface.co/setu4993/smaller-LaBSE): `poetry run convert_labse --output_path /path/to/models/setu4993/smaller-LaBSE --smaller --huggingface_path`
3. [LEALLA-base](https://huggingface.co/setu4993/LEALLA-base): `poetry run convert_lealla --size base --output_path /path/to/models/setu4993/LEALLA-base --huggingface_path`
4. [LEALLA-small](https://huggingface.co/setu4993/LEALLA-small): `poetry run convert_lealla --size small --output_path /path/to/models/setu4993/LEALLA-small --huggingface_path`
5. [LEALLA-large](https://huggingface.co/setu4993/LEALLA-large): `poetry run convert_lealla --size large --output_path /path/to/models/setu4993/LEALLA-large --huggingface_path`

## Model Cards

See the [`model-cards` directory](https://github.com/setu4993/convert-labse-tf-pt/tree/main/model-cards) for a copy of the model cards.

## License

This repository and the conversion code is licensed under the MIT license, but the **model** is distributed with an Apache-2.0 license.

            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/setu4993/convert-labse-tf-pt",
    "name": "convert-labse-tf-pt",
    "maintainer": "",
    "docs_url": null,
    "requires_python": ">=3.8,<3.11",
    "maintainer_email": "",
    "keywords": "transformers,bert,labse,pytorch,tensorflow",
    "author": "Setu Shah",
    "author_email": "setu+labse@setu.me",
    "download_url": "https://files.pythonhosted.org/packages/97/9f/daed960aa54cfa25bcffe2052cd9d7b34d179e7d3c0edc8c488b989dcbbd/convert_labse_tf_pt-3.0.0.tar.gz",
    "platform": null,
    "description": "# LaBSE\n\n## Project\n\nThis project is an implementation to convert Google's [LaBSE](https://tfhub.dev/google/LaBSE/2) model from TensorFlow to PyTorch. It also offers extensions to convert the [smaller-LaBSE model](https://tfhub.dev/jeongukjae/smaller_LaBSE_15lang/1) from TensorFlow to PyTorch, and the [LEALLA family](https://tfhub.dev/google/collections/LEALLA/1) of models.\n\nThe models are uploaded to the [HuggingFace Model Hub](https://huggingface.co/setu4993/) in the PyTorch HF-compatible (original and `safetensors`), TensorFlow and Flax formats, alongwith a compatible tokenizer.\n\n- [LaBSE](https://huggingface.co/setu4993/LaBSE)\n- [smaller-LaBSE](https://huggingface.co/setu4993/smaller-LaBSE)\n- [LEALLA-base](https://huggingface.co/setu4993/LEALLA-base)\n- [LEALLA-small](https://huggingface.co/setu4993/LEALLA-small)\n- [LEALLA-large](https://huggingface.co/setu4993/LEALLA-large)\n\n## Export\n\nTo convert and export the models:\n\n```shell\npoetry install\npoetry run convert_labse --output_path /path/to/models\n```\n\nTo update the models on the [HuggingFace Model Hub](https://huggingface.co/setu4993/LaBSE):\n\n```shell\n# Clone the already uploaded models.\ncd /path/to/model\ngit clone https://huggingface.co/setu4993/LaBSE.git\n\n# Export models anew and update.\ncd /path/to/repo\npoetry install\npoetry run convert_labse --output_path /path/to/models/LaBSE --huggingface_path\n```\n\n### Export Commands by Model\n\n1. [LaBSE](https://huggingface.co/setu4993/LaBSE): `poetry run convert_labse --output_path /path/to/models/setu4993/LaBSE --huggingface_path`\n2. [smaller-LaBSE](https://huggingface.co/setu4993/smaller-LaBSE): `poetry run convert_labse --output_path /path/to/models/setu4993/smaller-LaBSE --smaller --huggingface_path`\n3. [LEALLA-base](https://huggingface.co/setu4993/LEALLA-base): `poetry run convert_lealla --size base --output_path /path/to/models/setu4993/LEALLA-base --huggingface_path`\n4. [LEALLA-small](https://huggingface.co/setu4993/LEALLA-small): `poetry run convert_lealla --size small --output_path /path/to/models/setu4993/LEALLA-small --huggingface_path`\n5. [LEALLA-large](https://huggingface.co/setu4993/LEALLA-large): `poetry run convert_lealla --size large --output_path /path/to/models/setu4993/LEALLA-large --huggingface_path`\n\n## Model Cards\n\nSee the [`model-cards` directory](https://github.com/setu4993/convert-labse-tf-pt/tree/main/model-cards) for a copy of the model cards.\n\n## License\n\nThis repository and the conversion code is licensed under the MIT license, but the **model** is distributed with an Apache-2.0 license.\n",
    "bugtrack_url": null,
    "license": "MIT",
    "summary": "Convert LaBSE model from TensorFlow to PyTorch.",
    "version": "3.0.0",
    "project_urls": {
        "Homepage": "https://github.com/setu4993/convert-labse-tf-pt",
        "Repository": "https://github.com/setu4993/convert-labse-tf-pt"
    },
    "split_keywords": [
        "transformers",
        "bert",
        "labse",
        "pytorch",
        "tensorflow"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "60e5a747d2d6ed1948df6d81e1beccb957e0f38ad9baafbc12d011fe07ddbf54",
                "md5": "84e9fdac43acaac6fc4fabcde8347332",
                "sha256": "00368c3e5b29cdbc3ecb533680bdee9ae1a3987163c48e7bfad79a9e2c20433d"
            },
            "downloads": -1,
            "filename": "convert_labse_tf_pt-3.0.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "84e9fdac43acaac6fc4fabcde8347332",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8,<3.11",
            "size": 3451851,
            "upload_time": "2023-06-10T08:10:49",
            "upload_time_iso_8601": "2023-06-10T08:10:49.173827Z",
            "url": "https://files.pythonhosted.org/packages/60/e5/a747d2d6ed1948df6d81e1beccb957e0f38ad9baafbc12d011fe07ddbf54/convert_labse_tf_pt-3.0.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "979fdaed960aa54cfa25bcffe2052cd9d7b34d179e7d3c0edc8c488b989dcbbd",
                "md5": "51ae0fc95579ec436aceadb8814a4b79",
                "sha256": "07d5818df35440b03806bf3a79a98a19b1f50e43a52d0d2e17f63dbdbffcc117"
            },
            "downloads": -1,
            "filename": "convert_labse_tf_pt-3.0.0.tar.gz",
            "has_sig": false,
            "md5_digest": "51ae0fc95579ec436aceadb8814a4b79",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8,<3.11",
            "size": 3446448,
            "upload_time": "2023-06-10T08:10:51",
            "upload_time_iso_8601": "2023-06-10T08:10:51.793732Z",
            "url": "https://files.pythonhosted.org/packages/97/9f/daed960aa54cfa25bcffe2052cd9d7b34d179e7d3c0edc8c488b989dcbbd/convert_labse_tf_pt-3.0.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-06-10 08:10:51",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "setu4993",
    "github_project": "convert-labse-tf-pt",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "convert-labse-tf-pt"
}
        
Elapsed time: 0.07558s