sign-language-datasets


Namesign-language-datasets JSON
Version 0.2.0 PyPI version JSON
download
home_pagehttps://github.com/sign-language-processing/datasets
SummaryTFDS Datasets for sign language
upload_time2023-11-07 09:27:59
maintainer
docs_urlNone
authorAmit Moryossef
requires_python
license
keywords
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls.
            # Sign Language Datasets

This repository includes TFDS data loaders for sign language datasets.

## Installation

#### From Source

```bash
pip install git+https://github.com/sign-language-processing/datasets.git
```

#### PyPi

```bash
pip install sign-language-datasets
```

## Usage

We demonstrate a loading script for every dataset in [examples/load.ipynb](examples/load.ipynb)
[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/sign-language-processing/datasets/blob/master/examples/load.ipynb)

Our config includes the option to choose the resolution and fps, for example:

```python
import tensorflow_datasets as tfds
import sign_language_datasets.datasets
from sign_language_datasets.datasets.config import SignDatasetConfig

# Loading a dataset with default configuration
aslg_pc12 = tfds.load("aslg_pc12")

# Loading a dataset with custom configuration
config = SignDatasetConfig(name="videos_and_poses256x256:12",
                           version="3.0.0",  # Specific version
                           include_video=True,  # Download and load dataset videos
                           process_video=True,  # Process videos to tensors, or only save path to video
                           fps=12,  # Load videos at constant, 12 fps
                           resolution=(256, 256),  # Convert videos to a constant resolution, 256x256
                           include_pose="holistic")  # Download and load Holistic pose estimation
rwth_phoenix2014_t = tfds.load(name='rwth_phoenix2014_t', builder_kwargs=dict(config=config))
```

## Datasets

| Dataset            | Videos                                                       | Poses                                                 | Versions |
|--------------------|--------------------------------------------------------------|-------------------------------------------------------|----------|
| aslg_pc12          | N/A                                                          | N/A                                                   | 0.0.1    |
| asl-lex            | No                                                           |                                                     | 2.0.0    |
| rwth_phoenix2014_t | Yes                                                          | Holistic                                              | 3.0.0    |
| autsl              | Yes                                                          | OpenPose, Holistic                                    | 1.0.0    |
| dgs_corpus         | Yes                                                          | OpenPose, Holistic                                    | 3.0.0    |
| dgs_types          | Yes                                                          |                                                       | 3.0.0    |
| how2sign           | Yes                                                          | OpenPose                                              | 1.0.0    |
| sign2mint          | Yes                                                          |                                                       | 1.0.0    |
| signtyp            | Links                                                        |                                                       | 1.0.0    |
| swojs_glossario    | Yes                                                          |                                                       | 1.0.0    |
| SignBank           | N/A                                                          |                                                       | 1.0.0    |
| wlasl              | [Failed](https://github.com/tensorflow/datasets/issues/2960) | [OpenPose](https://github.com/gulvarol/bsl1k/issues/4) | None     |
| wmtslt             | Yes                                                          | OpenPose, Holistic                                    | 1.2.0    |
| signsuisse         | Yes                                                          |                                                       | 1.0.0    |
| msasl              |                                                              |                                                       | None     |
| Video-Based CSL    |                                                              |                                                       | None     |
| RVL-SLLL ASL       |                                                              |                                                       | None     |
| ngt_corpus         | Yes                                                          |                                                       | 3.0.0    |
| bsl_corpus         | No                                                           | No                                                    | 3.0.0    |

## Data Interface

We follow the following interface wherever possible to make it easy to swap datasets.

```python
{
    "id": tfds.features.Text(),
    "signer": tfds.features.Text() | tf.int32,
    "video": tfds.features.Video(shape=(None, HEIGHT, WIDTH, 3)),
    "depth_video": tfds.features.Video(shape=(None, HEIGHT, WIDTH, 1)),
    "fps": tf.int32,
    "pose": {
        "data": tfds.features.Tensor(shape=(None, 1, POINTS, CHANNELS), dtype=tf.float32),
        "conf": tfds.features.Tensor(shape=(None, 1, POINTS), dtype=tf.float32)
    },
    "gloss": tfds.features.Text(),
    "text": tfds.features.Text()
}
```

## Adding a new dataset

For general instructions, see the
[TFDS guide to writing custom datasets](https://github.com/tensorflow/datasets/blob/master/docs/add_dataset.md).
Instructions below are specific to this repository.

Make a new folder inside `sign_language_datasets/datasets` with the same name as the dataset. As a convention, the name
of the dataset should be lowercase and words should be separated by an underscore. Example:

```sh
cd sign_language_datasets/datasets
tfds new new_dataset
```

For our purposes, creating a custom TFDS dataset means writing a new class which inherits
from `tfds.core.GeneratorBasedBuilder`. If you use `tfds new` to create a new dataset then the dataset class is stored
in a file with the exact same name as the dataset, i.e. `new_dataset.py`. `new_dataset.py` must contain a line similar
to:

```python
class NewDataset(tfds.core.GeneratorBasedBuilder):
```

### Registering a new dataset

The mechanism to add a custom dataset to TFDS' dataset registry is to import the class `NewDataset`. For this reason the
folder
`sign_language_datasets/datasets/new_dataset` must have an `__init__.py` file that imports the class
`NewDataset`:

```python
from .new_dataset import NewDataset
```

Even though the name of the class is `NewDataset`, it will be available for loading in lowercase and uppercase
characters are interpreted as the start of a new word that should be separated with an underscore. This means that the
class can be loaded as follows:

```python
ds = tfds.load('new_dataset')
```

### Generating checksums

The folder for the new dataset should contain a file `checksums.tsv` with checksums for every file in the dataset. This
allows the TFDS download manager to check the integrity of the data it downloads. Use the `tfds build` tool to generate
the checksum file:

```sh
tfds build --register_checksums new_dataset.py
```

Use a dataset configuration which includes all files (e.g. does include the video files if any) using the `--config`
argument. The default behaviour is to build all configurations which might be redundant.

## Why not Huggingface Datasets?

Huggingface datasets do not work well with videos. From the lack of native support of the video type, to lack of support
of arbitrary tensors. Furthermore, they currently have memory leaks that prevent from saving even the smallest of video
datasets.

## Cite

```bibtex
@misc{moryossef2021datasets, 
    title={Sign Language Datasets},
    author={Moryossef, Amit and M\"{u}ller, Mathias},
    howpublished={\url{https://github.com/sign-language-processing/datasets}},
    year={2021}
}
```


            

Raw data

            {
    "_id": null,
    "home_page": "https://github.com/sign-language-processing/datasets",
    "name": "sign-language-datasets",
    "maintainer": "",
    "docs_url": null,
    "requires_python": "",
    "maintainer_email": "",
    "keywords": "",
    "author": "Amit Moryossef",
    "author_email": "amitmoryossef@gmail.com",
    "download_url": "https://files.pythonhosted.org/packages/f9/33/d088a84981f380bc0eaf2222aeec6fb76d8277f88b478eaa90de3c240945/sign-language-datasets-0.2.0.tar.gz",
    "platform": null,
    "description": "# Sign Language Datasets\n\nThis repository includes TFDS data loaders for sign language datasets.\n\n## Installation\n\n#### From Source\n\n```bash\npip install git+https://github.com/sign-language-processing/datasets.git\n```\n\n#### PyPi\n\n```bash\npip install sign-language-datasets\n```\n\n## Usage\n\nWe demonstrate a loading script for every dataset in [examples/load.ipynb](examples/load.ipynb)\n[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/sign-language-processing/datasets/blob/master/examples/load.ipynb)\n\nOur config includes the option to choose the resolution and fps, for example:\n\n```python\nimport tensorflow_datasets as tfds\nimport sign_language_datasets.datasets\nfrom sign_language_datasets.datasets.config import SignDatasetConfig\n\n# Loading a dataset with default configuration\naslg_pc12 = tfds.load(\"aslg_pc12\")\n\n# Loading a dataset with custom configuration\nconfig = SignDatasetConfig(name=\"videos_and_poses256x256:12\",\n                           version=\"3.0.0\",  # Specific version\n                           include_video=True,  # Download and load dataset videos\n                           process_video=True,  # Process videos to tensors, or only save path to video\n                           fps=12,  # Load videos at constant, 12 fps\n                           resolution=(256, 256),  # Convert videos to a constant resolution, 256x256\n                           include_pose=\"holistic\")  # Download and load Holistic pose estimation\nrwth_phoenix2014_t = tfds.load(name='rwth_phoenix2014_t', builder_kwargs=dict(config=config))\n```\n\n## Datasets\n\n| Dataset            | Videos                                                       | Poses                                                 | Versions |\n|--------------------|--------------------------------------------------------------|-------------------------------------------------------|----------|\n| aslg_pc12          | N/A                                                          | N/A                                                   | 0.0.1    |\n| asl-lex            | No                                                           |                                                     | 2.0.0    |\n| rwth_phoenix2014_t | Yes                                                          | Holistic                                              | 3.0.0    |\n| autsl              | Yes                                                          | OpenPose, Holistic                                    | 1.0.0    |\n| dgs_corpus         | Yes                                                          | OpenPose, Holistic                                    | 3.0.0    |\n| dgs_types          | Yes                                                          |                                                       | 3.0.0    |\n| how2sign           | Yes                                                          | OpenPose                                              | 1.0.0    |\n| sign2mint          | Yes                                                          |                                                       | 1.0.0    |\n| signtyp            | Links                                                        |                                                       | 1.0.0    |\n| swojs_glossario    | Yes                                                          |                                                       | 1.0.0    |\n| SignBank           | N/A                                                          |                                                       | 1.0.0    |\n| wlasl              | [Failed](https://github.com/tensorflow/datasets/issues/2960) | [OpenPose](https://github.com/gulvarol/bsl1k/issues/4) | None     |\n| wmtslt             | Yes                                                          | OpenPose, Holistic                                    | 1.2.0    |\n| signsuisse         | Yes                                                          |                                                       | 1.0.0    |\n| msasl              |                                                              |                                                       | None     |\n| Video-Based CSL    |                                                              |                                                       | None     |\n| RVL-SLLL ASL       |                                                              |                                                       | None     |\n| ngt_corpus         | Yes                                                          |                                                       | 3.0.0    |\n| bsl_corpus         | No                                                           | No                                                    | 3.0.0    |\n\n## Data Interface\n\nWe follow the following interface wherever possible to make it easy to swap datasets.\n\n```python\n{\n    \"id\": tfds.features.Text(),\n    \"signer\": tfds.features.Text() | tf.int32,\n    \"video\": tfds.features.Video(shape=(None, HEIGHT, WIDTH, 3)),\n    \"depth_video\": tfds.features.Video(shape=(None, HEIGHT, WIDTH, 1)),\n    \"fps\": tf.int32,\n    \"pose\": {\n        \"data\": tfds.features.Tensor(shape=(None, 1, POINTS, CHANNELS), dtype=tf.float32),\n        \"conf\": tfds.features.Tensor(shape=(None, 1, POINTS), dtype=tf.float32)\n    },\n    \"gloss\": tfds.features.Text(),\n    \"text\": tfds.features.Text()\n}\n```\n\n## Adding a new dataset\n\nFor general instructions, see the\n[TFDS guide to writing custom datasets](https://github.com/tensorflow/datasets/blob/master/docs/add_dataset.md).\nInstructions below are specific to this repository.\n\nMake a new folder inside `sign_language_datasets/datasets` with the same name as the dataset. As a convention, the name\nof the dataset should be lowercase and words should be separated by an underscore. Example:\n\n```sh\ncd sign_language_datasets/datasets\ntfds new new_dataset\n```\n\nFor our purposes, creating a custom TFDS dataset means writing a new class which inherits\nfrom `tfds.core.GeneratorBasedBuilder`. If you use `tfds new` to create a new dataset then the dataset class is stored\nin a file with the exact same name as the dataset, i.e. `new_dataset.py`. `new_dataset.py` must contain a line similar\nto:\n\n```python\nclass NewDataset(tfds.core.GeneratorBasedBuilder):\n```\n\n### Registering a new dataset\n\nThe mechanism to add a custom dataset to TFDS' dataset registry is to import the class `NewDataset`. For this reason the\nfolder\n`sign_language_datasets/datasets/new_dataset` must have an `__init__.py` file that imports the class\n`NewDataset`:\n\n```python\nfrom .new_dataset import NewDataset\n```\n\nEven though the name of the class is `NewDataset`, it will be available for loading in lowercase and uppercase\ncharacters are interpreted as the start of a new word that should be separated with an underscore. This means that the\nclass can be loaded as follows:\n\n```python\nds = tfds.load('new_dataset')\n```\n\n### Generating checksums\n\nThe folder for the new dataset should contain a file `checksums.tsv` with checksums for every file in the dataset. This\nallows the TFDS download manager to check the integrity of the data it downloads. Use the `tfds build` tool to generate\nthe checksum file:\n\n```sh\ntfds build --register_checksums new_dataset.py\n```\n\nUse a dataset configuration which includes all files (e.g. does include the video files if any) using the `--config`\nargument. The default behaviour is to build all configurations which might be redundant.\n\n## Why not Huggingface Datasets?\n\nHuggingface datasets do not work well with videos. From the lack of native support of the video type, to lack of support\nof arbitrary tensors. Furthermore, they currently have memory leaks that prevent from saving even the smallest of video\ndatasets.\n\n## Cite\n\n```bibtex\n@misc{moryossef2021datasets, \n    title={Sign Language Datasets},\n    author={Moryossef, Amit and M\\\"{u}ller, Mathias},\n    howpublished={\\url{https://github.com/sign-language-processing/datasets}},\n    year={2021}\n}\n```\n\n",
    "bugtrack_url": null,
    "license": "",
    "summary": "TFDS Datasets for sign language",
    "version": "0.2.0",
    "project_urls": {
        "Homepage": "https://github.com/sign-language-processing/datasets"
    },
    "split_keywords": [],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "1870f36294a5d962eff58e468c6b67806b75dfedd7f97994d560d8e415b89602",
                "md5": "c1d7b608f53e30331d8d508298453748",
                "sha256": "236c9ff25120f69d43bbfa9dae354601416b8cc53c82574d5936ecdf0a64bad5"
            },
            "downloads": -1,
            "filename": "sign_language_datasets-0.2.0-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "c1d7b608f53e30331d8d508298453748",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": null,
            "size": 3503360,
            "upload_time": "2023-11-07T09:27:57",
            "upload_time_iso_8601": "2023-11-07T09:27:57.164713Z",
            "url": "https://files.pythonhosted.org/packages/18/70/f36294a5d962eff58e468c6b67806b75dfedd7f97994d560d8e415b89602/sign_language_datasets-0.2.0-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "f933d088a84981f380bc0eaf2222aeec6fb76d8277f88b478eaa90de3c240945",
                "md5": "ed6eb321445c5369531938e3d42227d4",
                "sha256": "fd9d11d680675ad3411f2a2672934d6aec6b13dafe08fc40eea6cc4d61fba3a4"
            },
            "downloads": -1,
            "filename": "sign-language-datasets-0.2.0.tar.gz",
            "has_sig": false,
            "md5_digest": "ed6eb321445c5369531938e3d42227d4",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": null,
            "size": 3445146,
            "upload_time": "2023-11-07T09:27:59",
            "upload_time_iso_8601": "2023-11-07T09:27:59.381597Z",
            "url": "https://files.pythonhosted.org/packages/f9/33/d088a84981f380bc0eaf2222aeec6fb76d8277f88b478eaa90de3c240945/sign-language-datasets-0.2.0.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2023-11-07 09:27:59",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "sign-language-processing",
    "github_project": "datasets",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "requirements": [],
    "lcname": "sign-language-datasets"
}
        
Elapsed time: 0.13621s