# Whisper
Speech recognition with Whisper in MLX. Whisper is a set of open source speech
recognition models from OpenAI, ranging from 39 million to 1.5 billion
parameters.[^1]
### Setup
Install [`ffmpeg`](https://ffmpeg.org/):
```
# on macOS using Homebrew (https://brew.sh/)
brew install ffmpeg
```
Install the `mlx-whisper` package with:
```
pip install mlx-whisper
```
### Run
#### CLI
At its simplest:
```sh
mlx_whisper audio_file.mp3
```
This will make a text file `audio_file.txt` with the results.
Use `-f` to specify the output format and `--model` to specify the model. There
are many other supported command line options. To see them all, run
`mlx_whisper -h`.
You can also pipe the audio content of other programs via stdin:
```sh
some-process | mlx_whisper -
```
The default output file name will be `content.*`. You can specify the name with
the `--output-name` flag.
#### API
Transcribe audio with:
```python
import mlx_whisper
text = mlx_whisper.transcribe(speech_file)["text"]
```
The default model is "mlx-community/whisper-tiny". Choose the model by
setting `path_or_hf_repo`. For example:
```python
result = mlx_whisper.transcribe(speech_file, path_or_hf_repo="models/large")
```
This will load the model contained in `models/large`. The `path_or_hf_repo` can
also point to an MLX-style Whisper model on the Hugging Face Hub. In this case,
the model will be automatically downloaded. A [collection of pre-converted
Whisper
models](https://huggingface.co/collections/mlx-community/whisper-663256f9964fbb1177db93dc)
are in the Hugging Face MLX Community.
The `transcribe` function also supports word-level timestamps. You can generate
these with:
```python
output = mlx_whisper.transcribe(speech_file, word_timestamps=True)
print(output["segments"][0]["words"])
```
To see more transcription options use:
```
>>> help(mlx_whisper.transcribe)
```
### Converting models
> [!TIP]
> Skip the conversion step by using pre-converted checkpoints from the Hugging
> Face Hub. There are a few available in the [MLX
> Community](https://huggingface.co/mlx-community) organization.
To convert a model, first clone the MLX Examples repo:
```
git clone https://github.com/ml-explore/mlx-examples.git
```
Then run `convert.py` from `mlx-examples/whisper`. For example, to convert the
`tiny` model use:
```
python convert.py --torch-name-or-path tiny --mlx-path mlx_models/tiny
```
Note you can also convert a local PyTorch checkpoint which is in the original
OpenAI format.
To generate a 4-bit quantized model, use `-q`. For a full list of options:
```
python convert.py --help
```
By default, the conversion script will make the directory `mlx_models`
and save the converted `weights.npz` and `config.json` there.
Each time it is run, `convert.py` will overwrite any model in the provided
path. To save different models, make sure to set `--mlx-path` to a unique
directory for each converted model. For example:
```bash
model="tiny"
python convert.py --torch-name-or-path ${model} --mlx-path mlx_models/${model}_fp16
python convert.py --torch-name-or-path ${model} --dtype float32 --mlx-path mlx_models/${model}_fp32
python convert.py --torch-name-or-path ${model} -q --q_bits 4 --mlx-path mlx_models/${model}_quantized_4bits
```
[^1]: Refer to the [arXiv paper](https://arxiv.org/abs/2212.04356), [blog post](https://openai.com/research/whisper), and [code](https://github.com/openai/whisper) for more details.
Raw data
{
"_id": null,
"home_page": "https://github.com/ml-explore/mlx-examples",
"name": "mlx-whisper",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": null,
"author": "MLX Contributors",
"author_email": "mlx@group.apple.com",
"download_url": "https://files.pythonhosted.org/packages/4b/be/da654e46741fb07aafbfe44b8e8227f890a6874f17dd068db310d75d491b/mlx_whisper-0.4.2.tar.gz",
"platform": null,
"description": "# Whisper\n\nSpeech recognition with Whisper in MLX. Whisper is a set of open source speech\nrecognition models from OpenAI, ranging from 39 million to 1.5 billion\nparameters.[^1]\n\n### Setup\n\nInstall [`ffmpeg`](https://ffmpeg.org/):\n\n```\n# on macOS using Homebrew (https://brew.sh/)\nbrew install ffmpeg\n```\n\nInstall the `mlx-whisper` package with:\n\n```\npip install mlx-whisper\n```\n\n### Run\n\n#### CLI\n\nAt its simplest:\n\n```sh\nmlx_whisper audio_file.mp3\n```\n\nThis will make a text file `audio_file.txt` with the results.\n\nUse `-f` to specify the output format and `--model` to specify the model. There\nare many other supported command line options. To see them all, run\n`mlx_whisper -h`.\n\nYou can also pipe the audio content of other programs via stdin:\n\n```sh\nsome-process | mlx_whisper -\n```\n\nThe default output file name will be `content.*`. You can specify the name with\nthe `--output-name` flag.\n\n#### API\n\nTranscribe audio with:\n\n```python\nimport mlx_whisper\n\ntext = mlx_whisper.transcribe(speech_file)[\"text\"]\n```\n\nThe default model is \"mlx-community/whisper-tiny\". Choose the model by\nsetting `path_or_hf_repo`. For example:\n\n```python\nresult = mlx_whisper.transcribe(speech_file, path_or_hf_repo=\"models/large\")\n```\n\nThis will load the model contained in `models/large`. The `path_or_hf_repo` can\nalso point to an MLX-style Whisper model on the Hugging Face Hub. In this case,\nthe model will be automatically downloaded. A [collection of pre-converted\nWhisper\nmodels](https://huggingface.co/collections/mlx-community/whisper-663256f9964fbb1177db93dc)\nare in the Hugging Face MLX Community.\n\nThe `transcribe` function also supports word-level timestamps. You can generate\nthese with:\n\n```python\noutput = mlx_whisper.transcribe(speech_file, word_timestamps=True)\nprint(output[\"segments\"][0][\"words\"])\n```\n\nTo see more transcription options use:\n\n```\n>>> help(mlx_whisper.transcribe)\n```\n\n### Converting models\n\n> [!TIP]\n> Skip the conversion step by using pre-converted checkpoints from the Hugging\n> Face Hub. There are a few available in the [MLX\n> Community](https://huggingface.co/mlx-community) organization.\n\nTo convert a model, first clone the MLX Examples repo:\n\n```\ngit clone https://github.com/ml-explore/mlx-examples.git\n```\n\nThen run `convert.py` from `mlx-examples/whisper`. For example, to convert the\n`tiny` model use:\n\n```\npython convert.py --torch-name-or-path tiny --mlx-path mlx_models/tiny\n```\n\nNote you can also convert a local PyTorch checkpoint which is in the original\nOpenAI format.\n\nTo generate a 4-bit quantized model, use `-q`. For a full list of options:\n\n```\npython convert.py --help\n```\n\nBy default, the conversion script will make the directory `mlx_models`\nand save the converted `weights.npz` and `config.json` there.\n\nEach time it is run, `convert.py` will overwrite any model in the provided\npath. To save different models, make sure to set `--mlx-path` to a unique\ndirectory for each converted model. For example:\n\n```bash\nmodel=\"tiny\"\npython convert.py --torch-name-or-path ${model} --mlx-path mlx_models/${model}_fp16\npython convert.py --torch-name-or-path ${model} --dtype float32 --mlx-path mlx_models/${model}_fp32\npython convert.py --torch-name-or-path ${model} -q --q_bits 4 --mlx-path mlx_models/${model}_quantized_4bits\n```\n\n[^1]: Refer to the [arXiv paper](https://arxiv.org/abs/2212.04356), [blog post](https://openai.com/research/whisper), and [code](https://github.com/openai/whisper) for more details.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "OpenAI Whisper on Apple silicon with MLX and the Hugging Face Hub",
"version": "0.4.2",
"project_urls": {
"Homepage": "https://github.com/ml-explore/mlx-examples"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3fb59887e68f5488314a57f7baf2c706b06a97d9020433d1ce774bb6cefc1c22",
"md5": "e39d20c17c979578c9dd4bd1ba2eb41b",
"sha256": "19a343deaa85f461be4fdc9c14e6e8b61d65260374872e07f9f6101ad397a053"
},
"downloads": -1,
"filename": "mlx_whisper-0.4.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e39d20c17c979578c9dd4bd1ba2eb41b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 782695,
"upload_time": "2025-02-10T21:06:07",
"upload_time_iso_8601": "2025-02-10T21:06:07.047403Z",
"url": "https://files.pythonhosted.org/packages/3f/b5/9887e68f5488314a57f7baf2c706b06a97d9020433d1ce774bb6cefc1c22/mlx_whisper-0.4.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4bbeda654e46741fb07aafbfe44b8e8227f890a6874f17dd068db310d75d491b",
"md5": "47377f3124d31b680b7e69cfab50027e",
"sha256": "5487d967245291fd45d5f11c98a69da1130b9304d0767663412d56fffd71e088"
},
"downloads": -1,
"filename": "mlx_whisper-0.4.2.tar.gz",
"has_sig": false,
"md5_digest": "47377f3124d31b680b7e69cfab50027e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 778610,
"upload_time": "2025-02-10T21:06:09",
"upload_time_iso_8601": "2025-02-10T21:06:09.882013Z",
"url": "https://files.pythonhosted.org/packages/4b/be/da654e46741fb07aafbfe44b8e8227f890a6874f17dd068db310d75d491b/mlx_whisper-0.4.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-02-10 21:06:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ml-explore",
"github_project": "mlx-examples",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"circle": true,
"lcname": "mlx-whisper"
}
JSON
