# Oremi Sound Detector Server
[](https://www.buymeacoffee.com/demsking)
Oremi Sound Detector Server is a WebSocket server designed to detect sound
events, including wake words and a predefined list of songs, for the Oremi
Personal Assistant.
The server listens on port `5023` for incoming connections from clients, which
can continuously stream audio data. Once connected, clients can send audio
data in `bytes`, and the server will process it in real-time, detecting sounds
and sending JSON messages back to the client when a sound is recognized.
Oremi SDS detects the Oremi wake word including sounds: Shout, Bellows,
Children shouting, Laughter, Baby laughter, Crying, sobbing, Baby cry, infant
cry, Whistling, Wheeze, Snoring, Cough, Sneeze, Burping, and Hiccup.
## Table of Contents
- [Install](#install)
- [Usage](#usage)
- [Protocol](#protocol)
* [Initialization](#initialization)
* [Sound Detection](#sound-detection)
* [Connection Closure Codes](#connection-closure-codes)
* [Example Implementation](#example-implementation)
- [Contribute](#contribute)
- [Versioning](#versioning)
- [License](#license)
## Install
```sh
pip install Oremi-SDS
```
## Usage
```sh
usage: oremi-sds [-h] [--host HOST] [-p PORT] -m MODEL [-t THRESHOLD] [-n NUM_THREADS] [-c CONFIG] [--verbose] [-v]
Oremi Sound Detector Server
options:
-h, --help show this help message and exit
-m MODEL, --model MODEL
Path to the TensorFlow Lite model filename (required).
-t THRESHOLD, --threshold THRESHOLD
Detection threshold for filtering predictions (default: 0.1).
-n NUM_THREADS, --num-threads NUM_THREADS
Number of threads for TensorFlow Lite interpreter (default: -1, auto-select).
-c CONFIG, --config CONFIG
Path to the configuration file (default: config.json).
--host HOST Host address to connect to (default: 127.0.0.1).
-p PORT, --port PORT Port number to connect to (default: 5023).
--verbose Enable verbose logging.
-v, --version Show the version of the application.
```
You can download the model Tensorflow Lite model here:
https://storage.googleapis.com/download.tensorflow.org/models/tflite/task_library/audio_classification/rpi/lite-model_yamnet_classification_tflite_1.tflite
## Protocol
Oremi Sound Detector Server operates using a WebSocket-based protocol for
real-time sound detection. The protocol involves an initialization step where
the client provides essential details such as the number of audio channels,
sample rate, block size, language, and features to enable. Once the session is
initialized, the client can continuously stream audio data to the server. The
server processes the audio in real-time and sends JSON messages back to the
client when it detects specific sounds, such as wake words or predefined
songs.
The section outlines the message structures, initialization process, sound
detection mechanism, and possible connection closure codes. Developers can use
this protocol documentation as a reference to interact with the server and
build their applications accordingly.
### Initialization
**1. Client**
When a client connects to the server, it must send an initial
[JSON initiation message](https://gitlab.com/demsking/oremi-sds/blob/main/schemas/InitMessage.json)
within 5 seconds or else the connection will be closed with code `1002` and
reason `Init Timeout`.
```json
{
"type": "init",
"language": "fr",
"samplerate": 16000,
"num_channel": 1,
"blocksize": 4000,
"features": ["wakeword-detector", "sound-detector"]
}
```
**2. Server**
The server responds with:
```json
{
"type": "init",
"server": "Oremi Sound Detector Server/1.0.0"
}
```
**Note:** If the client doesn't send the initialization message within 5
seconds, the server will close the connection with code `1002` and reason
`Init Timeout`.
### Sound Detection
**1. Client**
Once the session is initialized, the client can continuously send audio
stream in bytes.
**2. Server**
The server processes the audio stream in real-time and sends a
[JSON sound message](https://gitlab.com/demsking/oremi-sds/blob/main/schemas/DetectedSoundSchema.json)
when it detects a sound:
**Example for wakeword**
```json
{
"type": "sound",
"sound": "wakeword",
"score": 1.0,
"datetime": "2023-08-02T20:33:22.805154"
}
```
**Example for cough**
```json
{
"type": "sound",
"sound": "cough",
"score": 0.4140625,
"datetime": "2023-08-02T20:41:05.058204"
}
```
### Connection Closure Codes
Possible connection closure codes:
- `1000`: Indicates a normal closure, meaning that the purpose for which the connection was established has been fulfilled.
- `1002`: Init Timeout
- `1003`: Invalid Message
- `4000`: Unexpected Error
### Example Implementation
For an example of how to implement a client for the "Oremi Sound Detector
Server," you can refer to the [client.py file](https://gitlab.com/demsking/oremi-sds/blob/main/client.py)
in the GitLab repository.
The example demonstrates how to connect to the
server, send audio data, and handle the JSON messages received from the
server.
## Contribute
Please follow [CONTRIBUTING.md](https://gitlab.com/demsking/oremi-sds/blob/main/CONTRIBUTING.md).
## Versioning
Given a version number `MAJOR.MINOR.PATCH`, increment the:
- `MAJOR` version when you make incompatible API changes,
- `MINOR` version when you add functionality in a backwards-compatible manner,
and
- `PATCH` version when you make backwards-compatible bug fixes.
Additional labels for pre-release and build metadata are available as extensions
to the `MAJOR.MINOR.PATCH` format.
See [SemVer.org](https://semver.org/) for more details.
## License
Licensed under the Apache License, Version 2.0 (the "License"); you may not use
this file except in compliance with the License.
You may obtain a copy of the License at [LICENSE](https://gitlab.com/demsking/oremi-sds/blob/main/LICENSE).
Raw data
{
"_id": null,
"home_page": "https://gitlab.com/demsking/oremi-sds",
"name": "oremi-sds",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.10,<3.11",
"maintainer_email": "",
"keywords": "sound,detector",
"author": "S\u00e9bastien Demanou",
"author_email": "demsking@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/ee/53/1d9845d3bebf6406c378fb6108d2ba9cb66dbfd35e94f9cf9ca3dfa87590/oremi_sds-1.0.0b1.tar.gz",
"platform": null,
"description": "# Oremi Sound Detector Server\n\n[](https://www.buymeacoffee.com/demsking)\n\nOremi Sound Detector Server is a WebSocket server designed to detect sound\nevents, including wake words and a predefined list of songs, for the Oremi\nPersonal Assistant.\n\nThe server listens on port `5023` for incoming connections from clients, which\ncan continuously stream audio data. Once connected, clients can send audio\ndata in `bytes`, and the server will process it in real-time, detecting sounds\nand sending JSON messages back to the client when a sound is recognized.\n\nOremi SDS detects the Oremi wake word including sounds: Shout, Bellows,\nChildren shouting, Laughter, Baby laughter, Crying, sobbing, Baby cry, infant\ncry, Whistling, Wheeze, Snoring, Cough, Sneeze, Burping, and Hiccup.\n\n## Table of Contents\n\n- [Install](#install)\n- [Usage](#usage)\n- [Protocol](#protocol)\n * [Initialization](#initialization)\n * [Sound Detection](#sound-detection)\n * [Connection Closure Codes](#connection-closure-codes)\n * [Example Implementation](#example-implementation)\n- [Contribute](#contribute)\n- [Versioning](#versioning)\n- [License](#license)\n\n## Install\n\n```sh\npip install Oremi-SDS\n```\n\n## Usage\n\n```sh\nusage: oremi-sds [-h] [--host HOST] [-p PORT] -m MODEL [-t THRESHOLD] [-n NUM_THREADS] [-c CONFIG] [--verbose] [-v]\n\nOremi Sound Detector Server\n\noptions:\n -h, --help show this help message and exit\n -m MODEL, --model MODEL\n Path to the TensorFlow Lite model filename (required).\n -t THRESHOLD, --threshold THRESHOLD\n Detection threshold for filtering predictions (default: 0.1).\n -n NUM_THREADS, --num-threads NUM_THREADS\n Number of threads for TensorFlow Lite interpreter (default: -1, auto-select).\n -c CONFIG, --config CONFIG\n Path to the configuration file (default: config.json).\n --host HOST Host address to connect to (default: 127.0.0.1).\n -p PORT, --port PORT Port number to connect to (default: 5023).\n --verbose Enable verbose logging.\n -v, --version Show the version of the application.\n```\n\nYou can download the model Tensorflow Lite model here:\nhttps://storage.googleapis.com/download.tensorflow.org/models/tflite/task_library/audio_classification/rpi/lite-model_yamnet_classification_tflite_1.tflite\n\n## Protocol\n\nOremi Sound Detector Server operates using a WebSocket-based protocol for\nreal-time sound detection. The protocol involves an initialization step where\nthe client provides essential details such as the number of audio channels,\nsample rate, block size, language, and features to enable. Once the session is\ninitialized, the client can continuously stream audio data to the server. The\nserver processes the audio in real-time and sends JSON messages back to the\nclient when it detects specific sounds, such as wake words or predefined\nsongs.\n\nThe section outlines the message structures, initialization process, sound\ndetection mechanism, and possible connection closure codes. Developers can use\nthis protocol documentation as a reference to interact with the server and\nbuild their applications accordingly.\n\n### Initialization\n\n**1. Client**\n\nWhen a client connects to the server, it must send an initial\n[JSON initiation message](https://gitlab.com/demsking/oremi-sds/blob/main/schemas/InitMessage.json)\nwithin 5 seconds or else the connection will be closed with code `1002` and\nreason `Init Timeout`.\n\n```json\n{\n \"type\": \"init\",\n \"language\": \"fr\",\n \"samplerate\": 16000,\n \"num_channel\": 1,\n \"blocksize\": 4000,\n \"features\": [\"wakeword-detector\", \"sound-detector\"]\n}\n```\n\n**2. Server**\n\nThe server responds with:\n\n```json\n{\n \"type\": \"init\",\n \"server\": \"Oremi Sound Detector Server/1.0.0\"\n}\n```\n\n**Note:** If the client doesn't send the initialization message within 5\nseconds, the server will close the connection with code `1002` and reason\n`Init Timeout`.\n\n### Sound Detection\n\n**1. Client**\n\nOnce the session is initialized, the client can continuously send audio\nstream in bytes.\n\n**2. Server**\n\nThe server processes the audio stream in real-time and sends a\n[JSON sound message](https://gitlab.com/demsking/oremi-sds/blob/main/schemas/DetectedSoundSchema.json)\nwhen it detects a sound:\n\n**Example for wakeword**\n\n```json\n{\n \"type\": \"sound\",\n \"sound\": \"wakeword\",\n \"score\": 1.0,\n \"datetime\": \"2023-08-02T20:33:22.805154\"\n}\n```\n\n**Example for cough**\n\n```json\n{\n \"type\": \"sound\",\n \"sound\": \"cough\",\n \"score\": 0.4140625,\n \"datetime\": \"2023-08-02T20:41:05.058204\"\n}\n```\n\n### Connection Closure Codes\n\nPossible connection closure codes:\n\n- `1000`: Indicates a normal closure, meaning that the purpose for which the connection was established has been fulfilled.\n- `1002`: Init Timeout\n- `1003`: Invalid Message\n- `4000`: Unexpected Error\n\n### Example Implementation\n\nFor an example of how to implement a client for the \"Oremi Sound Detector\nServer,\" you can refer to the [client.py file](https://gitlab.com/demsking/oremi-sds/blob/main/client.py)\nin the GitLab repository.\nThe example demonstrates how to connect to the\nserver, send audio data, and handle the JSON messages received from the\nserver.\n\n## Contribute\n\nPlease follow [CONTRIBUTING.md](https://gitlab.com/demsking/oremi-sds/blob/main/CONTRIBUTING.md).\n\n## Versioning\n\nGiven a version number `MAJOR.MINOR.PATCH`, increment the:\n\n- `MAJOR` version when you make incompatible API changes,\n- `MINOR` version when you add functionality in a backwards-compatible manner,\n and\n- `PATCH` version when you make backwards-compatible bug fixes.\n\nAdditional labels for pre-release and build metadata are available as extensions\nto the `MAJOR.MINOR.PATCH` format.\n\nSee [SemVer.org](https://semver.org/) for more details.\n\n## License\n\nLicensed under the Apache License, Version 2.0 (the \"License\"); you may not use\nthis file except in compliance with the License.\nYou may obtain a copy of the License at [LICENSE](https://gitlab.com/demsking/oremi-sds/blob/main/LICENSE).\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "Oremi Sound Detector Server",
"version": "1.0.0b1",
"project_urls": {
"Documentation": "https://gitlab.com/demsking/oremi-sds/-/blob/main/README.md",
"Homepage": "https://gitlab.com/demsking/oremi-sds",
"Repository": "https://gitlab.com/demsking/oremi-sds",
"Say Thanks!": "https://www.buymeacoffee.com/demsking",
"Source": "https://gitlab.com/demsking/oremi-sds",
"Tracker": "https://gitlab.com/demsking/oremi-sds/-/issues"
},
"split_keywords": [
"sound",
"detector"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3290ae4f5ff835da673e56353bf467c7f86f322c5fa18d5b0513290f01824833",
"md5": "ec1bdce9d7395740b99095ef82c3d78c",
"sha256": "b3443f9326d8de951cedbcd8d4f331795b2eb959dffff5c182d73cc898e7a5ea"
},
"downloads": -1,
"filename": "oremi_sds-1.0.0b1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ec1bdce9d7395740b99095ef82c3d78c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.10,<3.11",
"size": 20037,
"upload_time": "2023-08-03T08:54:09",
"upload_time_iso_8601": "2023-08-03T08:54:09.347718Z",
"url": "https://files.pythonhosted.org/packages/32/90/ae4f5ff835da673e56353bf467c7f86f322c5fa18d5b0513290f01824833/oremi_sds-1.0.0b1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ee531d9845d3bebf6406c378fb6108d2ba9cb66dbfd35e94f9cf9ca3dfa87590",
"md5": "2962829175161f2cbaec048508206360",
"sha256": "fd5b0aee3e979fc57e7743d508e302d6b2b9edcc00158e82c38e53d3f763a1c4"
},
"downloads": -1,
"filename": "oremi_sds-1.0.0b1.tar.gz",
"has_sig": false,
"md5_digest": "2962829175161f2cbaec048508206360",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10,<3.11",
"size": 11047762,
"upload_time": "2023-08-03T08:54:12",
"upload_time_iso_8601": "2023-08-03T08:54:12.286045Z",
"url": "https://files.pythonhosted.org/packages/ee/53/1d9845d3bebf6406c378fb6108d2ba9cb66dbfd35e94f9cf9ca3dfa87590/oremi_sds-1.0.0b1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-08-03 08:54:12",
"github": false,
"gitlab": true,
"bitbucket": false,
"codeberg": false,
"gitlab_user": "demsking",
"gitlab_project": "oremi-sds",
"lcname": "oremi-sds"
}
JSON