# fs-monitor-mqtt
## Overview
The monitor script listens to filesystem events such as `created`, `modified`, `deleted`, `moved`, and publishes them to a MQTT broker.
The topic of the published message is the absolute path of the file or directory that is being monitored. An example payload is a JSON object as follows:
```
{
"timestamp": "2023-10-03T15:42:20.373182",
"event_type": "modified",
"is_directory": false,
"src_path": "/tmp/1"
}
```
## Dependencies
- [watchdog](https://github.com/gorakhargosh/watchdog): Cross-platform file system events monitoring.
- [paho-mqtt](https://github.com/eclipse/paho.mqtt.python): MQTT client library.
- [typer](https://github.com/tiangolo/typer): CLI application framework based on Python type hints.
## PyPI Package
The package is published on [PyPI](https://pypi.org/project/fs-monitor-mqtt/).
## Getting Started
### Prerequisites
- Docker
### Instructions (Run as a Docker Container)
1. Build the Docker image including the application and its dependencies.
```
docker build -t fs-monitor-mqtt .
```
2. Start a MQTT broker.
```
docker run -d -p 1883:1883 --name mosquitto eclipse-mosquitto mosquitto -c /mosquitto-no-auth.conf
```
3. Start a MQTT client that subscribes to all MQTT topics (so that messages can be received and verified easily).
```
docker run --network=host eclipse-mosquitto mosquitto_sub -t '#' -h 'localhost' -p 1883
```
4. In a separate terminal, start `fs-monitor-mqtt` docker container in the background.
The default configuration is to monitor the `/tmp` directory, publish to the MQTT broker running on the host machine with the default port of `1883`.
```
docker run -d --network=host --name fs-monitor-mqtt fs-monitor-mqtt --path /tmp --address localhost --port 1883
```
5. (optional) Follow the output of the `fs-monitor-mqtt` container.
```
docker logs fs-monitor-mqtt -f
```
6. Make a change to the file system that is being monitored. For example, create a new file.
```
docker exec fs-monitor-mqtt touch /tmp/1
```
7. Verify that the MQTT client receives the message. For example,
```
{"timestamp": "2023-10-03T20:03:55.026184", "event_type": "created", "is_directory": false, "src_path": "/tmp/1"}
{"timestamp": "2023-10-03T20:03:55.026759", "event_type": "modified", "is_directory": true, "src_path": "/tmp"}
{"timestamp": "2023-10-03T20:03:55.026860", "event_type": "modified", "is_directory": false, "src_path": "/tmp/1"}
{"timestamp": "2023-10-03T20:03:55.026920", "event_type": "modified", "is_directory": true, "src_path": "/tmp"}
```
8. Stop the `fs-monitor-mqtt` container.
```
docker rm -f fs-monitor-mqtt
```
9. Stop the MQTT broker.
```
docker rm -f mosquitto
```
## Development
### Prerequisites
- Python > 3.9
### Instructions
Run default `make` command which does the following:
- install poetry
- install dependencies
- install pre-commit hooks
- run unit tests (Pytest)
- start the MQTT broker
- run integration tests (Robot)
- finally stopping the MQTT broker.
Check more commands in the `Makefile`.
#### Unit Tests
Run unit tests with `make test`. The unit tests are written using Pytest.
#### Integration Tests
Run integration tests with `make integration_test`. The integration tests are written using Robot Framework.
## Future Work
- [ ] Performance tests.
- [ ] GitHub Actions for CI/CD.
- [ ] Add more unit tests.
- [ ] Add more integration tests.
MIT License
Copyright (c) 2023 Likai R.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Raw data
{
"_id": null,
"home_page": "https://github.com/likair/fs-monitor-mqtt",
"name": "fs-monitor-mqtt",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.9,<4.0",
"maintainer_email": "",
"keywords": "filesystem,mqtt,monitor",
"author": "Likai Ren",
"author_email": "lebsfi@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/0b/27/693c5151bdbb4dcabc6801e206ac63f3a3d7f3eb2128fd23354888c40710/fs_monitor_mqtt-0.3.0.tar.gz",
"platform": null,
"description": "# fs-monitor-mqtt\n\n## Overview\n\nThe monitor script listens to filesystem events such as `created`, `modified`, `deleted`, `moved`, and publishes them to a MQTT broker.\n\nThe topic of the published message is the absolute path of the file or directory that is being monitored. An example payload is a JSON object as follows:\n\n```\n{\n \"timestamp\": \"2023-10-03T15:42:20.373182\",\n \"event_type\": \"modified\",\n \"is_directory\": false,\n \"src_path\": \"/tmp/1\"\n}\n```\n\n## Dependencies\n\n- [watchdog](https://github.com/gorakhargosh/watchdog): Cross-platform file system events monitoring.\n- [paho-mqtt](https://github.com/eclipse/paho.mqtt.python): MQTT client library.\n- [typer](https://github.com/tiangolo/typer): CLI application framework based on Python type hints.\n\n## PyPI Package\n\nThe package is published on [PyPI](https://pypi.org/project/fs-monitor-mqtt/).\n\n## Getting Started\n\n### Prerequisites\n\n- Docker\n\n### Instructions (Run as a Docker Container)\n\n1. Build the Docker image including the application and its dependencies.\n\n ```\n docker build -t fs-monitor-mqtt .\n ```\n\n2. Start a MQTT broker.\n\n ```\n docker run -d -p 1883:1883 --name mosquitto eclipse-mosquitto mosquitto -c /mosquitto-no-auth.conf\n ```\n\n3. Start a MQTT client that subscribes to all MQTT topics (so that messages can be received and verified easily).\n\n ```\n docker run --network=host eclipse-mosquitto mosquitto_sub -t '#' -h 'localhost' -p 1883\n ```\n\n4. In a separate terminal, start `fs-monitor-mqtt` docker container in the background.\n The default configuration is to monitor the `/tmp` directory, publish to the MQTT broker running on the host machine with the default port of `1883`.\n\n ```\n docker run -d --network=host --name fs-monitor-mqtt fs-monitor-mqtt --path /tmp --address localhost --port 1883\n ```\n\n5. (optional) Follow the output of the `fs-monitor-mqtt` container.\n\n ```\n docker logs fs-monitor-mqtt -f\n ```\n\n6. Make a change to the file system that is being monitored. For example, create a new file.\n\n ```\n docker exec fs-monitor-mqtt touch /tmp/1\n ```\n\n7. Verify that the MQTT client receives the message. For example,\n\n ```\n {\"timestamp\": \"2023-10-03T20:03:55.026184\", \"event_type\": \"created\", \"is_directory\": false, \"src_path\": \"/tmp/1\"}\n {\"timestamp\": \"2023-10-03T20:03:55.026759\", \"event_type\": \"modified\", \"is_directory\": true, \"src_path\": \"/tmp\"}\n {\"timestamp\": \"2023-10-03T20:03:55.026860\", \"event_type\": \"modified\", \"is_directory\": false, \"src_path\": \"/tmp/1\"}\n {\"timestamp\": \"2023-10-03T20:03:55.026920\", \"event_type\": \"modified\", \"is_directory\": true, \"src_path\": \"/tmp\"}\n ```\n\n8. Stop the `fs-monitor-mqtt` container.\n\n ```\n docker rm -f fs-monitor-mqtt\n ```\n\n9. Stop the MQTT broker.\n\n ```\n docker rm -f mosquitto\n ```\n\n## Development\n\n### Prerequisites\n\n- Python > 3.9\n\n### Instructions\n\nRun default `make` command which does the following:\n\n - install poetry\n - install dependencies\n - install pre-commit hooks\n - run unit tests (Pytest)\n - start the MQTT broker\n - run integration tests (Robot)\n - finally stopping the MQTT broker.\n\n\n\nCheck more commands in the `Makefile`.\n\n#### Unit Tests\n\nRun unit tests with `make test`. The unit tests are written using Pytest.\n\n#### Integration Tests\n\nRun integration tests with `make integration_test`. The integration tests are written using Robot Framework.\n\n## Future Work\n\n- [ ] Performance tests.\n- [ ] GitHub Actions for CI/CD.\n- [ ] Add more unit tests.\n- [ ] Add more integration tests.\n\nMIT License\n\nCopyright (c) 2023 Likai R.\n\nPermission is hereby granted, free of charge, to any person obtaining a copy\nof this software and associated documentation files (the \"Software\"), to deal\nin the Software without restriction, including without limitation the rights\nto use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is\nfurnished to do so, subject to the following conditions:\n\nThe above copyright notice and this permission notice shall be included in all\ncopies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR\nIMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,\nFITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE\nAUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER\nLIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,\nOUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE\nSOFTWARE.\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A file system monitor that can publish file change events to MQTT broker.",
"version": "0.3.0",
"project_urls": {
"Homepage": "https://github.com/likair/fs-monitor-mqtt",
"Repository": "https://github.com/likair/fs-monitor-mqtt"
},
"split_keywords": [
"filesystem",
"mqtt",
"monitor"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "13b2d28a0826c67b7ed825057eb699eabbcc154a7ac9018e3e56fdf6fe7b1a9d",
"md5": "edca5dcb13607bf86288272ce9f8738e",
"sha256": "51776b5736c0a8b11bce3f444e14c69ed47ee780c8fc477cdf5882303c36263d"
},
"downloads": -1,
"filename": "fs_monitor_mqtt-0.3.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "edca5dcb13607bf86288272ce9f8738e",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9,<4.0",
"size": 7791,
"upload_time": "2023-10-04T07:38:09",
"upload_time_iso_8601": "2023-10-04T07:38:09.780694Z",
"url": "https://files.pythonhosted.org/packages/13/b2/d28a0826c67b7ed825057eb699eabbcc154a7ac9018e3e56fdf6fe7b1a9d/fs_monitor_mqtt-0.3.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0b27693c5151bdbb4dcabc6801e206ac63f3a3d7f3eb2128fd23354888c40710",
"md5": "1095a9cf0da33ecae32f1ff8987f9f24",
"sha256": "db2fba167cd28ee62d96de6d164e89dbb51d8b6cce1cf23fd44f9caf2cbf8b80"
},
"downloads": -1,
"filename": "fs_monitor_mqtt-0.3.0.tar.gz",
"has_sig": false,
"md5_digest": "1095a9cf0da33ecae32f1ff8987f9f24",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9,<4.0",
"size": 6327,
"upload_time": "2023-10-04T07:38:11",
"upload_time_iso_8601": "2023-10-04T07:38:11.419107Z",
"url": "https://files.pythonhosted.org/packages/0b/27/693c5151bdbb4dcabc6801e206ac63f3a3d7f3eb2128fd23354888c40710/fs_monitor_mqtt-0.3.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-10-04 07:38:11",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "likair",
"github_project": "fs-monitor-mqtt",
"github_not_found": true,
"lcname": "fs-monitor-mqtt"
}
JSON
