# NetFL
**NetFL** is a framework that extends [Fogbed](https://github.com/larsid/fogbed) by integrating [Flower](https://github.com/adap/flower), enabling simulation of Federated Learning experiments within Fog/Edge computing environments. It supports the modeling of heterogeneous and resource-constrained edge scenarios, incorporating factors such as computational disparities among clients and dynamic network conditions, including bandwidth limitations, latency variations, and packet loss. This facilitates realistic evaluations of FL systems under non-ideal, real-world conditions.
## Installation
> **Requirements**: Ubuntu 22.04 LTS or later, Python 3.9.
### 1. Set up Containernet
Refer to the [Containernet documentation](https://github.com/containernet/containernet) for further details.
Install Ansible:
```
sudo apt-get install ansible
```
Clone the Containernet repository:
```
git clone https://github.com/containernet/containernet.git
```
Run the installation playbook:
```
sudo ansible-playbook -i "localhost," -c local containernet/ansible/install.yml
```
Create and activate a virtual environment:
```
python3 -m venv venv
source venv/bin/activate
```
> **Note:** The virtual environment **must be activated** before installing or using any Python packages, including Containernet and NetFL.
Install Containernet into the active virtual environment:
```
pip install containernet/.
```
### 2. Install NetFL
While the virtual environment is still active, run:
```
pip install netfl
```
## Running an Experiment with NetFL and Fogbed
Follow the steps below to set up and run an experiment using **NetFL**. This is an example using the **MNIST** dataset. You can find more examples in the `examples` folder:
### 1. Define the Dataset, the Model, and the Training Configurations
```py
from keras import models, optimizers
from flwr.server.strategy import FedAvg
from netfl.core.task import Task, Dataset, DatasetInfo, DatasetPartitioner, TrainConfigs
from netfl.core.models import cnn3
from netfl.core.partitioner import IidPartitioner
class MNIST(Task):
def dataset_info(self) -> DatasetInfo:
return DatasetInfo(
huggingface_path="ylecun/mnist",
item_name="image",
label_name="label"
)
def dataset_partitioner(self) -> DatasetPartitioner:
return IidPartitioner()
def normalized_dataset(self, raw_dataset: Dataset) -> Dataset:
return Dataset(
x=(raw_dataset.x / 255.0),
y=raw_dataset.y
)
def model(self) -> models.Model:
return cnn3(
input_shape=(28, 28, 1),
output_classes=10,
optimizer=optimizers.SGD(learning_rate=0.01)
)
def aggregation_strategy(self) -> type[FedAvg]:
return FedAvg
def train_configs(self) -> TrainConfigs:
return TrainConfigs(
batch_size=16,
epochs=2,
num_clients=4,
num_partitions=4,
num_rounds=10,
seed_data=42,
shuffle_data=True
)
class MainTask(MNIST):
pass
```
### 2. Start Fogbed Workers and Define the Experiment Network Topology
Refer to the [Fogbed documentation](https://larsid.github.io/fogbed/distributed_emulation) for detailed instructions on starting workers.
### 3. Create and Run the Experiment
```py
from fogbed import HardwareResources, CloudResourceModel, EdgeResourceModel
from netfl.core.experiment import NetflExperiment
from netfl.utils.resources import LinkResources
from task import MainTask
exp = NetflExperiment(name="mnist-exp", task=MainTask(), max_cu=2.0, max_mu=3072)
cloud_resources = CloudResourceModel(max_cu=1.0, max_mu=1024)
edge_0_resources = EdgeResourceModel(max_cu=0.5, max_mu=1024)
edge_1_resources = EdgeResourceModel(max_cu=0.5, max_mu=1024)
server_resources = HardwareResources(cu=1.0, mu=1024)
server_link = LinkResources(bw=1000)
edge_0_total_devices = 2
edge_0_device_resources = HardwareResources(cu=0.25, mu=512)
edge_0_device_link = LinkResources(bw=100)
total_edge_1_devices = 2
device_edge_1_resources = HardwareResources(cu=0.25, mu=512)
edge_1_device_link = LinkResources(bw=50)
cloud_edge_0_link = LinkResources(bw=10)
cloud_edge_1_link = LinkResources(bw=5)
cloud = exp.add_virtual_instance("cloud", cloud_resources)
edge_0 = exp.add_virtual_instance("edge_0", edge_0_resources)
edge_1 = exp.add_virtual_instance("edge_1", edge_1_resources)
server = exp.create_server("server", server_resources, server_link)
edge_0_devices = exp.create_devices(
"edge_0_device", edge_0_device_resources, edge_0_device_link, edge_0_total_devices
)
edge_1_devices = exp.create_devices(
"edge_1_device", device_edge_1_resources, edge_1_device_link, total_edge_1_devices
)
exp.add_docker(server, cloud)
for device in edge_0_devices: exp.add_docker(device, edge_0)
for device in edge_1_devices: exp.add_docker(device, edge_1)
worker = exp.add_worker("127.0.0.1", port=5000)
worker.add(cloud)
worker.add(edge_0)
worker.add(edge_1)
worker.add_link(cloud, edge_0, **cloud_edge_0_link.params)
worker.add_link(cloud, edge_1, **cloud_edge_1_link.params)
try:
exp.start()
input("Press enter to finish")
except Exception as ex:
print(ex)
finally:
exp.stop()
```
## Running a Simple Example with a Basic Network Topology Using Docker
### 1. Create the Task
In the project root directory, create or modify a **NetFL Task** and name the file `task.py`. Refer to the examples in the `examples` folder for guidance on task creation.
### 2. Create the Infrastructure
Use Docker Compose to set up the infrastructure, including the server and clients:
```
docker compose up -d
```
### 3. View Training Results
To check the server logs, run:
```
docker logs server
```
Training logs are also stored in the logs folder within the project root directory.
### 4. Shut Down the Infrastructure
To stop and remove all running containers, use the following command:
```
docker compose down
```
## More information
- [NetFL on PyPI](https://pypi.org/project/netfl)
- [NetFL Docker Images](https://hub.docker.com/r/netfl/netfl/tags)
Raw data
{
"_id": null,
"home_page": null,
"name": "netfl",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": "edge-computing, federated-learning, fog-computing, iot, machine-learning",
"author": null,
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/34/c4/374bd291ba23d3716f61aeffb6ce7e386c1dbeb92a98e0ff46c1062939f0/netfl-1.0.0.tar.gz",
"platform": null,
"description": "# NetFL\n\n**NetFL** is a framework that extends [Fogbed](https://github.com/larsid/fogbed) by integrating [Flower](https://github.com/adap/flower), enabling simulation of Federated Learning experiments within Fog/Edge computing environments. It supports the modeling of heterogeneous and resource-constrained edge scenarios, incorporating factors such as computational disparities among clients and dynamic network conditions, including bandwidth limitations, latency variations, and packet loss. This facilitates realistic evaluations of FL systems under non-ideal, real-world conditions.\n\n## Installation\n\n> **Requirements**: Ubuntu 22.04 LTS or later, Python 3.9.\n\n### 1. Set up Containernet\n\nRefer to the [Containernet documentation](https://github.com/containernet/containernet) for further details.\n\nInstall Ansible:\n\n```\nsudo apt-get install ansible\n```\n\nClone the Containernet repository:\n\n```\ngit clone https://github.com/containernet/containernet.git\n```\n\nRun the installation playbook:\n\n```\nsudo ansible-playbook -i \"localhost,\" -c local containernet/ansible/install.yml\n```\n\nCreate and activate a virtual environment:\n\n```\npython3 -m venv venv\nsource venv/bin/activate\n```\n\n> **Note:** The virtual environment **must be activated** before installing or using any Python packages, including Containernet and NetFL.\n\nInstall Containernet into the active virtual environment:\n\n```\npip install containernet/.\n```\n\n### 2. Install NetFL\n\nWhile the virtual environment is still active, run:\n\n```\npip install netfl\n```\n\n## Running an Experiment with NetFL and Fogbed\n\nFollow the steps below to set up and run an experiment using **NetFL**. This is an example using the **MNIST** dataset. You can find more examples in the `examples` folder:\n\n### 1. Define the Dataset, the Model, and the Training Configurations\n\n```py\nfrom keras import models, optimizers\nfrom flwr.server.strategy import FedAvg\n\nfrom netfl.core.task import Task, Dataset, DatasetInfo, DatasetPartitioner, TrainConfigs\nfrom netfl.core.models import cnn3\nfrom netfl.core.partitioner import IidPartitioner\n\n\nclass MNIST(Task):\n def dataset_info(self) -> DatasetInfo:\n return DatasetInfo(\n huggingface_path=\"ylecun/mnist\",\n item_name=\"image\",\n label_name=\"label\"\n )\n \n def dataset_partitioner(self) -> DatasetPartitioner:\n return IidPartitioner()\n\n def normalized_dataset(self, raw_dataset: Dataset) -> Dataset:\n return Dataset(\n x=(raw_dataset.x / 255.0),\n y=raw_dataset.y\n )\n\n def model(self) -> models.Model: \n return cnn3(\n input_shape=(28, 28, 1), \n output_classes=10,\n optimizer=optimizers.SGD(learning_rate=0.01)\n )\n\n def aggregation_strategy(self) -> type[FedAvg]:\n return FedAvg\n \n def train_configs(self) -> TrainConfigs:\n return TrainConfigs(\n batch_size=16,\n epochs=2,\n num_clients=4,\n num_partitions=4,\n num_rounds=10,\n seed_data=42,\n shuffle_data=True\n )\n\n\nclass MainTask(MNIST):\n pass\n\n```\n\n### 2. Start Fogbed Workers and Define the Experiment Network Topology\n\nRefer to the [Fogbed documentation](https://larsid.github.io/fogbed/distributed_emulation) for detailed instructions on starting workers.\n\n\n\n### 3. Create and Run the Experiment\n\n```py\nfrom fogbed import HardwareResources, CloudResourceModel, EdgeResourceModel\nfrom netfl.core.experiment import NetflExperiment\nfrom netfl.utils.resources import LinkResources\nfrom task import MainTask\n\n\nexp = NetflExperiment(name=\"mnist-exp\", task=MainTask(), max_cu=2.0, max_mu=3072)\n\ncloud_resources = CloudResourceModel(max_cu=1.0, max_mu=1024)\nedge_0_resources = EdgeResourceModel(max_cu=0.5, max_mu=1024)\nedge_1_resources = EdgeResourceModel(max_cu=0.5, max_mu=1024)\n\nserver_resources = HardwareResources(cu=1.0, mu=1024)\nserver_link = LinkResources(bw=1000)\n\nedge_0_total_devices = 2\nedge_0_device_resources = HardwareResources(cu=0.25, mu=512)\nedge_0_device_link = LinkResources(bw=100)\n\ntotal_edge_1_devices = 2\ndevice_edge_1_resources = HardwareResources(cu=0.25, mu=512)\nedge_1_device_link = LinkResources(bw=50)\n\ncloud_edge_0_link = LinkResources(bw=10)\ncloud_edge_1_link = LinkResources(bw=5)\n\ncloud = exp.add_virtual_instance(\"cloud\", cloud_resources)\nedge_0 = exp.add_virtual_instance(\"edge_0\", edge_0_resources)\nedge_1 = exp.add_virtual_instance(\"edge_1\", edge_1_resources)\n\nserver = exp.create_server(\"server\", server_resources, server_link)\n\nedge_0_devices = exp.create_devices(\n \"edge_0_device\", edge_0_device_resources, edge_0_device_link, edge_0_total_devices\n)\n\nedge_1_devices = exp.create_devices(\n \"edge_1_device\", device_edge_1_resources, edge_1_device_link, total_edge_1_devices\n)\n\nexp.add_docker(server, cloud)\nfor device in edge_0_devices: exp.add_docker(device, edge_0)\nfor device in edge_1_devices: exp.add_docker(device, edge_1)\n\nworker = exp.add_worker(\"127.0.0.1\", port=5000)\n\nworker.add(cloud)\nworker.add(edge_0)\nworker.add(edge_1)\n\nworker.add_link(cloud, edge_0, **cloud_edge_0_link.params)\nworker.add_link(cloud, edge_1, **cloud_edge_1_link.params)\n\ntry:\n exp.start()\n input(\"Press enter to finish\")\nexcept Exception as ex: \n print(ex)\nfinally:\n exp.stop()\n\n```\n\n## Running a Simple Example with a Basic Network Topology Using Docker\n\n### 1. Create the Task\n\nIn the project root directory, create or modify a **NetFL Task** and name the file `task.py`. Refer to the examples in the `examples` folder for guidance on task creation.\n\n### 2. Create the Infrastructure\n\nUse Docker Compose to set up the infrastructure, including the server and clients:\n\n```\ndocker compose up -d\n```\n\n### 3. View Training Results\n\nTo check the server logs, run:\n\n```\ndocker logs server\n```\n\nTraining logs are also stored in the logs folder within the project root directory. \n\n### 4. Shut Down the Infrastructure\n\nTo stop and remove all running containers, use the following command:\n\n```\ndocker compose down\n```\n\n## More information\n\n- [NetFL on PyPI](https://pypi.org/project/netfl)\n\n- [NetFL Docker Images](https://hub.docker.com/r/netfl/netfl/tags)\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "NetFL enables simulation of Federated Learning experiments within Fog/Edge computing environments.",
"version": "1.0.0",
"project_urls": {
"Homepage": "https://github.com/larsid/netfl"
},
"split_keywords": [
"edge-computing",
" federated-learning",
" fog-computing",
" iot",
" machine-learning"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "6b715d560b627eeb3654c9efaf9af1b127d7ce7cb36e001c09e10d9454674d41",
"md5": "08369d490f6aec379f92d06e54c68d3c",
"sha256": "c68dd37dd00ecc1c00d6c61e875003e1f0231b970e6cf1edb1f2d618150584d9"
},
"downloads": -1,
"filename": "netfl-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "08369d490f6aec379f92d06e54c68d3c",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 17375,
"upload_time": "2025-07-23T09:24:51",
"upload_time_iso_8601": "2025-07-23T09:24:51.027087Z",
"url": "https://files.pythonhosted.org/packages/6b/71/5d560b627eeb3654c9efaf9af1b127d7ce7cb36e001c09e10d9454674d41/netfl-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "34c4374bd291ba23d3716f61aeffb6ce7e386c1dbeb92a98e0ff46c1062939f0",
"md5": "7eb0977396476f9ec15d326fc07d6abb",
"sha256": "fc48b0ee3290f80ecb99edc95342a14c2972692ac871b0b0619f0d6e316c7b87"
},
"downloads": -1,
"filename": "netfl-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "7eb0977396476f9ec15d326fc07d6abb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 151106,
"upload_time": "2025-07-23T09:24:52",
"upload_time_iso_8601": "2025-07-23T09:24:52.516611Z",
"url": "https://files.pythonhosted.org/packages/34/c4/374bd291ba23d3716f61aeffb6ce7e386c1dbeb92a98e0ff46c1062939f0/netfl-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-07-23 09:24:52",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "larsid",
"github_project": "netfl",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "netfl"
}
JSON
