# Connected Data Lake
## Usage
```bash
docker run --rm 'quay.io/ulagbulag/connected-data-lake:latest'
```
## Install K8S Operator
```bash
# Register the Connected Data Lake repository
helm repo add cdl "https://smartx-team.github.io/connected-data-lake"
# Deploy a Connected Data Lake operator
helm install -n "cdl-operator" "cdl-operator" "cdl/cdl-operator"
```
### Install Dependencies on Ubuntu 24.04
```bash
# Install os dependencies
sudo apt-get update
sudo apt-get install -y \
build-essential \
fuse \
libfuse-dev \
libprotoc-dev \
protobuf-compiler \
rustup
# Install & Update the latest stable rust
rustup default stable
```
## Build on the local machine
### Build Requirements
- gcc
- fuse
- protobuf
- rust >=1.82
### Build CDL rust CLI
```bash
cargo run --release --
```
### Build CDL python API
Please check your python virtual environment (i.e. conda) before running.
```bash
cd python
maturin develop --release
```
### Build K8S Operator
Please check your kubernetes config file `~/.kube/config` before running.
```bash
cargo run --package 'cdl-k8s-operator' --release --
```
Raw data
{
"_id": null,
"home_page": "https://github.com/ulagbulag/connected-data-lake",
"name": "cdlake",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.10",
"maintainer_email": null,
"keywords": "cdl, cdlake, connecteddatalake, datalake",
"author": "Ho Kim <ho.kim@ulagbulag.io>",
"author_email": "Ho Kim <ho.kim@ulagbulag.io>",
"download_url": "https://files.pythonhosted.org/packages/79/5e/7631ec4fd205886402a5f31c1665c141e4c5c3d5fc450421bfb38c97d1a3/cdlake-0.1.4.tar.gz",
"platform": null,
"description": "# Connected Data Lake\n\n## Usage\n\n```bash\ndocker run --rm 'quay.io/ulagbulag/connected-data-lake:latest'\n```\n\n## Install K8S Operator\n\n```bash\n# Register the Connected Data Lake repository\nhelm repo add cdl \"https://smartx-team.github.io/connected-data-lake\"\n\n# Deploy a Connected Data Lake operator\nhelm install -n \"cdl-operator\" \"cdl-operator\" \"cdl/cdl-operator\"\n```\n\n### Install Dependencies on Ubuntu 24.04\n\n```bash\n# Install os dependencies\nsudo apt-get update\nsudo apt-get install -y \\\n build-essential \\\n fuse \\\n libfuse-dev \\\n libprotoc-dev \\\n protobuf-compiler \\\n rustup\n\n# Install & Update the latest stable rust\nrustup default stable\n```\n\n## Build on the local machine\n\n### Build Requirements\n\n- gcc\n- fuse\n- protobuf\n- rust >=1.82\n\n### Build CDL rust CLI\n\n```bash\ncargo run --release --\n```\n\n### Build CDL python API\n\nPlease check your python virtual environment (i.e. conda) before running.\n\n```bash\ncd python\nmaturin develop --release\n```\n\n### Build K8S Operator\n\nPlease check your kubernetes config file `~/.kube/config` before running.\n\n```bash\ncargo run --package 'cdl-k8s-operator' --release --\n```\n\n",
"bugtrack_url": null,
"license": "AGPL-3.0-or-later",
"summary": "Connected Data Lake Framework",
"version": "0.1.4",
"project_urls": {
"Homepage": "https://github.com/ulagbulag/connected-data-lake",
"documentation": "https://github.com/ulagbulag/connected-data-lake",
"repository": "https://github.com/ulagbulag/connected-data-lake"
},
"split_keywords": [
"cdl",
" cdlake",
" connecteddatalake",
" datalake"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "b3abe1185cb5db7c7eef6321487d46499f290201df6f1fe24115ade6283e6ad1",
"md5": "b8dfe8bd0d3503c39c445c5817117fda",
"sha256": "af89e10be0bd0b3fb3385448cb37f13b155a06164302f86a3996ecdde56c0afb"
},
"downloads": -1,
"filename": "cdlake-0.1.4-cp310-abi3-manylinux_2_39_x86_64.whl",
"has_sig": false,
"md5_digest": "b8dfe8bd0d3503c39c445c5817117fda",
"packagetype": "bdist_wheel",
"python_version": "cp310",
"requires_python": ">=3.10",
"size": 34233455,
"upload_time": "2024-11-23T11:03:40",
"upload_time_iso_8601": "2024-11-23T11:03:40.039360Z",
"url": "https://files.pythonhosted.org/packages/b3/ab/e1185cb5db7c7eef6321487d46499f290201df6f1fe24115ade6283e6ad1/cdlake-0.1.4-cp310-abi3-manylinux_2_39_x86_64.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "795e7631ec4fd205886402a5f31c1665c141e4c5c3d5fc450421bfb38c97d1a3",
"md5": "fe5345938a5fb57b9d10666c9f12fef3",
"sha256": "01e08b84d7c7df531931254e4e89d965b0da18b7baf7f364d3783dd02e417e8f"
},
"downloads": -1,
"filename": "cdlake-0.1.4.tar.gz",
"has_sig": false,
"md5_digest": "fe5345938a5fb57b9d10666c9f12fef3",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.10",
"size": 76342,
"upload_time": "2024-11-23T11:03:44",
"upload_time_iso_8601": "2024-11-23T11:03:44.958151Z",
"url": "https://files.pythonhosted.org/packages/79/5e/7631ec4fd205886402a5f31c1665c141e4c5c3d5fc450421bfb38c97d1a3/cdlake-0.1.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-23 11:03:44",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "ulagbulag",
"github_project": "connected-data-lake",
"github_not_found": true,
"lcname": "cdlake"
}
JSON
