# KerasCV
[](https://github.com/keras-team/keras-cv/actions?query=workflow%3ATests+branch%3Amaster)
[](https://github.com/keras-team/keras-cv/issues)
KerasCV is a library of modular computer vision components that work natively
with TensorFlow, JAX, or PyTorch. Built on Keras 3, these models, layers,
metrics, callbacks, etc., can be trained and serialized in any framework and
re-used in another without costly migrations. See "Configuring your backend"
below for more details on multi-framework KerasCV.
<img style="width: 440px; max-width: 90%;" src="https://storage.googleapis.com/keras-cv/guides/keras-cv-augmentations.gif">
KerasCV can be understood as a horizontal extension of the Keras API: the
components are new first-party Keras objects that are too specialized to be
added to core Keras. They receive the same level of polish and backwards
compatibility guarantees as the core Keras API, and they are maintained by the
Keras team.
Our APIs assist in common computer vision tasks such as data augmentation,
classification, object detection, segmentation, image generation, and more.
Applied computer vision engineers can leverage KerasCV to quickly assemble
production-grade, state-of-the-art training and inference pipelines for all of
these common tasks.
## Quick Links
- [List of available models and presets](https://keras.io/api/keras_cv/models/)
- [Developer Guides](https://keras.io/guides/keras_cv/)
- [Contributing Guide](.github/CONTRIBUTING.md)
- [Call for Contributions](https://github.com/keras-team/keras-cv/issues?q=is%3Aopen+is%3Aissue+label%3Acontribution-welcome)
- [API Design Guidelines](.github/API_DESIGN.md)
## Installation
KerasCV supports both Keras 2 and Keras 3. We recommend Keras 3 for all new
users, as it enables using KerasCV models and layers with JAX, TensorFlow and
PyTorch.
### Keras 2 Installation
To install the latest KerasCV release with Keras 2, simply run:
```
pip install --upgrade keras-cv tensorflow
```
### Keras 3 Installation
There are currently two ways to install Keras 3 with KerasCV. To install the
latest changes for KerasCV and Keras, you can use our nightly package.
```
pip install --upgrade keras-cv-nightly tf-nightly
```
To install the stable versions of KerasCV and Keras 3, you should install Keras
3 **after** installing KerasCV. This is a temporary step while TensorFlow is
pinned to Keras 2, and will no longer be necessary after TensorFlow 2.16.
```
pip install --upgrade keras-cv tensorflow
pip install --upgrade keras
```
> [!IMPORTANT]
> Keras 3 will not function with TensorFlow 2.14 or earlier.
## Configuring your backend
If you have Keras 3 installed in your environment (see installation above),
you can use KerasCV with any of JAX, TensorFlow and PyTorch. To do so, set the
`KERAS_BACKEND` environment variable. For example:
so by setting the `KERAS_BACKEND` environment variable. For example:
```shell
export KERAS_BACKEND=jax
```
Or in Colab, with:
```python
import os
os.environ["KERAS_BACKEND"] = "jax"
import keras_cv
```
> [!IMPORTANT]
> Make sure to set the `KERAS_BACKEND` before import any Keras libraries, it
> will be used to set up Keras when it is first imported.
Once that configuration step is done, you can just import KerasCV and start
using it on top of your backend of choice:
```python
import keras_cv
import keras
filepath = keras.utils.get_file(origin="https://i.imgur.com/gCNcJJI.jpg")
image = np.array(keras.utils.load_img(filepath))
image_resized = keras.ops.image.resize(image, (640, 640))[None, ...]
model = keras_cv.models.YOLOV8Detector.from_preset(
"yolo_v8_m_pascalvoc",
bounding_box_format="xywh",
)
predictions = model.predict(image_resized)
```
## Quickstart
```python
import tensorflow as tf
import keras_cv
import tensorflow_datasets as tfds
import keras
# Create a preprocessing pipeline with augmentations
BATCH_SIZE = 16
NUM_CLASSES = 3
augmenter = keras_cv.layers.Augmenter(
[
keras_cv.layers.RandomFlip(),
keras_cv.layers.RandAugment(value_range=(0, 255)),
keras_cv.layers.CutMix(),
],
)
def preprocess_data(images, labels, augment=False):
labels = tf.one_hot(labels, NUM_CLASSES)
inputs = {"images": images, "labels": labels}
outputs = inputs
if augment:
outputs = augmenter(outputs)
return outputs['images'], outputs['labels']
train_dataset, test_dataset = tfds.load(
'rock_paper_scissors',
as_supervised=True,
split=['train', 'test'],
)
train_dataset = train_dataset.batch(BATCH_SIZE).map(
lambda x, y: preprocess_data(x, y, augment=True),
num_parallel_calls=tf.data.AUTOTUNE).prefetch(
tf.data.AUTOTUNE)
test_dataset = test_dataset.batch(BATCH_SIZE).map(
preprocess_data, num_parallel_calls=tf.data.AUTOTUNE).prefetch(
tf.data.AUTOTUNE)
# Create a model using a pretrained backbone
backbone = keras_cv.models.EfficientNetV2Backbone.from_preset(
"efficientnetv2_b0_imagenet"
)
model = keras_cv.models.ImageClassifier(
backbone=backbone,
num_classes=NUM_CLASSES,
activation="softmax",
)
model.compile(
loss='categorical_crossentropy',
optimizer=keras.optimizers.Adam(learning_rate=1e-5),
metrics=['accuracy']
)
# Train your model
model.fit(
train_dataset,
validation_data=test_dataset,
epochs=8,
)
```
## Contributors
If you'd like to contribute, please see our [contributing guide](.github/CONTRIBUTING.md).
To find an issue to tackle, please check our [call for contributions](.github/CALL_FOR_CONTRIBUTIONS.md).
We would like to leverage/outsource the Keras community not only for bug reporting,
but also for active development for feature delivery. To achieve this, here is the predefined
process for how to contribute to this repository:
1) Contributors are always welcome to help us fix an issue, add tests, better documentation.
2) If contributors would like to create a backbone, we usually require a pre-trained weight set
with the model for one dataset as the first PR, and a training script as a follow-up. The training script will preferably help us reproduce the results claimed from paper. The backbone should be generic but the training script can contain paper specific parameters such as learning rate schedules and weight decays. The training script will be used to produce leaderboard results.
Exceptions apply to large transformer-based models which are difficult to train. If this is the case,
contributors should let us know so the team can help in training the model or providing GCP resources.
3) If contributors would like to create a meta arch, please try to be aligned with our roadmap and create a PR for design review to make sure the meta arch is modular.
4) If contributors would like to create a new input formatting which is not in our roadmap for the next 6 months, e.g., keypoint, please create an issue and ask for a sponsor.
5) If contributors would like to support a new task which is not in our roadmap for the next 6 months, e.g., 3D reconstruction, please create an issue and ask for a sponsor.
Thank you to all of our wonderful contributors!
<a href="https://github.com/keras-team/keras-cv/graphs/contributors">
<img src="https://contrib.rocks/image?repo=keras-team/keras-cv" />
</a>
## Pretrained Weights
Many models in KerasCV come with pre-trained weights.
With the exception of StableDiffusion and the standard Vision Transformer, all of these weights are trained using Keras and
KerasCV components and training scripts in this repository.
While some models are not trained with the same parameters or preprocessing pipeline
as defined in their original publications, the KerasCV team ensures strong numerical performance.
Performance metrics for the provided pre-trained weights can be found
in the training history for each documented task.
An example of this can be found in the ImageNet classification training
[history for backbone models](examples/training/classification/imagenet/training_history.json).
All results are reproducible using the training scripts in this repository.
Historically, many models have been trained on image datasets rescaled via manually
crafted normalization schemes.
The most common variant of manually crafted normalization scheme is subtraction of the
imagenet mean pixel followed by standard deviation normalization based on the imagenet
pixel standard deviation.
This scheme is an artifact of the days of manual feature engineering, but is no longer
required to score state of the art scores using modern deep learning architectures.
Due to this, KerasCV is standardized to operate on images that have been rescaled using
a simple `1/255` rescaling layer.
This can be seen in all KerasCV training pipelines and code examples.
## Custom Ops
Note that in some of the 3D Object Detection layers, custom TF ops are used. The
binaries for these ops are not shipped in our PyPi package in order to keep our
wheels pure-Python.
If you'd like to use these custom ops, you can install from source using the
instructions below.
### Installing KerasCV with Custom Ops from Source
Installing custom ops from source requires the [Bazel](https://bazel.build/) build
system (version >= 5.4.0). Steps to install Bazel can be [found here](https://github.com/keras-team/keras/blob/v2.11.0/.devcontainer/Dockerfile#L21-L23).
```
git clone https://github.com/keras-team/keras-cv.git
cd keras-cv
python3 build_deps/configure.py
bazel build build_pip_pkg
export BUILD_WITH_CUSTOM_OPS=true
bazel-bin/build_pip_pkg wheels
pip install wheels/keras_cv-*.whl
```
Note that GitHub actions exist to release KerasCV with custom ops, but are
currently disabled. You can use these [actions](https://github.com/keras-team/keras-cv/blob/master/.github/workflows/release.yml)
in your own fork to create wheels for Linux (manylinux2014), MacOS (both x86 and ARM),
and Windows.
## Disclaimer
KerasCV provides access to pre-trained models via the `keras_cv.models` API.
These pre-trained models are provided on an "as is" basis, without warranties
or conditions of any kind.
The following underlying models are provided by third parties, and are subject to separate
licenses:
StableDiffusion, Vision Transformer
## Citing KerasCV
If KerasCV helps your research, we appreciate your citations.
Here is the BibTeX entry:
```bibtex
@misc{wood2022kerascv,
title={KerasCV},
author={Wood, Luke and Tan, Zhenyu and Stenbit, Ian and Bischof, Jonathan and Zhu, Scott and Chollet, Fran\c{c}ois and Sreepathihalli, Divyashree and Sampath, Ramesh and others},
year={2022},
howpublished={\url{https://github.com/keras-team/keras-cv}},
}
```
Raw data
{
"_id": null,
"home_page": "https://github.com/keras-team/keras-cv",
"name": "keras-cv-nightly",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.9",
"maintainer_email": null,
"keywords": null,
"author": "Keras team",
"author_email": "keras-cv@google.com",
"download_url": "https://files.pythonhosted.org/packages/ff/ae/b497a0558b15d2c20deac0b1ae42bcb621f68dfbd42625e8bd2cf65dcad3/keras_cv_nightly-0.10.0.dev2024100303.tar.gz",
"platform": null,
"description": "# KerasCV\n\n[](https://github.com/keras-team/keras-cv/actions?query=workflow%3ATests+branch%3Amaster)\n\n\n\n[](https://github.com/keras-team/keras-cv/issues)\n\nKerasCV is a library of modular computer vision components that work natively\nwith TensorFlow, JAX, or PyTorch. Built on Keras 3, these models, layers, \nmetrics, callbacks, etc., can be trained and serialized in any framework and \nre-used in another without costly migrations. See \"Configuring your backend\" \nbelow for more details on multi-framework KerasCV.\n\n<img style=\"width: 440px; max-width: 90%;\" src=\"https://storage.googleapis.com/keras-cv/guides/keras-cv-augmentations.gif\">\n\nKerasCV can be understood as a horizontal extension of the Keras API: the\ncomponents are new first-party Keras objects that are too specialized to be\nadded to core Keras. They receive the same level of polish and backwards\ncompatibility guarantees as the core Keras API, and they are maintained by the\nKeras team.\n\nOur APIs assist in common computer vision tasks such as data augmentation,\nclassification, object detection, segmentation, image generation, and more.\nApplied computer vision engineers can leverage KerasCV to quickly assemble\nproduction-grade, state-of-the-art training and inference pipelines for all of\nthese common tasks.\n\n## Quick Links\n- [List of available models and presets](https://keras.io/api/keras_cv/models/)\n- [Developer Guides](https://keras.io/guides/keras_cv/)\n- [Contributing Guide](.github/CONTRIBUTING.md)\n- [Call for Contributions](https://github.com/keras-team/keras-cv/issues?q=is%3Aopen+is%3Aissue+label%3Acontribution-welcome)\n- [API Design Guidelines](.github/API_DESIGN.md)\n\n## Installation\nKerasCV supports both Keras 2 and Keras 3. We recommend Keras 3 for all new \nusers, as it enables using KerasCV models and layers with JAX, TensorFlow and \nPyTorch.\n\n### Keras 2 Installation\n\nTo install the latest KerasCV release with Keras 2, simply run:\n\n```\npip install --upgrade keras-cv tensorflow\n```\n\n### Keras 3 Installation\n\nThere are currently two ways to install Keras 3 with KerasCV. To install the\nlatest changes for KerasCV and Keras, you can use our nightly package.\n\n\n```\npip install --upgrade keras-cv-nightly tf-nightly\n```\n\nTo install the stable versions of KerasCV and Keras 3, you should install Keras\n3 **after** installing KerasCV. This is a temporary step while TensorFlow is\npinned to Keras 2, and will no longer be necessary after TensorFlow 2.16.\n\n```\npip install --upgrade keras-cv tensorflow\npip install --upgrade keras\n```\n> [!IMPORTANT]\n> Keras 3 will not function with TensorFlow 2.14 or earlier.\n\n## Configuring your backend\n\nIf you have Keras 3 installed in your environment (see installation above),\nyou can use KerasCV with any of JAX, TensorFlow and PyTorch. To do so, set the\n`KERAS_BACKEND` environment variable. For example:\nso by setting the `KERAS_BACKEND` environment variable. For example:\n\n```shell\nexport KERAS_BACKEND=jax\n```\n\nOr in Colab, with:\n\n```python\nimport os\nos.environ[\"KERAS_BACKEND\"] = \"jax\"\n\nimport keras_cv\n```\n\n> [!IMPORTANT]\n> Make sure to set the `KERAS_BACKEND` before import any Keras libraries, it\n> will be used to set up Keras when it is first imported.\n\nOnce that configuration step is done, you can just import KerasCV and start\nusing it on top of your backend of choice:\n\n```python\nimport keras_cv\nimport keras\n\nfilepath = keras.utils.get_file(origin=\"https://i.imgur.com/gCNcJJI.jpg\")\nimage = np.array(keras.utils.load_img(filepath))\nimage_resized = keras.ops.image.resize(image, (640, 640))[None, ...]\n\nmodel = keras_cv.models.YOLOV8Detector.from_preset(\n \"yolo_v8_m_pascalvoc\",\n bounding_box_format=\"xywh\",\n)\npredictions = model.predict(image_resized)\n```\n\n## Quickstart\n\n```python\nimport tensorflow as tf\nimport keras_cv\nimport tensorflow_datasets as tfds\nimport keras\n\n# Create a preprocessing pipeline with augmentations\nBATCH_SIZE = 16\nNUM_CLASSES = 3\naugmenter = keras_cv.layers.Augmenter(\n [\n keras_cv.layers.RandomFlip(),\n keras_cv.layers.RandAugment(value_range=(0, 255)),\n keras_cv.layers.CutMix(),\n ],\n)\n\ndef preprocess_data(images, labels, augment=False):\n labels = tf.one_hot(labels, NUM_CLASSES)\n inputs = {\"images\": images, \"labels\": labels}\n outputs = inputs\n if augment:\n outputs = augmenter(outputs)\n return outputs['images'], outputs['labels']\n\ntrain_dataset, test_dataset = tfds.load(\n 'rock_paper_scissors',\n as_supervised=True,\n split=['train', 'test'],\n)\ntrain_dataset = train_dataset.batch(BATCH_SIZE).map(\n lambda x, y: preprocess_data(x, y, augment=True),\n num_parallel_calls=tf.data.AUTOTUNE).prefetch(\n tf.data.AUTOTUNE)\ntest_dataset = test_dataset.batch(BATCH_SIZE).map(\n preprocess_data, num_parallel_calls=tf.data.AUTOTUNE).prefetch(\n tf.data.AUTOTUNE)\n\n# Create a model using a pretrained backbone\nbackbone = keras_cv.models.EfficientNetV2Backbone.from_preset(\n \"efficientnetv2_b0_imagenet\"\n)\nmodel = keras_cv.models.ImageClassifier(\n backbone=backbone,\n num_classes=NUM_CLASSES,\n activation=\"softmax\",\n)\nmodel.compile(\n loss='categorical_crossentropy',\n optimizer=keras.optimizers.Adam(learning_rate=1e-5),\n metrics=['accuracy']\n)\n\n# Train your model\nmodel.fit(\n train_dataset,\n validation_data=test_dataset,\n epochs=8,\n)\n```\n\n## Contributors\nIf you'd like to contribute, please see our [contributing guide](.github/CONTRIBUTING.md).\n\nTo find an issue to tackle, please check our [call for contributions](.github/CALL_FOR_CONTRIBUTIONS.md).\n\nWe would like to leverage/outsource the Keras community not only for bug reporting,\nbut also for active development for feature delivery. To achieve this, here is the predefined\nprocess for how to contribute to this repository:\n\n1) Contributors are always welcome to help us fix an issue, add tests, better documentation.\n2) If contributors would like to create a backbone, we usually require a pre-trained weight set\nwith the model for one dataset as the first PR, and a training script as a follow-up. The training script will preferably help us reproduce the results claimed from paper. The backbone should be generic but the training script can contain paper specific parameters such as learning rate schedules and weight decays. The training script will be used to produce leaderboard results.\nExceptions apply to large transformer-based models which are difficult to train. If this is the case,\ncontributors should let us know so the team can help in training the model or providing GCP resources.\n3) If contributors would like to create a meta arch, please try to be aligned with our roadmap and create a PR for design review to make sure the meta arch is modular.\n4) If contributors would like to create a new input formatting which is not in our roadmap for the next 6 months, e.g., keypoint, please create an issue and ask for a sponsor.\n5) If contributors would like to support a new task which is not in our roadmap for the next 6 months, e.g., 3D reconstruction, please create an issue and ask for a sponsor.\n\nThank you to all of our wonderful contributors!\n\n<a href=\"https://github.com/keras-team/keras-cv/graphs/contributors\">\n <img src=\"https://contrib.rocks/image?repo=keras-team/keras-cv\" />\n</a>\n\n## Pretrained Weights\nMany models in KerasCV come with pre-trained weights.\nWith the exception of StableDiffusion and the standard Vision Transformer, all of these weights are trained using Keras and\nKerasCV components and training scripts in this repository.\nWhile some models are not trained with the same parameters or preprocessing pipeline\nas defined in their original publications, the KerasCV team ensures strong numerical performance.\nPerformance metrics for the provided pre-trained weights can be found\nin the training history for each documented task.\nAn example of this can be found in the ImageNet classification training\n[history for backbone models](examples/training/classification/imagenet/training_history.json).\nAll results are reproducible using the training scripts in this repository.\n\nHistorically, many models have been trained on image datasets rescaled via manually\ncrafted normalization schemes.\nThe most common variant of manually crafted normalization scheme is subtraction of the\nimagenet mean pixel followed by standard deviation normalization based on the imagenet\npixel standard deviation.\nThis scheme is an artifact of the days of manual feature engineering, but is no longer\nrequired to score state of the art scores using modern deep learning architectures.\nDue to this, KerasCV is standardized to operate on images that have been rescaled using\na simple `1/255` rescaling layer.\nThis can be seen in all KerasCV training pipelines and code examples.\n\n## Custom Ops\nNote that in some of the 3D Object Detection layers, custom TF ops are used. The\nbinaries for these ops are not shipped in our PyPi package in order to keep our\nwheels pure-Python.\n\nIf you'd like to use these custom ops, you can install from source using the\ninstructions below.\n\n### Installing KerasCV with Custom Ops from Source\n\nInstalling custom ops from source requires the [Bazel](https://bazel.build/) build\nsystem (version >= 5.4.0). Steps to install Bazel can be [found here](https://github.com/keras-team/keras/blob/v2.11.0/.devcontainer/Dockerfile#L21-L23).\n\n```\ngit clone https://github.com/keras-team/keras-cv.git\ncd keras-cv\n\npython3 build_deps/configure.py\n\nbazel build build_pip_pkg\nexport BUILD_WITH_CUSTOM_OPS=true\nbazel-bin/build_pip_pkg wheels\n\npip install wheels/keras_cv-*.whl\n```\n\nNote that GitHub actions exist to release KerasCV with custom ops, but are\ncurrently disabled. You can use these [actions](https://github.com/keras-team/keras-cv/blob/master/.github/workflows/release.yml)\nin your own fork to create wheels for Linux (manylinux2014), MacOS (both x86 and ARM),\nand Windows.\n\n## Disclaimer\n\nKerasCV provides access to pre-trained models via the `keras_cv.models` API.\nThese pre-trained models are provided on an \"as is\" basis, without warranties\nor conditions of any kind.\nThe following underlying models are provided by third parties, and are subject to separate\nlicenses:\nStableDiffusion, Vision Transformer\n\n## Citing KerasCV\n\nIf KerasCV helps your research, we appreciate your citations.\nHere is the BibTeX entry:\n\n```bibtex\n@misc{wood2022kerascv,\n title={KerasCV},\n author={Wood, Luke and Tan, Zhenyu and Stenbit, Ian and Bischof, Jonathan and Zhu, Scott and Chollet, Fran\\c{c}ois and Sreepathihalli, Divyashree and Sampath, Ramesh and others},\n year={2022},\n howpublished={\\url{https://github.com/keras-team/keras-cv}},\n}\n```\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "Industry-strength computer Vision extensions for Keras.",
"version": "0.10.0.dev2024100303",
"project_urls": {
"Homepage": "https://github.com/keras-team/keras-cv"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "9c19d4fcd686949faef22fb87042ce5b178a87fda47b6a1d0bdf5daf0054f310",
"md5": "53a6c9f25a155bac5dfb4c0ec114dbde",
"sha256": "b05b0404bc8c3687ad8cde0062a37d25ffc9dce96729d30b30cf46952c7c52b4"
},
"downloads": -1,
"filename": "keras_cv_nightly-0.10.0.dev2024100303-py3-none-any.whl",
"has_sig": false,
"md5_digest": "53a6c9f25a155bac5dfb4c0ec114dbde",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.9",
"size": 702603,
"upload_time": "2024-10-03T03:21:27",
"upload_time_iso_8601": "2024-10-03T03:21:27.466562Z",
"url": "https://files.pythonhosted.org/packages/9c/19/d4fcd686949faef22fb87042ce5b178a87fda47b6a1d0bdf5daf0054f310/keras_cv_nightly-0.10.0.dev2024100303-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ffaeb497a0558b15d2c20deac0b1ae42bcb621f68dfbd42625e8bd2cf65dcad3",
"md5": "c216ef16ce23273685c928fedb6dbda4",
"sha256": "cf0a5321433481e16ae9db8b1de9cbfbcb6141a044ab1870cc11dfda8e6af647"
},
"downloads": -1,
"filename": "keras_cv_nightly-0.10.0.dev2024100303.tar.gz",
"has_sig": false,
"md5_digest": "c216ef16ce23273685c928fedb6dbda4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.9",
"size": 415140,
"upload_time": "2024-10-03T03:21:29",
"upload_time_iso_8601": "2024-10-03T03:21:29.433653Z",
"url": "https://files.pythonhosted.org/packages/ff/ae/b497a0558b15d2c20deac0b1ae42bcb621f68dfbd42625e8bd2cf65dcad3/keras_cv_nightly-0.10.0.dev2024100303.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-03 03:21:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "keras-team",
"github_project": "keras-cv",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "keras-cv-nightly"
}
JSON
