# AstraPy
A pythonic client for [DataStax Astra DB](https://astra.datastax.com).
_This README targets AstraPy version **1.0.0+**, which introduces a whole new API.
Click [here](https://github.com/datastax/astrapy/blob/cd3f5ce8146093e10a095709c0f5c3f8e3f2c7da/README.md) for the pre-existing API (fully compatible with newer versions)._
## Quickstart
Install with `pip install astrapy`.
Get the *API Endpoint* and the *Token* to your Astra DB instance at [astra.datastax.com](https://astra.datastax.com).
Try the following code after replacing the connection parameters:
```python
import astrapy
ASTRA_DB_APPLICATION_TOKEN = "AstraCS:..."
ASTRA_DB_API_ENDPOINT = "https://01234567-....apps.astra.datastax.com"
my_client = astrapy.DataAPIClient()
my_database = my_client.get_database(
ASTRA_DB_API_ENDPOINT,
token=ASTRA_DB_APPLICATION_TOKEN,
)
my_collection = my_database.create_collection(
"dreams",
dimension=3,
metric=astrapy.constants.VectorMetric.COSINE,
)
my_collection.insert_one({"summary": "I was flying", "$vector": [-0.4, 0.7, 0]})
my_collection.insert_many(
[
{
"_id": astrapy.ids.UUID("018e65c9-e33d-749b-9386-e848739582f0"),
"summary": "A dinner on the Moon",
"$vector": [0.2, -0.3, -0.5],
},
{
"summary": "Riding the waves",
"tags": ["sport"],
"$vector": [0, 0.2, 1],
},
{
"summary": "Friendly aliens in town",
"tags": ["scifi"],
"$vector": [-0.3, 0, 0.8],
},
{
"summary": "Meeting Beethoven at the dentist",
"$vector": [0.2, 0.6, 0],
},
],
)
my_collection.update_one(
{"tags": "sport"},
{"$set": {"summary": "Surfers' paradise"}},
)
cursor = my_collection.find(
{},
sort={"$vector": [0, 0.2, 0.4]},
limit=2,
include_similarity=True,
)
for result in cursor:
print(f"{result['summary']}: {result['$similarity']}")
# This would print:
# Surfers' paradise: 0.98238194
# Friendly aliens in town: 0.91873914
```
Next steps:
- More info and usage patterns are given in the docstrings of classes and methods
- [Data API reference](https://docs.datastax.com/en/astra-db-serverless/api-reference/overview.html)
- [AstraPy reference](https://docs.datastax.com/en/astra-api-docs/_attachments/python-client/astrapy/index.html)
- Package on [PyPI](https://pypi.org/project/astrapy/)
### Usage with HCD and other non-Astra installations
The main difference to target e.g. a Hyper-Converged Database (HCD)
installation is how the client is
initialized. Here is a short example showing just how to get to a `Database`
(what comes next is unchaged compared to using Astra DB).
```python
from astrapy import DataAPIClient
from astrapy.constants import Environment
from astrapy.authentication import UsernamePasswordTokenProvider
# Build a token
tp = UsernamePasswordTokenProvider("username", "password")
# Initialize the client and get a "Database" object
client = DataAPIClient(token=tp, environment=Environment.HCD)
database = client.get_database("http://localhost:8181", token=tp)
```
For more on this case, please consult the [dedicated reference](https://docs.datastax.com/en/hyper-converged-database/1.0/connect/python-client.html).
## AstraPy's API
### Abstraction diagram
AstraPy's abstractions for working at the data and admin layers are structured
as depicted by this diagram:
Here's a small admin-oriented example:
```python
import astrapy
# this must have "Database Administrator" permissions:
ASTRA_DB_APPLICATION_TOKEN = "AstraCS:..."
my_client = astrapy.DataAPIClient(ASTRA_DB_APPLICATION_TOKEN)
my_astra_admin = my_client.get_admin()
database_list = list(my_astra_admin.list_databases())
db_info = database_list[0].info
print(db_info.name, db_info.id, db_info.region)
my_database_admin = my_astra_admin.get_database_admin(db_info.id)
my_database_admin.list_keyspaces()
my_database_admin.create_keyspace("my_dreamspace")
```
### Exceptions
The package comes with its own set of exceptions, arranged in this hierarchy:
For more information, and code examples, check out the docstrings and consult
the API reference linked above.
### Working with dates
Date and datetime objects, i.e. instances of the standard library
`datetime.datetime` and `datetime.date` classes, can be used anywhere in documents:
```python
import datetime
import astrapy
ASTRA_DB_APPLICATION_TOKEN = "AstraCS:..."
ASTRA_DB_API_ENDPOINT = "https://01234567-....apps.astra.datastax.com"
my_client = astrapy.DataAPIClient()
my_database = my_client.get_database(
ASTRA_DB_API_ENDPOINT,
token=ASTRA_DB_APPLICATION_TOKEN,
)
my_collection = my_database.dreams
my_collection.insert_one({"when": datetime.datetime.now()})
my_collection.insert_one({"date_of_birth": datetime.date(2000, 1, 1)})
my_collection.update_one(
{"registered_at": datetime.date(1999, 11, 14)},
{"$set": {"message": "happy Sunday!"}},
)
print(
my_collection.find_one(
{"date_of_birth": {"$lt": datetime.date(2001, 1, 1)}},
projection={"_id": False},
)
)
# This would print:
# {'date_of_birth': datetime.datetime(2000, 1, 1, 0, 0)}
```
_**Note**: reads from a collection will always_
_return the `datetime` class regardless of wheter a `date` or a `datetime` was provided_
_in the insertion._
### Working with ObjectIds and UUIDs
Astrapy repackages the ObjectId from `bson` and the UUID class and utilities
from the `uuid` package and its `uuidv6` extension. You can also use them directly.
Even when setting a default ID type for a collection, you still retain the freedom
to use any ID type for any document:
```python
import astrapy
import bson
ASTRA_DB_APPLICATION_TOKEN = "AstraCS:..."
ASTRA_DB_API_ENDPOINT = "https://01234567-....apps.astra.datastax.com"
my_client = astrapy.DataAPIClient()
my_database = my_client.get_database(
ASTRA_DB_API_ENDPOINT,
token=ASTRA_DB_APPLICATION_TOKEN,
)
my_collection = my_database.create_collection(
"ecommerce",
default_id_type=astrapy.constants.DefaultIdType.UUIDV6,
)
my_collection.insert_one({"_id": astrapy.ids.ObjectId("65fd9b52d7fabba03349d013")})
my_collection.find({
"_id": astrapy.ids.UUID("018e65c9-e33d-749b-9386-e848739582f0"),
})
my_collection.update_one(
{"tag": "in_stock"},
{"$set": {"inventory_id": bson.objectid.ObjectId()}},
upsert=True,
)
my_collection.insert_one({"_id": astrapy.ids.uuid8()})
```
## For contributors
First install poetry with `pip install poetry` and then the project dependencies with `poetry install --with dev`.
Linter, style and typecheck should all pass for a PR:
```bash
make format
```
With `make format-fix` the style and imports are autofixed (by `ruff`)
Features must be thoroughly covered in tests (see `tests/idiomatic/*` for
naming convention and module structure).
### Running tests
Tests are grouped in three _blocks_ (in as many subdirs of `tests/`):
- **idiomatic**: all 1.0+ classes and APIs, except...
- **vectorize**: ... everything making use of `$vectorize` (within the idiomatic classes)
- _(core: pre-1.0 classes). Frozen as of v1.5, deprecated for removal in v2.0_
Actually, for convenience, _sub-blocks_ of tests are considered:
- **idiomatic regular**: everything except the admin parts
- **idiomatic admin Astra**: the Astra-specific admin operations
- **idiomatic admin nonAstra**: the nonAstra-specific admin operations
- **vectorize in-depth**: many Data API interactions for a single choice of provider/model. This is mostly test the client
- **vectorize all-providers**: a slightly more shallow test repeated for all providers, models, auth methods etc. This is mostly testing the API
- _(core regular: everything except DevOps interactions)_
- _(core ops: core DevOps operations)_
Tests can be run on three types of Data API _targets_ (with slight differences in what is applicable):
- **DockerCompose**: HCD started by the test initialization with `docker-compose`. _Note that in this case you will have to manually destroy the created containers._
- **nonAstra**: a ready-to-use (user-supplied) local Data API
- **Astra**: an Astra DB target account (or two, as some tests are specific to dev environment)
Depending on the (sub-block, target) combination, some environment variables may be needed.
Templates for the environment variables are to be found in `tests/env_templates`.
The general expectation is that idiomatic non-Admin tests, and vectorize in-depth tests, are
part of the main CI flow; conversely, core, admin and vectorize all-providers are kept as a
manual task to run (locally in most cases) when circumstances require it (use your judgement).
#### Required environment variables
Below is a detail of the reference template files needed for the various types
of testing:
- **DockerCompose**: generally no variables needed, except:
- **vectorize in-depth**: provide as in `env.vectorize-minimal.template`
- **vectorize all-providers**: provide as in `env.vectorize.template`
- (also note that _core ops_ and _idiomatic admin Astra_ amount to nothing in this case)
- **nonAstra**: all tests require as in `env.local.template`, plus:
- **vectorize in-depth**: also provide as in `env.vectorize-minimal.template`
- **vectorize all-providers**: also provide as in `env.vectorize.template`
- (also note that _core ops_ and _idiomatic admin Astra_ amount to nothing in this case)
- **Astra**: all tests require as in `env.astra.template`, plus:
- **core ops**: the token must have at least "Database Administrator" role (possibly through definition of a separate `ASTRA_DB_OPS_APPLICATION_TOKEN`), and `ASTRA_DB_ID` must also be defined
- **idiomatic admin Astra**: also provide as in `env.astra.admin.template`
- **vectorize in-depth**: also provide as in `env.vectorize-minimal.template`
- **vectorize all-providers**: also provide as in `env.vectorize.template`
- (also note that _idiomatic admin nonAstra_ amounts to nothing in this case)
#### Sample testing commands
For the **DockerCompose** case, prepend all of the following with `DOCKER_COMPOSE_LOCAL_DATA_API="yes" `.
All the usual `pytest` ways of restricting the test selection hold in addition
(e.g. `poetry run pytest tests/idiomatic/unit` or `[...] -k <test_name_selector>`).
##### _core regular_:
```
poetry run pytest tests/core
```
##### _core ops_:
Note the special variable needed to actually run this. You will have to manually clean up afterwards.
```
TEST_ASTRADBOPS="1" poetry run pytest tests/core/test_ops.py
```
##### _idiomatic regular_:
Warning: this will also trigger the very long-running _idiomatic admin Astra_ if the vars as in `env.astra.admin.template` are also detected. Likewise, the _idiomatic admin nonAstra_ may start (if `DO_IDIOMATIC_ADMIN_TESTS` is set), which however takes few seconds.
```
poetry run pytest tests/idiomatic
```
##### _idiomatic admin Astra_:
```
poetry run pytest tests/idiomatic/integration/test_admin.py
```
##### _idiomatic admin nonAstra_:
```
DO_IDIOMATIC_ADMIN_TESTS="1" poetry run pytest tests/idiomatic/integration/test_nonastra_admin.py
```
##### _vectorize in-depth_:
```
poetry run pytest tests/vectorize_idiomatic/integration/test_vectorize_methods*.py
```
or just:
```
poetry run pytest tests/vectorize_idiomatic/integration/test_vectorize_methods_sync.py
```
##### _vectorize all-providers_:
This generates all possible test cases and runs them:
```
poetry run pytest tests/vectorize_idiomatic
```
For a spot test, you may restrict to one case, e.g.
```
EMBEDDING_MODEL_TAGS="openai/text-embedding-3-large/HEADER/0" poetry run pytest tests/vectorize_idiomatic/integration/test_vectorize_providers.py -k test_vectorize_usage_auth_type_header_sync
```
#### Useful flags for testing
Remove logging noise with:
```
poetry run pytest [...] -o log_cli=0
```
Increase logging level to `DEBUG` (i.e. level `10`):
```
poetry run pytest [...] -o log_cli=1 --log-cli-level=10
```
Do not drop collections (valid for core):
```
TEST_SKIP_COLLECTION_DELETE=1 poetry run pytest [...]
```
## Appendices
### Appendix A: quick reference for imports
Client, data and admin abstractions:
```python
from astrapy import (
DataAPIClient,
Database,
AsyncDatabase,
Collection,
AsyncCollection,
AstraDBAdmin,
AstraDBDatabaseAdmin,
DataAPIDatabaseAdmin,
)
```
Constants for data-related use:
```python
from astrapy.constants import (
ReturnDocument,
SortDocuments,
VectorMetric,
DefaultIdType,
Environment,
)
```
ObjectIds and UUIDs:
```python
from astrapy.ids import (
ObjectId,
uuid1,
uuid3,
uuid4,
uuid5,
uuid6,
uuid7,
uuid8,
UUID,
)
```
Operations (for `bulk_write` collection method):
```python
from astrapy.operations import (
BaseOperation,
InsertOne,
InsertMany,
UpdateOne,
UpdateMany,
ReplaceOne,
DeleteOne,
DeleteMany,
AsyncBaseOperation,
AsyncInsertOne,
AsyncInsertMany,
AsyncUpdateOne,
AsyncUpdateMany,
AsyncReplaceOne,
AsyncDeleteOne,
AsyncDeleteMany,
)
```
Result classes:
```python
from astrapy.results import (
OperationResult,
DeleteResult,
InsertOneResult,
InsertManyResult,
UpdateResult,
BulkWriteResult,
)
```
Exceptions:
```python
from astrapy.exceptions import (
BulkWriteException,
CollectionAlreadyExistsException,
CollectionNotFoundException,
CumulativeOperationException,
CursorIsStartedException,
DataAPIDetailedErrorDescriptor,
DataAPIErrorDescriptor,
DataAPIException,
DataAPIFaultyResponseException,
DataAPIHttpException,
DataAPIResponseException,
DataAPITimeoutException,
DeleteManyException,
DevOpsAPIErrorDescriptor,
DevOpsAPIException,
DevOpsAPIFaultyResponseException,
DevOpsAPIHttpException,
DevOpsAPIResponseException,
DevOpsAPITimeoutException,
InsertManyException,
TooManyDocumentsToCountException,
UpdateManyException,
)
```
Info/metadata classes:
```python
from astrapy.info import (
AdminDatabaseInfo,
DatabaseInfo,
CollectionInfo,
CollectionVectorServiceOptions,
CollectionDefaultIDOptions,
CollectionVectorOptions,
CollectionOptions,
CollectionDescriptor,
EmbeddingProviderParameter,
EmbeddingProviderModel,
EmbeddingProviderToken,
EmbeddingProviderAuthentication,
EmbeddingProvider,
FindEmbeddingProvidersResult,
)
```
Admin-related classes and constants:
```python
from astrapy.admin import (
ParsedAPIEndpoint,
)
```
Cursors:
```python
from astrapy.cursors import (
BaseCursor,
Cursor,
AsyncCursor,
CommandCursor,
AsyncCommandCursor,
)
```
### Appendix B: compatibility with pre-1.0.0 library
If your code still uses the pre-1.0.0 astrapy (i.e. `from astrapy.db import AstraDB, AstraDBCollection` and so on)
you are strongly advised to migrate to the current API, which has more capabilities and improved interfaces.
All of the astrapy pre-1.0 API (now dubbed "core") works throughout *astrapy v1*, albeit with a deprecation warning
on astrapy v. 1.5.
Version 1.5 (the first to not wrap internally "core" as the engine of its own "idiomatic" API) introduces
several deprecation notices (nothing is retired yet), including a submodule-wide deprecation of "core".
**Version 2 of astrapy will finally remove "core" entirely (along with a few other things).**
#### v1 is fully compatible with "core", i.e. with pre-1.0.0
That being said, there are no known breakings of backward compatibility:
**legacy code would run with astrapy v1 just as well**
Here is a recap of the minor changes that came _to the old API_ with 1.0.0 (and beyond, up to 1.5):
- added a submodule-wide deprecation warning of the whole "core" library (v 1.5)
- added 'options' parameter to [Async]AstraDBCollection.update_one (v. 1.4.2+)
- prefetched find iterators: fix second-thread hangups in some cases (v. 1.4.2+)
- Added support for null tokens (with the effect of no authentication/token header in requests)
- Added Content-Type header to all HTTP requests to the API
- Added methods to `[Async]AstraDBCollection`: `delete_one_filter`,
- Paginated find methods (sync/async) type change from Iterable to Generator
- Bugfix: handling of the mutable caller identity in copy and convert (sync/async) methods
- Default value of `sort` is `None` and not `{}` for `find` (sync/async)
- Introduction of `[Async]AstraDBCollection.chunked_delete_many` method
- Added `projection` parameter to `find_one_and[replace/update]` (sync/async)
- Bugfix: projection was silently ignored in `vector_find_one_and_[replace/update]` (sync/async)
- Added `options` to `update_many` (sync/async)
- `[Async]AstraDBDatabase.chunked_insert_many` does not intercept generic exceptions anymore, only `APIRequestError`
- Bugfix: `AsyncAstraDBCollection.async chunked_insert_many` stops at the first error when `ordered=True`
- Added payload info to `DataAPIException`
- Added `find_one_and_delete` method (sync/async)
- Added `skip_error_check` parameter to `delete_many` (sync/async)
- Timeout support throughout the library
- Added `sort` to `update_one`, `delete_one` and `delete_one_by_predicate` methods (sync/async)
- Full support for UUID v1,3,4,5,6,7,8 and ObjectID at the collection data I/O level
- `AstraDBOps.create_database` raises errors in case of failures
- `AstraDBOps.create_database`, return type corrected
- Fixed behaviour and return type of `AstraDBOps.create_keyspace` and `AstraDBOps.terminate_db`
- Added `AstraDBOps.delete_keyspace` method
- Method `create_collection` of `AstraDB` relaxes checks on passing `dimensions` for vector collections
- AstraDBOps core class acquired async methods: `async_get_databases`, `async_get_database`, `async_create_database`, `async_terminate_database`, `async_create_keyspace`, `async_delete_keyspace`
Raw data
{
"_id": null,
"home_page": "https://github.com/datastax/astrapy",
"name": "astrapy",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0.0,>=3.8.0",
"maintainer_email": null,
"keywords": "DataStax, Astra",
"author": "Stefano Lottini",
"author_email": "stefano.lottini@datastax.com",
"download_url": "https://files.pythonhosted.org/packages/29/cc/5996efd0598b97d44699f04b5c025080fb1665f30794010f06543f3322a7/astrapy-1.5.2.tar.gz",
"platform": null,
"description": "# AstraPy\n\nA pythonic client for [DataStax Astra DB](https://astra.datastax.com).\n\n_This README targets AstraPy version **1.0.0+**, which introduces a whole new API.\nClick [here](https://github.com/datastax/astrapy/blob/cd3f5ce8146093e10a095709c0f5c3f8e3f2c7da/README.md) for the pre-existing API (fully compatible with newer versions)._\n\n\n## Quickstart\n\nInstall with `pip install astrapy`.\n\nGet the *API Endpoint* and the *Token* to your Astra DB instance at [astra.datastax.com](https://astra.datastax.com).\n\nTry the following code after replacing the connection parameters:\n\n```python\nimport astrapy\n\nASTRA_DB_APPLICATION_TOKEN = \"AstraCS:...\"\nASTRA_DB_API_ENDPOINT = \"https://01234567-....apps.astra.datastax.com\"\n\nmy_client = astrapy.DataAPIClient()\nmy_database = my_client.get_database(\n ASTRA_DB_API_ENDPOINT,\n token=ASTRA_DB_APPLICATION_TOKEN,\n)\n\nmy_collection = my_database.create_collection(\n \"dreams\",\n dimension=3,\n metric=astrapy.constants.VectorMetric.COSINE,\n)\n\nmy_collection.insert_one({\"summary\": \"I was flying\", \"$vector\": [-0.4, 0.7, 0]})\n\nmy_collection.insert_many(\n [\n {\n \"_id\": astrapy.ids.UUID(\"018e65c9-e33d-749b-9386-e848739582f0\"),\n \"summary\": \"A dinner on the Moon\",\n \"$vector\": [0.2, -0.3, -0.5],\n },\n {\n \"summary\": \"Riding the waves\",\n \"tags\": [\"sport\"],\n \"$vector\": [0, 0.2, 1],\n },\n {\n \"summary\": \"Friendly aliens in town\",\n \"tags\": [\"scifi\"],\n \"$vector\": [-0.3, 0, 0.8],\n },\n {\n \"summary\": \"Meeting Beethoven at the dentist\",\n \"$vector\": [0.2, 0.6, 0],\n },\n ],\n)\n\nmy_collection.update_one(\n {\"tags\": \"sport\"},\n {\"$set\": {\"summary\": \"Surfers' paradise\"}},\n)\n\ncursor = my_collection.find(\n {},\n sort={\"$vector\": [0, 0.2, 0.4]},\n limit=2,\n include_similarity=True,\n)\n\nfor result in cursor:\n print(f\"{result['summary']}: {result['$similarity']}\")\n\n# This would print:\n# Surfers' paradise: 0.98238194\n# Friendly aliens in town: 0.91873914\n```\n\nNext steps:\n\n- More info and usage patterns are given in the docstrings of classes and methods\n- [Data API reference](https://docs.datastax.com/en/astra-db-serverless/api-reference/overview.html)\n- [AstraPy reference](https://docs.datastax.com/en/astra-api-docs/_attachments/python-client/astrapy/index.html)\n- Package on [PyPI](https://pypi.org/project/astrapy/)\n\n### Usage with HCD and other non-Astra installations\n\nThe main difference to target e.g. a Hyper-Converged Database (HCD)\ninstallation is how the client is\ninitialized. Here is a short example showing just how to get to a `Database`\n(what comes next is unchaged compared to using Astra DB).\n\n```python\nfrom astrapy import DataAPIClient\nfrom astrapy.constants import Environment\nfrom astrapy.authentication import UsernamePasswordTokenProvider\n\n\n# Build a token\ntp = UsernamePasswordTokenProvider(\"username\", \"password\")\n\n# Initialize the client and get a \"Database\" object\nclient = DataAPIClient(token=tp, environment=Environment.HCD)\ndatabase = client.get_database(\"http://localhost:8181\", token=tp)\n```\n\nFor more on this case, please consult the [dedicated reference](https://docs.datastax.com/en/hyper-converged-database/1.0/connect/python-client.html).\n\n## AstraPy's API\n\n### Abstraction diagram\n\nAstraPy's abstractions for working at the data and admin layers are structured\nas depicted by this diagram:\n\n\n\nHere's a small admin-oriented example:\n\n```python\nimport astrapy\n\n\n# this must have \"Database Administrator\" permissions:\nASTRA_DB_APPLICATION_TOKEN = \"AstraCS:...\"\n\nmy_client = astrapy.DataAPIClient(ASTRA_DB_APPLICATION_TOKEN)\n\nmy_astra_admin = my_client.get_admin()\n\ndatabase_list = list(my_astra_admin.list_databases())\n\ndb_info = database_list[0].info\nprint(db_info.name, db_info.id, db_info.region)\n\nmy_database_admin = my_astra_admin.get_database_admin(db_info.id)\n\nmy_database_admin.list_keyspaces()\nmy_database_admin.create_keyspace(\"my_dreamspace\")\n```\n\n### Exceptions\n\nThe package comes with its own set of exceptions, arranged in this hierarchy:\n\n\n\nFor more information, and code examples, check out the docstrings and consult\nthe API reference linked above.\n\n### Working with dates\n\nDate and datetime objects, i.e. instances of the standard library\n`datetime.datetime` and `datetime.date` classes, can be used anywhere in documents:\n\n```python\nimport datetime\nimport astrapy\n\n\nASTRA_DB_APPLICATION_TOKEN = \"AstraCS:...\"\nASTRA_DB_API_ENDPOINT = \"https://01234567-....apps.astra.datastax.com\"\n\nmy_client = astrapy.DataAPIClient()\nmy_database = my_client.get_database(\n ASTRA_DB_API_ENDPOINT,\n token=ASTRA_DB_APPLICATION_TOKEN,\n)\nmy_collection = my_database.dreams\n\nmy_collection.insert_one({\"when\": datetime.datetime.now()})\nmy_collection.insert_one({\"date_of_birth\": datetime.date(2000, 1, 1)})\n\nmy_collection.update_one(\n {\"registered_at\": datetime.date(1999, 11, 14)},\n {\"$set\": {\"message\": \"happy Sunday!\"}},\n)\n\nprint(\n my_collection.find_one(\n {\"date_of_birth\": {\"$lt\": datetime.date(2001, 1, 1)}},\n projection={\"_id\": False},\n )\n)\n# This would print:\n# {'date_of_birth': datetime.datetime(2000, 1, 1, 0, 0)}\n```\n\n_**Note**: reads from a collection will always_\n_return the `datetime` class regardless of wheter a `date` or a `datetime` was provided_\n_in the insertion._\n\n### Working with ObjectIds and UUIDs\n\nAstrapy repackages the ObjectId from `bson` and the UUID class and utilities\nfrom the `uuid` package and its `uuidv6` extension. You can also use them directly.\n\nEven when setting a default ID type for a collection, you still retain the freedom\nto use any ID type for any document:\n\n```python\nimport astrapy\nimport bson\n\nASTRA_DB_APPLICATION_TOKEN = \"AstraCS:...\"\nASTRA_DB_API_ENDPOINT = \"https://01234567-....apps.astra.datastax.com\"\n\nmy_client = astrapy.DataAPIClient()\nmy_database = my_client.get_database(\n ASTRA_DB_API_ENDPOINT,\n token=ASTRA_DB_APPLICATION_TOKEN,\n)\n\nmy_collection = my_database.create_collection(\n \"ecommerce\",\n default_id_type=astrapy.constants.DefaultIdType.UUIDV6,\n)\n\nmy_collection.insert_one({\"_id\": astrapy.ids.ObjectId(\"65fd9b52d7fabba03349d013\")})\nmy_collection.find({\n \"_id\": astrapy.ids.UUID(\"018e65c9-e33d-749b-9386-e848739582f0\"),\n})\n\nmy_collection.update_one(\n {\"tag\": \"in_stock\"},\n {\"$set\": {\"inventory_id\": bson.objectid.ObjectId()}},\n upsert=True,\n)\n\nmy_collection.insert_one({\"_id\": astrapy.ids.uuid8()})\n```\n\n## For contributors\n\nFirst install poetry with `pip install poetry` and then the project dependencies with `poetry install --with dev`.\n\nLinter, style and typecheck should all pass for a PR:\n\n```bash\nmake format\n```\n\nWith `make format-fix` the style and imports are autofixed (by `ruff`)\n\nFeatures must be thoroughly covered in tests (see `tests/idiomatic/*` for\nnaming convention and module structure).\n\n### Running tests\n\nTests are grouped in three _blocks_ (in as many subdirs of `tests/`):\n\n- **idiomatic**: all 1.0+ classes and APIs, except...\n- **vectorize**: ... everything making use of `$vectorize` (within the idiomatic classes)\n- _(core: pre-1.0 classes). Frozen as of v1.5, deprecated for removal in v2.0_\n\nActually, for convenience, _sub-blocks_ of tests are considered:\n\n- **idiomatic regular**: everything except the admin parts\n- **idiomatic admin Astra**: the Astra-specific admin operations\n- **idiomatic admin nonAstra**: the nonAstra-specific admin operations\n- **vectorize in-depth**: many Data API interactions for a single choice of provider/model. This is mostly test the client\n- **vectorize all-providers**: a slightly more shallow test repeated for all providers, models, auth methods etc. This is mostly testing the API\n- _(core regular: everything except DevOps interactions)_\n- _(core ops: core DevOps operations)_\n\nTests can be run on three types of Data API _targets_ (with slight differences in what is applicable):\n\n- **DockerCompose**: HCD started by the test initialization with `docker-compose`. _Note that in this case you will have to manually destroy the created containers._\n- **nonAstra**: a ready-to-use (user-supplied) local Data API\n- **Astra**: an Astra DB target account (or two, as some tests are specific to dev environment)\n\nDepending on the (sub-block, target) combination, some environment variables may be needed.\nTemplates for the environment variables are to be found in `tests/env_templates`.\n\nThe general expectation is that idiomatic non-Admin tests, and vectorize in-depth tests, are\npart of the main CI flow; conversely, core, admin and vectorize all-providers are kept as a\nmanual task to run (locally in most cases) when circumstances require it (use your judgement).\n\n#### Required environment variables\n\nBelow is a detail of the reference template files needed for the various types\nof testing:\n\n- **DockerCompose**: generally no variables needed, except:\n - **vectorize in-depth**: provide as in `env.vectorize-minimal.template`\n - **vectorize all-providers**: provide as in `env.vectorize.template`\n - (also note that _core ops_ and _idiomatic admin Astra_ amount to nothing in this case)\n- **nonAstra**: all tests require as in `env.local.template`, plus:\n - **vectorize in-depth**: also provide as in `env.vectorize-minimal.template`\n - **vectorize all-providers**: also provide as in `env.vectorize.template`\n - (also note that _core ops_ and _idiomatic admin Astra_ amount to nothing in this case)\n- **Astra**: all tests require as in `env.astra.template`, plus:\n - **core ops**: the token must have at least \"Database Administrator\" role (possibly through definition of a separate `ASTRA_DB_OPS_APPLICATION_TOKEN`), and `ASTRA_DB_ID` must also be defined\n - **idiomatic admin Astra**: also provide as in `env.astra.admin.template`\n - **vectorize in-depth**: also provide as in `env.vectorize-minimal.template`\n - **vectorize all-providers**: also provide as in `env.vectorize.template`\n - (also note that _idiomatic admin nonAstra_ amounts to nothing in this case)\n\n#### Sample testing commands\n\nFor the **DockerCompose** case, prepend all of the following with `DOCKER_COMPOSE_LOCAL_DATA_API=\"yes\" `.\n\nAll the usual `pytest` ways of restricting the test selection hold in addition\n(e.g. `poetry run pytest tests/idiomatic/unit` or `[...] -k <test_name_selector>`).\n\n##### _core regular_:\n\n```\npoetry run pytest tests/core\n```\n\n##### _core ops_:\n\nNote the special variable needed to actually run this. You will have to manually clean up afterwards.\n\n```\nTEST_ASTRADBOPS=\"1\" poetry run pytest tests/core/test_ops.py\n```\n\n##### _idiomatic regular_:\n\nWarning: this will also trigger the very long-running _idiomatic admin Astra_ if the vars as in `env.astra.admin.template` are also detected. Likewise, the _idiomatic admin nonAstra_ may start (if `DO_IDIOMATIC_ADMIN_TESTS` is set), which however takes few seconds.\n\n```\npoetry run pytest tests/idiomatic\n```\n\n##### _idiomatic admin Astra_:\n\n```\npoetry run pytest tests/idiomatic/integration/test_admin.py \n```\n\n##### _idiomatic admin nonAstra_:\n\n```\nDO_IDIOMATIC_ADMIN_TESTS=\"1\" poetry run pytest tests/idiomatic/integration/test_nonastra_admin.py\n```\n\n##### _vectorize in-depth_:\n\n```\npoetry run pytest tests/vectorize_idiomatic/integration/test_vectorize_methods*.py\n```\n\nor just:\n\n```\npoetry run pytest tests/vectorize_idiomatic/integration/test_vectorize_methods_sync.py\n```\n\n##### _vectorize all-providers_:\n\nThis generates all possible test cases and runs them:\n\n```\npoetry run pytest tests/vectorize_idiomatic\n```\n\nFor a spot test, you may restrict to one case, e.g.\n\n```\nEMBEDDING_MODEL_TAGS=\"openai/text-embedding-3-large/HEADER/0\" poetry run pytest tests/vectorize_idiomatic/integration/test_vectorize_providers.py -k test_vectorize_usage_auth_type_header_sync\n```\n\n#### Useful flags for testing\n\nRemove logging noise with:\n\n```\npoetry run pytest [...] -o log_cli=0\n```\n\nIncrease logging level to `DEBUG` (i.e. level `10`):\n\n```\npoetry run pytest [...] -o log_cli=1 --log-cli-level=10\n```\n\nDo not drop collections (valid for core):\n\n```\nTEST_SKIP_COLLECTION_DELETE=1 poetry run pytest [...]\n```\n\n\n## Appendices\n\n### Appendix A: quick reference for imports\n\nClient, data and admin abstractions:\n\n```python\nfrom astrapy import (\n DataAPIClient,\n Database,\n AsyncDatabase,\n Collection,\n AsyncCollection,\n AstraDBAdmin,\n AstraDBDatabaseAdmin,\n DataAPIDatabaseAdmin,\n)\n```\n\nConstants for data-related use:\n\n```python\nfrom astrapy.constants import (\n ReturnDocument,\n SortDocuments,\n VectorMetric,\n DefaultIdType,\n Environment,\n)\n```\n\nObjectIds and UUIDs:\n\n```python\nfrom astrapy.ids import (\n ObjectId,\n uuid1,\n uuid3,\n uuid4,\n uuid5,\n uuid6,\n uuid7,\n uuid8,\n UUID,\n)\n```\n\nOperations (for `bulk_write` collection method):\n\n```python\nfrom astrapy.operations import (\n BaseOperation,\n InsertOne,\n InsertMany,\n UpdateOne,\n UpdateMany,\n ReplaceOne,\n DeleteOne,\n DeleteMany,\n AsyncBaseOperation,\n AsyncInsertOne,\n AsyncInsertMany,\n AsyncUpdateOne,\n AsyncUpdateMany,\n AsyncReplaceOne,\n AsyncDeleteOne,\n AsyncDeleteMany,\n)\n```\n\nResult classes:\n\n```python\nfrom astrapy.results import (\n OperationResult,\n DeleteResult,\n InsertOneResult,\n InsertManyResult,\n UpdateResult,\n BulkWriteResult,\n)\n```\n\nExceptions:\n\n```python\nfrom astrapy.exceptions import (\n BulkWriteException,\n CollectionAlreadyExistsException,\n CollectionNotFoundException,\n CumulativeOperationException,\n CursorIsStartedException,\n DataAPIDetailedErrorDescriptor,\n DataAPIErrorDescriptor,\n DataAPIException,\n DataAPIFaultyResponseException,\n DataAPIHttpException,\n DataAPIResponseException,\n DataAPITimeoutException,\n DeleteManyException,\n DevOpsAPIErrorDescriptor,\n DevOpsAPIException,\n DevOpsAPIFaultyResponseException,\n DevOpsAPIHttpException,\n DevOpsAPIResponseException,\n DevOpsAPITimeoutException,\n InsertManyException,\n TooManyDocumentsToCountException,\n UpdateManyException,\n)\n```\n\nInfo/metadata classes:\n\n```python\nfrom astrapy.info import (\n AdminDatabaseInfo,\n DatabaseInfo,\n CollectionInfo,\n CollectionVectorServiceOptions,\n CollectionDefaultIDOptions,\n CollectionVectorOptions,\n CollectionOptions,\n CollectionDescriptor,\n EmbeddingProviderParameter,\n EmbeddingProviderModel,\n EmbeddingProviderToken,\n EmbeddingProviderAuthentication,\n EmbeddingProvider,\n FindEmbeddingProvidersResult,\n)\n```\n\nAdmin-related classes and constants:\n\n```python\nfrom astrapy.admin import (\n ParsedAPIEndpoint,\n)\n```\n\nCursors:\n\n```python\nfrom astrapy.cursors import (\n BaseCursor,\n Cursor,\n AsyncCursor,\n CommandCursor,\n AsyncCommandCursor,\n)\n```\n\n### Appendix B: compatibility with pre-1.0.0 library\n\nIf your code still uses the pre-1.0.0 astrapy (i.e. `from astrapy.db import AstraDB, AstraDBCollection` and so on)\nyou are strongly advised to migrate to the current API, which has more capabilities and improved interfaces.\n\nAll of the astrapy pre-1.0 API (now dubbed \"core\") works throughout *astrapy v1*, albeit with a deprecation warning\non astrapy v. 1.5.\n\nVersion 1.5 (the first to not wrap internally \"core\" as the engine of its own \"idiomatic\" API) introduces\nseveral deprecation notices (nothing is retired yet), including a submodule-wide deprecation of \"core\".\n\n**Version 2 of astrapy will finally remove \"core\" entirely (along with a few other things).**\n\n#### v1 is fully compatible with \"core\", i.e. with pre-1.0.0\n\nThat being said, there are no known breakings of backward compatibility:\n**legacy code would run with astrapy v1 just as well**\nHere is a recap of the minor changes that came _to the old API_ with 1.0.0 (and beyond, up to 1.5):\n\n- added a submodule-wide deprecation warning of the whole \"core\" library (v 1.5)\n- added 'options' parameter to [Async]AstraDBCollection.update_one (v. 1.4.2+)\n- prefetched find iterators: fix second-thread hangups in some cases (v. 1.4.2+)\n- Added support for null tokens (with the effect of no authentication/token header in requests)\n- Added Content-Type header to all HTTP requests to the API\n- Added methods to `[Async]AstraDBCollection`: `delete_one_filter`, \n- Paginated find methods (sync/async) type change from Iterable to Generator\n- Bugfix: handling of the mutable caller identity in copy and convert (sync/async) methods\n- Default value of `sort` is `None` and not `{}` for `find` (sync/async)\n- Introduction of `[Async]AstraDBCollection.chunked_delete_many` method\n- Added `projection` parameter to `find_one_and[replace/update]` (sync/async)\n- Bugfix: projection was silently ignored in `vector_find_one_and_[replace/update]` (sync/async)\n- Added `options` to `update_many` (sync/async)\n- `[Async]AstraDBDatabase.chunked_insert_many` does not intercept generic exceptions anymore, only `APIRequestError`\n- Bugfix: `AsyncAstraDBCollection.async chunked_insert_many` stops at the first error when `ordered=True`\n- Added payload info to `DataAPIException`\n- Added `find_one_and_delete` method (sync/async)\n- Added `skip_error_check` parameter to `delete_many` (sync/async)\n- Timeout support throughout the library\n- Added `sort` to `update_one`, `delete_one` and `delete_one_by_predicate` methods (sync/async)\n- Full support for UUID v1,3,4,5,6,7,8 and ObjectID at the collection data I/O level\n- `AstraDBOps.create_database` raises errors in case of failures\n- `AstraDBOps.create_database`, return type corrected\n- Fixed behaviour and return type of `AstraDBOps.create_keyspace` and `AstraDBOps.terminate_db`\n- Added `AstraDBOps.delete_keyspace` method\n- Method `create_collection` of `AstraDB` relaxes checks on passing `dimensions` for vector collections\n- AstraDBOps core class acquired async methods: `async_get_databases`, `async_get_database`, `async_create_database`, `async_terminate_database`, `async_create_keyspace`, `async_delete_keyspace`\n\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "AstraPy is a Pythonic SDK for DataStax Astra and its Data API",
"version": "1.5.2",
"project_urls": {
"Homepage": "https://github.com/datastax/astrapy",
"Repository": "https://github.com/datastax/astrapy"
},
"split_keywords": [
"datastax",
" astra"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "db48684c270724bc3f8d12714556d201aa4610623da919505a6a09e56f50ef6a",
"md5": "8f8b70ac57c867ba005e8e604311ad88",
"sha256": "598b86de723727a11ec43e1c7fe682ecb42d63d37a94165fb08de41c20103f56"
},
"downloads": -1,
"filename": "astrapy-1.5.2-py3-none-any.whl",
"has_sig": false,
"md5_digest": "8f8b70ac57c867ba005e8e604311ad88",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0.0,>=3.8.0",
"size": 177128,
"upload_time": "2024-10-08T21:51:46",
"upload_time_iso_8601": "2024-10-08T21:51:46.745158Z",
"url": "https://files.pythonhosted.org/packages/db/48/684c270724bc3f8d12714556d201aa4610623da919505a6a09e56f50ef6a/astrapy-1.5.2-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "29cc5996efd0598b97d44699f04b5c025080fb1665f30794010f06543f3322a7",
"md5": "ab584875cc681a790ec85c4162bfe475",
"sha256": "eaf703628b0d03891ae7c391ef04ff3aec1005837fdfa47c19f2ed4478c45a4a"
},
"downloads": -1,
"filename": "astrapy-1.5.2.tar.gz",
"has_sig": false,
"md5_digest": "ab584875cc681a790ec85c4162bfe475",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0.0,>=3.8.0",
"size": 163233,
"upload_time": "2024-10-08T21:51:48",
"upload_time_iso_8601": "2024-10-08T21:51:48.603516Z",
"url": "https://files.pythonhosted.org/packages/29/cc/5996efd0598b97d44699f04b5c025080fb1665f30794010f06543f3322a7/astrapy-1.5.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-10-08 21:51:48",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "datastax",
"github_project": "astrapy",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "astrapy"
}
JSON
