  [](https://pypi.org/project/pydantic-settings-vault/) [](https://github.com/aleksey925/pydantic-settings-vault/actions?query=branch%3Amaster) [](https://app.codecov.io/gh/aleksey925/pydantic-settings-vault) [](https://pypi.org/project/pydantic-settings-vault/)
pydantic-settings-vault
=======================
> `pydantic-settings-vault` is a fork `pydantic-vault` with `pydantic 2.x` support.
A simple extension to [pydantic-settings][pydantic-basesettings] that can retrieve secrets stored in [Hashicorp Vault][vault].
With pydantic-settings and pydantic-settings-vault, you can easily declare your configuration in a type-hinted class, and load configuration
from environment variables or Vault secrets. pydantic-settings-vault will work the same when developing locally (where you probably
login with the Vault CLI and your own user account) and when deploying in production (using a Vault Approle or Kubernetes
authentication for example).
<!-- toc -->
- [Installation](#installation)
- [Getting started](#getting-started)
- [Documentation](#documentation)
* [`Field` additional parameters](#field-additional-parameters)
* [Configuration](#configuration)
* [Authentication](#authentication)
+ [Approle](#approle)
+ [Kubernetes](#kubernetes)
+ [Vault token](#vault-token)
+ [JWT/OIDC](#jwtoidc)
* [Order of priority](#order-of-priority)
- [Logging](#logging)
- [Examples](#examples)
* [Retrieve a secret from a KV v2 secret engine](#retrieve-a-secret-from-a-kv-v2-secret-engine)
* [Retrieve a whole secret at once](#retrieve-a-whole-secret-at-once)
* [Retrieve a secret from a KV v1 secret engine](#retrieve-a-secret-from-a-kv-v1-secret-engine)
* [Retrieve a secret from a database secret engine](#retrieve-a-secret-from-a-database-secret-engine)
* [Use a dynamic path to retrieve secrets](#use-a-dynamic-path-to-retrieve-secrets)
- [Known limitations](#known-limitations)
- [Inspirations](#inspirations)
- [License](#license)
- [Development](#development)
* [Debugging with a real Vault server](#debugging-with-a-real-vault-server)
<!-- tocstop -->
## Installation
```shell
pip install pydantic-settings-vault
# or if you use Poetry or Pipenv
poetry add pydantic-settings-vault
pipenv install pydantic-settings-vault
```
## Getting started
With `pydantic_settings.BaseSettings` class, you can easily "create a clearly-defined, type-hinted
application configuration class" that gets its configuration from environment variables. It will work the same when
developing locally (where you probably login with the Vault CLI and your own user account) and when deploying in
production (using a Vault Approle, Kubernetes or JWT/OIDC authentication for example).
You can create a normal `BaseSettings` class, and define the `settings_customise_sources()` method to load secrets from your Vault instance using the `VaultSettingsSource` class:
```python
import os
from pydantic import Field, SecretStr
from pydantic_settings import BaseSettings, PydanticBaseSettingsSource
from pydantic_vault import VaultSettingsSource
class Settings(BaseSettings):
# The `vault_secret_path` is the full path (with mount point included) to the secret
# The `vault_secret_key` is the specific key to extract from a secret
username: str = Field(
...,
json_schema_extra={
"vault_secret_path": "secret/data/path/to/secret",
"vault_secret_key": "my_user",
},
)
password: SecretStr = Field(
...,
json_schema_extra={
"vault_secret_path": "secret/data/path/to/secret",
"vault_secret_key": "my_password",
},
)
model_config = {
"vault_url": "https://vault.tld",
"vault_token": os.environ["VAULT_TOKEN"],
"vault_namespace": "your/namespace", # Optional, pydantic-settings-vault supports Vault namespaces (for Vault Enterprise)
}
@classmethod
def settings_customise_sources(
cls,
settings_cls: type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
# This is where you can choose which settings sources to use and their priority
return (
init_settings,
env_settings,
dotenv_settings,
VaultSettingsSource(settings_cls),
file_secret_settings,
)
settings = Settings()
# These variables will come from the Vault secret you configured
settings.username
settings.password.get_secret_value()
# Now let's pretend we have already set the USERNAME in an environment variable
# (see the Pydantic documentation for more information and to know how to configure it)
# With the priority order we defined above, its value will override the Vault secret
os.environ["USERNAME"] = "my user"
settings = Settings()
settings.username # "my user", defined in the environment variable
settings.password.get_secret_value() # the value set in Vault
```
## Documentation
### `Field` additional parameters
You might have noticed that we import `Field` directly from Pydantic. pydantic-settings-vault doesn't add any custom logic to it, which means you can still use everything you know and love from Pydantic.
The additional parameters pydantic-settings-vault uses are:
| Parameter name | Required | Description |
|-----------------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------|
| `vault_secret_path` | **Yes** | The path to your secret in Vault<br>This needs to be the *full path* to the secret, including its mount point (see [examples](#examples) below) |
| `vault_secret_key` | No | The key to use in the secret<br>If it is not specified the whole secret content will be loaded as a dict (see [examples](#examples) below) |
For example, if you create a secret `database/prod` with a key `password` and a value of `a secret password` in a KV v2 secret engine mounted at the default `secret/` location, you would access it with
```python
password: SecretStr = Field(
...,
json_schema_extra={
"vault_secret_path": "secret/data/database/prod",
"vault_secret_key": "password",
},
)
```
### Configuration
You can configure the behaviour of pydantic-settings-vault in your `Settings.model_config` dict, or using environment variables:
| Settings name | Type | Required | Environment variable | Description |
|---------------------------------|-----------------------|----------|--------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------|
| `settings_customise_sources()` | | **Yes** | N/A | You need to implement this function to use Vault as a settings source, and choose the priority order you want |
| `vault_url` | `str` | **Yes** | `VAULT_ADDR` | Your Vault URL |
| `vault_namespace` | `str \| None` | No | `VAULT_NAMESPACE` | Your Vault namespace (if you use one, requires Vault Enterprise) |
| `vault_auth_path` | `str \| None` | No | `VAULT_AUTH_PATH` | The path of the authentication method, such as /auth/{path}/login, if different from its default, is only supported by the JWT authentication method. |
| `vault_auth_mount_point` | `str \| None` | No | `VAULT_AUTH_MOUNT_POINT` | The mount point of the authentication method, if different from its default mount point |
| `vault_certificate_verify` | `str \| bool \| None` | No | `VAULT_CA_BUNDLE` | The path to a CA bundle validating your Vault certificate, or `False` to disable verification (see [hvac docs][hvac-private-ca]) |
Environment variables override what has been defined in the `Config` class.
You can also configure everything available in the original Pydantic `BaseSettings` class.
### Authentication
pydantic-settings-vault supports the following authentication method (in descending order of priority):
- [direct token authentication][vault-auth-token]
- [kubernetes][vault-auth-kubernetes]
- [approle][vault-auth-approle]
- [jwt/oidc][vault-auth-jwt-oidc]
pydantic-settings-vault tries to be transparent and help you work, both during local development and in production. It will try to
find the required information for the first authentication method, if it can't it goes on to the next method, until it
has exhausted all authentication methods. In this case it gives up and logs the failure.
You only need to know this order of priority if you specify the authentication parameters for multiple methods.
Support is planned for GKE authentication methods (contributions welcome! :wink:).
#### Approle
To authenticate using the [Approle auth method][vault-auth-approle], you need to pass a role ID and a secret ID to your Settings class.
pydantic-settings-vault reads this information from the following sources (in descending order of priority):
- the `VAULT_ROLE_ID` and `VAULT_SECRET_ID` environment variables
- the `vault_role_id` and `vault_secret_id` configuration fields in your `Settings.model_config` dict (`vault_secret_id` can be a `str` or a `SecretStr`)
You can also mix-and-match, e.g. write the role ID in your `Settings.model_config` dict and retrieve the secret ID from the environment at runtime.
Example:
```python
from pydantic import Field, SecretStr
from pydantic_settings import BaseSettings, PydanticBaseSettingsSource
from pydantic_vault import VaultSettingsSource
class Settings(BaseSettings):
username: str = Field(
...,
json_schema_extra={
"vault_secret_path": "path/to/secret",
"vault_secret_key": "my_user",
},
)
password: SecretStr = Field(
...,
json_schema_extra={
"vault_secret_path": "path/to/secret",
"vault_secret_key": "my_password",
},
)
model_config = {
"vault_url": "https://vault.tld",
"vault_role_id": "my-role-id",
"vault_secret_id": SecretStr("my-secret-id"),
}
@classmethod
def settings_customise_sources(
cls,
settings_cls: type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
return (
init_settings,
env_settings,
dotenv_settings,
VaultSettingsSource(settings_cls),
file_secret_settings,
)
```
#### Kubernetes
To authenticate using the [Kubernetes auth method][vault-auth-kubernetes], you need to pass a role to your Settings class.
pydantic-settings-vault reads this information from the following sources (in descending order of priority):
- the `VAULT_KUBERNETES_ROLE` environment variable
- the `vault_kubernetes_role` configuration field in your `Settings.model_config` dict, which must be a `str`
The Kubernetes service account token will be read from the file at `/var/run/secrets/kubernetes.io/serviceaccount/token`.
Example:
```python
from pydantic import Field, SecretStr
from pydantic_settings import BaseSettings, PydanticBaseSettingsSource
from pydantic_vault import VaultSettingsSource
class Settings(BaseSettings):
username: str = Field(
...,
json_schema_extra={
"vault_secret_path": "path/to/secret",
"vault_secret_key": "my_user",
},
)
password: SecretStr = Field(
...,
json_schema_extra={
"vault_secret_path": "path/to/secret",
"vault_secret_key": "my_password",
},
)
model_config = {
"vault_url": "https://vault.tld",
"vault_kubernetes_role": "my-role",
}
@classmethod
def settings_customise_sources(
cls,
settings_cls: type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
return (
init_settings,
env_settings,
dotenv_settings,
VaultSettingsSource(settings_cls),
file_secret_settings,
)
```
#### Vault token
To authenticate using the [Token auth method][vault-auth-token], you need to pass a Vault token to your `Settings` class.
pydantic-settings-vault reads this token from the following sources (in descending order of priority):
- the `VAULT_TOKEN` environment variable
- the `~/.vault-token` file (so you can use the `vault` CLI to login locally, pydantic-settings-vault will transparently reuse its token)
- the `vault_token` configuration field in your `Settings.model_config` dict, which can be a `str` or a `SecretStr`
Example:
```python
from pydantic import Field, SecretStr
from pydantic_settings import BaseSettings, PydanticBaseSettingsSource
from pydantic_vault import VaultSettingsSource
class Settings(BaseSettings):
username: str = Field(
...,
json_schema_extra={
"vault_secret_path": "path/to/secret",
"vault_secret_key": "my_user",
},
)
password: SecretStr = Field(
...,
json_schema_extra={
"vault_secret_path": "path/to/secret",
"vault_secret_key": "my_password",
},
)
model_config = {
"vault_url": "https://vault.tld",
"vault_token": SecretStr("my-secret-token"),
}
@classmethod
def settings_customise_sources(
cls,
settings_cls: type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
return (
init_settings,
env_settings,
dotenv_settings,
VaultSettingsSource(settings_cls),
file_secret_settings,
)
```
#### JWT/OIDC
To authenticate using the [JWT/OIDC method][vault-auth-jwt-oidc], you need to pass
a token role and a token itself to your Settings class.
pydantic-settings-vault reads this information from the following sources (in descending order of priority):
- the `VAULT_JWT_ROLE` and `VAULT_JWT_TOKEN` environment variables
- the `vault_jwt_role` and `vault_jwt_token` configuration fields in your
`Settings.model_config` class (`vault_jwt_token` can be a `str` or a `SecretStr`)
You can also mix and match, for example, write the role in your `Settings.model_config`
class and retrieve the token from the environment at runtime.
Example:
```python
from pydantic import Field, SecretStr
from pydantic_settings import BaseSettings, PydanticBaseSettingsSource
from pydantic_vault import VaultSettingsSource
class Settings(BaseSettings):
username: str = Field(
...,
json_schema_extra={
"vault_secret_path": "path/to/secret",
"vault_secret_key": "my_user",
},
)
password: SecretStr = Field(
...,
json_schema_extra={
"vault_secret_path": "path/to/secret",
"vault_secret_key": "my_password",
},
)
model_config = {
"vault_url": "https://vault.tld",
"vault_jwt_role": "my-role",
"vault_jwt_token": SecretStr("my-token"),
}
@classmethod
def settings_customise_sources(
cls,
settings_cls: type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
return (
init_settings,
env_settings,
dotenv_settings,
VaultSettingsSource(settings_cls),
file_secret_settings,
)
```
### Order of priority
You can customize settings sources and choose the order of priority you want.
Here are some examples:
```python
from pydantic_settings import BaseSettings, PydanticBaseSettingsSource
from pydantic_vault import VaultSettingsSource
class Settings(BaseSettings):
"""
In descending order of priority:
- arguments passed to the `Settings` class initializer
- environment variables
- Vault variables
- variables loaded from the secrets directory, such as Docker Secrets
- the default field values for the `Settings` model
"""
@classmethod
def settings_customise_sources(
cls,
settings_cls: type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
return (
init_settings,
env_settings,
dotenv_settings,
VaultSettingsSource(settings_cls),
file_secret_settings,
)
class Settings(BaseSettings):
"""
In descending order of priority:
- Vault variables
- environment variables
- variables loaded from the secrets directory, such as Docker Secrets
- the default field values for the `Settings` model
Here we chose to remove the "init arguments" source,
and move the Vault source up before the environment source
"""
@classmethod
def settings_customise_sources(
cls,
settings_cls: type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
return (
VaultSettingsSource(settings_cls),
env_settings,
dotenv_settings,
file_secret_settings,
)
```
## Logging
The library exports a logger called `pydantic-vault`.
To help debugging you can change the log level. A simple way to do that if you do not have a custom log setup is:
```py
# At the beginning of your main file or entrypoint
import logging
logging.basicConfig()
logging.getLogger("pydantic-vault").setLevel(logging.DEBUG) # Change the log level here
```
## Examples
All examples use the following structure, so we will omit the imports and the `model_config` dict:
```python
from pydantic import Field
from pydantic_settings import BaseSettings, PydanticBaseSettingsSource
from pydantic_vault import VaultSettingsSource
class Settings(BaseSettings):
###############################################
# THIS PART CHANGES IN THE DIFFERENT EXAMPLES #
username: str = Field(
...,
json_schema_extra={
"vault_secret_path": "secret/data/path/to/secret",
"vault_secret_key": "my_user",
},
)
###############################################
model_config = {"vault_url": "https://vault.tld"}
@classmethod
def settings_customise_sources(
cls,
settings_cls: type[BaseSettings],
init_settings: PydanticBaseSettingsSource,
env_settings: PydanticBaseSettingsSource,
dotenv_settings: PydanticBaseSettingsSource,
file_secret_settings: PydanticBaseSettingsSource,
) -> tuple[PydanticBaseSettingsSource, ...]:
return (
init_settings,
env_settings,
dotenv_settings,
VaultSettingsSource(settings_cls),
file_secret_settings,
)
```
### Retrieve a secret from a KV v2 secret engine
Suppose your secret is at `my-api/prod` and looks like this:
```
Key Value
--- -----
root_user root
root_password a_v3ry_s3cur3_p4ssw0rd
```
Your settings class would be:
```python
class Settings(BaseSettings):
# The `vault_secret_path` is the full path (with mount point included) to the secret.
# For a KV v2 secret engine, there is always a `data/` sub-path between the mount point and
# the secret actual path, eg. if your mount point is `secret/` (the default) and your secret
# path is `my-api/prod`, the full path to use is `secret/data/my-api/prod`.
# The `vault_secret_key` is the specific key to extract from a secret.
username: str = Field(
...,
json_schema_extra={
"vault_secret_path": "secret/data/my-api/prod",
"vault_secret_key": "root_user",
},
)
password: SecretStr = Field(
...,
json_schema_extra={
"vault_secret_path": "secret/data/my-api/prod",
"vault_secret_key": "root_password",
},
)
settings = Settings()
settings.username # "root"
settings.password.get_secret_value() # "a_v3ry_s3cur3_p4ssw0rd"
```
### Retrieve a whole secret at once
If you omit the `vault_secret_key` parameter in your `Field`, pydantic-settings-vault will load
the whole secret in your class field.
With the same secret as before, located at `my-api/prod` and with this data:
```
Key Value
--- -----
root_user root
root_password a_v3ry_s3cur3_p4ssw0rd
```
You could use a settings class like this to retrieve everything in the secret:
```python
class Settings(BaseSettings):
# The `vault_secret_path` is the full path (with mount point included) to the secret.
# For a KV v2 secret engine, there is always a `data/` sub-path between the mount point and
# the secret actual path, eg. if your mount point is `secret/` (the default) and your secret
# path is `my-api/prod`, the full path to use is `secret/data/my-api/prod`.
# We don't pass a `vault_secret_key` here so that pydantic-settings-vault fetches all fields at once.
credentials: dict = Field(
..., json_schema_extra={"vault_secret_path": "secret/data/my-api/prod"}
)
settings = Settings()
settings.credentials # { "root_user": "root", "root_password": "a_v3ry_s3cur3_p4ssw0rd" }
```
You can also use a Pydantic `BaseModel` class to parse and validate the incoming secret:
```python
class Credentials(BaseModel):
root_user: str
root_password: SecretStr
class Settings(BaseSettings):
# The `vault_secret_path` is the full path (with mount point included) to the secret.
# For a KV v2 secret engine, there is always a `data/` sub-path between the mount point and
# the secret actual path, eg. if your mount point is `secret/` (the default) and your secret
# path is `my-api/prod`, the full path to use is `secret/data/my-api/prod`.
# We don't pass a `vault_secret_key` here so that pydantic-settings-vault fetches all fields at once.
credentials: Credentials = Field(
..., json_schema_extra={"vault_secret_path": "secret/data/my-api/prod"}
)
settings = Settings()
settings.credentials.root_user # "root"
settings.credentials.root_password.get_secret_value() # "a_v3ry_s3cur3_p4ssw0rd"
```
### Retrieve a secret from a KV v1 secret engine
Suppose your secret is at `my-api/prod` and looks like this:
```
Key Value
--- -----
root_user root
root_password a_v3ry_s3cur3_p4ssw0rd
```
Your settings class would be:
```python
class Settings(BaseSettings):
# The `vault_secret_path` is the full path (with mount point included) to the secret.
# For a KV v1 secret engine, the secret path is directly appended to the mount point,
# eg. if your mount point is `kv/` (the default) and your secret path is `my-api/prod`,
# the full path to use is `kv/my-api/prod` (unlike with KV v2 secret engines).
# The `vault_secret_key` is the specific key to extract from a secret.
username: str = Field(
...,
json_schema_extra={
"vault_secret_path": "kv/my-api/prod",
"vault_secret_key": "root_user",
},
)
password: SecretStr = Field(
...,
json_schema_extra={
"vault_secret_path": "kv/my-api/prod",
"vault_secret_key": "root_password",
},
)
settings = Settings()
settings.username # "root"
settings.password.get_secret_value() # "a_v3ry_s3cur3_p4ssw0rd"
```
⚠ Beware of the [known limitations](#known-limitations) on KV v1 secrets!
### Retrieve a secret from a database secret engine
Database secrets can be "dynamic", generated by Vault every time you request access.
Because every call to Vault will create a new database account, you cannot store the username
and password in two different fields in your settings class, or you would get the username of the
*first* generated account and the password of the *second* account. This means that you must *not*
pass a `vault_secret_key`, so that pydantic-settings-vault retrieves the whole secret at once.
You can store the credentials in a dict or in a custom `BaseModel` class:
```python
class DbCredentials(BaseModel):
username: str
password: SecretStr
class Settings(BaseSettings):
# The `vault_secret_path` is the full path (with mount point included) to the secret.
# For a database secret engine, the secret path is `<mount point>/creds/<role name>`.
# For example if your mount point is `database/` (the default) and your role name is
# `my-db-prod`, the full path to use is `database/creds/my-db-prod`. You will receive
# `username` and `password` fields in response.
# You must *not* pass a `vault_secret_key` so that pydantic-settings-vault fetches both fields at once.
db_creds: DbCredentials = Field(
..., json_schema_extra={"vault_secret_path": "database/creds/my-db-prod"}
)
db_creds_in_dict: dict = Field(
..., json_schema_extra={"vault_secret_path": "database/creds/my-db-prod"}
)
settings = Settings()
settings.db_creds.username # "generated-username-1"
settings.db_creds.password.get_secret_value() # "generated-password-for-username-1"
settings.db_creds_in_dict["username"] # "generated-username-2"
settings.db_creds_in_dict["password"] # "generated-password-for-username-2"
```
### Use a dynamic path to retrieve secrets
If you have different paths for your secrets (for example if you have different environments) you can use string formatting
to dynamically generate the paths depending on an environment variable.
```python
import os
# You will need to specify the environment in an environment variable, but by
# default it falls back to "dev"
ENV = os.getenv("ENV", "dev")
class Settings(BaseSettings):
# This will load different secrets depending on the value of the ENV environment variable
username: str = Field(
...,
json_schema_extra={
"vault_secret_path": f"kv/my-api/{ENV}",
"vault_secret_key": "root_user",
},
)
password: SecretStr = Field(
...,
json_schema_extra={
"vault_secret_path": f"kv/my-api/{ENV}",
"vault_secret_key": "root_password",
},
)
settings = Settings()
settings.username # "root"
settings.password.get_secret_value() # "a_v3ry_s3cur3_p4ssw0rd"
```
## Known limitations
- On KV v1 secret engines, if your secret has a `data` key and you do not specify a `vault_secret_key`
to load the whole secret at once, pydantic-settings-vault will only load the content of the `data` key.
For example, with a secret `kv/my-secret`
```
Key Value
--- -----
user root
password a_v3ry_s3cur3_p4ssw0rd
data a very important piece of data
```
and the settings class
```python
class Settings(BaseSettings):
my_secret: dict = Field(
..., json_schema_extra={"vault_secret_path": "kv/my-secret"}
)
```
pydantic-settings-vault will try to load only the `data` value (`a very important piece of data`) in
`my_secret`, which will fail validation from Pydantic because it is not a dict.
**Workaround:** Rename the `data` key in your secret 😅
**Workaround:** Migrate to KV v2
## Inspirations
- [Ansible `hashi_vault` lookup plugin][ansible hashi_vault] for the API and some code
- [Hashicorp's Vault GitHub Action][vault-action] for the API
## License
pydantic-settings-vault is available under the [MIT license](./LICENSE).
## Development
### Debugging with a real Vault server
You can use a real Vault server to debug this project. To make this process
easier, this project includes a `docker-compose.yml` file that can run a
ready-to-use Vault server.
To run the server and set it up, run the following commands:
```shell
docker-compose up
make setup-vault
```
After that, you will have a Vault server running at `http://localhost:8200`, where you can authorize in two ways:
- using the root token (which is `token`)
- using the JWT method (role=jwt_role, token=[link](./configs/vault/jwt_token.txt))
- using the AppRole method (the values of role_id and secret_id can be found in the logs of the `make setup-vault` command).
[ansible hashi_vault]: https://docs.ansible.com/ansible/latest/collections/community/hashi_vault/hashi_vault_lookup.html
[hvac-private-ca]: https://hvac.readthedocs.io/en/stable/advanced_usage.html#making-use-of-private-ca
[pydantic]: https://docs.pydantic.dev/latest/
[pydantic-basesettings]: https://docs.pydantic.dev/latest/usage/pydantic_settings/
[pydantic-basesettings-customsource]: https://docs.pydantic.dev/latest/usage/pydantic_settings/#adding-sources
[vault]: https://www.vaultproject.io/
[vault-action]: https://github.com/hashicorp/vault-action
[vault-auth-approle]: https://www.vaultproject.io/docs/auth/approle
[vault-auth-kubernetes]: https://www.vaultproject.io/docs/auth/kubernetes
[vault-auth-token]: https://www.vaultproject.io/docs/auth/token
[vault-auth-jwt-oidc]: https://developer.hashicorp.com/vault/docs/auth/jwt
[vault-kv-v2]: https://www.vaultproject.io/docs/secrets/kv/kv-v2/
Raw data
{
"_id": null,
"home_page": "https://github.com/aleksey925/pydantic-settings-vault/",
"name": "pydantic-settings-vault",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": "hashicorp, vault, hvac, pydantic",
"author": "Aleksey Petrunnik",
"author_email": "petrunnik.a@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/e9/f5/ed68fbdb8ff006fc2b6d64c6fb62eaa9550f1b47fa426ef6af03760f4a75/pydantic_settings_vault-2.1.0.tar.gz",
"platform": null,
"description": "  [](https://pypi.org/project/pydantic-settings-vault/) [](https://github.com/aleksey925/pydantic-settings-vault/actions?query=branch%3Amaster) [](https://app.codecov.io/gh/aleksey925/pydantic-settings-vault) [](https://pypi.org/project/pydantic-settings-vault/)\n\npydantic-settings-vault\n=======================\n\n> `pydantic-settings-vault` is a fork `pydantic-vault` with `pydantic 2.x` support.\n\nA simple extension to [pydantic-settings][pydantic-basesettings] that can retrieve secrets stored in [Hashicorp Vault][vault].\n\nWith pydantic-settings and pydantic-settings-vault, you can easily declare your configuration in a type-hinted class, and load configuration\nfrom environment variables or Vault secrets. pydantic-settings-vault will work the same when developing locally (where you probably\nlogin with the Vault CLI and your own user account) and when deploying in production (using a Vault Approle or Kubernetes\nauthentication for example).\n\n<!-- toc -->\n\n- [Installation](#installation)\n- [Getting started](#getting-started)\n- [Documentation](#documentation)\n * [`Field` additional parameters](#field-additional-parameters)\n * [Configuration](#configuration)\n * [Authentication](#authentication)\n + [Approle](#approle)\n + [Kubernetes](#kubernetes)\n + [Vault token](#vault-token)\n + [JWT/OIDC](#jwtoidc)\n * [Order of priority](#order-of-priority)\n- [Logging](#logging)\n- [Examples](#examples)\n * [Retrieve a secret from a KV v2 secret engine](#retrieve-a-secret-from-a-kv-v2-secret-engine)\n * [Retrieve a whole secret at once](#retrieve-a-whole-secret-at-once)\n * [Retrieve a secret from a KV v1 secret engine](#retrieve-a-secret-from-a-kv-v1-secret-engine)\n * [Retrieve a secret from a database secret engine](#retrieve-a-secret-from-a-database-secret-engine)\n * [Use a dynamic path to retrieve secrets](#use-a-dynamic-path-to-retrieve-secrets)\n- [Known limitations](#known-limitations)\n- [Inspirations](#inspirations)\n- [License](#license)\n- [Development](#development)\n * [Debugging with a real Vault server](#debugging-with-a-real-vault-server)\n\n<!-- tocstop -->\n\n## Installation\n\n```shell\npip install pydantic-settings-vault\n\n# or if you use Poetry or Pipenv\npoetry add pydantic-settings-vault\npipenv install pydantic-settings-vault\n```\n\n## Getting started\n\nWith `pydantic_settings.BaseSettings` class, you can easily \"create a clearly-defined, type-hinted\napplication configuration class\" that gets its configuration from environment variables. It will work the same when \ndeveloping locally (where you probably login with the Vault CLI and your own user account) and when deploying in \nproduction (using a Vault Approle, Kubernetes or JWT/OIDC authentication for example).\n\nYou can create a normal `BaseSettings` class, and define the `settings_customise_sources()` method to load secrets from your Vault instance using the `VaultSettingsSource` class:\n\n```python\nimport os\n\nfrom pydantic import Field, SecretStr\nfrom pydantic_settings import BaseSettings, PydanticBaseSettingsSource\nfrom pydantic_vault import VaultSettingsSource\n\n\nclass Settings(BaseSettings):\n # The `vault_secret_path` is the full path (with mount point included) to the secret\n # The `vault_secret_key` is the specific key to extract from a secret\n username: str = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"secret/data/path/to/secret\",\n \"vault_secret_key\": \"my_user\",\n },\n )\n password: SecretStr = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"secret/data/path/to/secret\",\n \"vault_secret_key\": \"my_password\",\n },\n )\n\n model_config = {\n \"vault_url\": \"https://vault.tld\",\n \"vault_token\": os.environ[\"VAULT_TOKEN\"],\n \"vault_namespace\": \"your/namespace\", # Optional, pydantic-settings-vault supports Vault namespaces (for Vault Enterprise)\n }\n\n @classmethod\n def settings_customise_sources(\n cls,\n settings_cls: type[BaseSettings],\n init_settings: PydanticBaseSettingsSource,\n env_settings: PydanticBaseSettingsSource,\n dotenv_settings: PydanticBaseSettingsSource,\n file_secret_settings: PydanticBaseSettingsSource,\n ) -> tuple[PydanticBaseSettingsSource, ...]:\n # This is where you can choose which settings sources to use and their priority\n return (\n init_settings,\n env_settings,\n dotenv_settings,\n VaultSettingsSource(settings_cls),\n file_secret_settings,\n )\n\n\nsettings = Settings()\n# These variables will come from the Vault secret you configured\nsettings.username\nsettings.password.get_secret_value()\n\n\n# Now let's pretend we have already set the USERNAME in an environment variable\n# (see the Pydantic documentation for more information and to know how to configure it)\n# With the priority order we defined above, its value will override the Vault secret\nos.environ[\"USERNAME\"] = \"my user\"\n\nsettings = Settings()\nsettings.username # \"my user\", defined in the environment variable\nsettings.password.get_secret_value() # the value set in Vault\n```\n\n## Documentation\n\n### `Field` additional parameters\n\nYou might have noticed that we import `Field` directly from Pydantic. pydantic-settings-vault doesn't add any custom logic to it, which means you can still use everything you know and love from Pydantic.\n\nThe additional parameters pydantic-settings-vault uses are:\n\n| Parameter name | Required | Description |\n|-----------------------------|----------|-------------------------------------------------------------------------------------------------------------------------------------------------|\n| `vault_secret_path` | **Yes** | The path to your secret in Vault<br>This needs to be the *full path* to the secret, including its mount point (see [examples](#examples) below) |\n| `vault_secret_key` | No | The key to use in the secret<br>If it is not specified the whole secret content will be loaded as a dict (see [examples](#examples) below) |\n\nFor example, if you create a secret `database/prod` with a key `password` and a value of `a secret password` in a KV v2 secret engine mounted at the default `secret/` location, you would access it with\n\n```python\npassword: SecretStr = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"secret/data/database/prod\",\n \"vault_secret_key\": \"password\",\n },\n)\n```\n\n### Configuration\n\nYou can configure the behaviour of pydantic-settings-vault in your `Settings.model_config` dict, or using environment variables:\n\n| Settings name | Type | Required | Environment variable | Description |\n|---------------------------------|-----------------------|----------|--------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `settings_customise_sources()` | | **Yes** | N/A | You need to implement this function to use Vault as a settings source, and choose the priority order you want |\n| `vault_url` | `str` | **Yes** | `VAULT_ADDR` | Your Vault URL |\n| `vault_namespace` | `str \\| None` | No | `VAULT_NAMESPACE` | Your Vault namespace (if you use one, requires Vault Enterprise) |\n| `vault_auth_path` | `str \\| None` | No | `VAULT_AUTH_PATH` | The path of the authentication method, such as /auth/{path}/login, if different from its default, is only supported by the JWT authentication method. |\n| `vault_auth_mount_point` | `str \\| None` | No | `VAULT_AUTH_MOUNT_POINT` | The mount point of the authentication method, if different from its default mount point |\n| `vault_certificate_verify` | `str \\| bool \\| None` | No | `VAULT_CA_BUNDLE` | The path to a CA bundle validating your Vault certificate, or `False` to disable verification (see [hvac docs][hvac-private-ca]) |\n\nEnvironment variables override what has been defined in the `Config` class.\n\nYou can also configure everything available in the original Pydantic `BaseSettings` class.\n\n### Authentication\n\npydantic-settings-vault supports the following authentication method (in descending order of priority):\n - [direct token authentication][vault-auth-token]\n - [kubernetes][vault-auth-kubernetes]\n - [approle][vault-auth-approle]\n - [jwt/oidc][vault-auth-jwt-oidc]\n\npydantic-settings-vault tries to be transparent and help you work, both during local development and in production. It will try to\nfind the required information for the first authentication method, if it can't it goes on to the next method, until it\nhas exhausted all authentication methods. In this case it gives up and logs the failure.\n\nYou only need to know this order of priority if you specify the authentication parameters for multiple methods.\n\nSupport is planned for GKE authentication methods (contributions welcome! :wink:).\n\n#### Approle\n\nTo authenticate using the [Approle auth method][vault-auth-approle], you need to pass a role ID and a secret ID to your Settings class.\n\npydantic-settings-vault reads this information from the following sources (in descending order of priority):\n - the `VAULT_ROLE_ID` and `VAULT_SECRET_ID` environment variables\n - the `vault_role_id` and `vault_secret_id` configuration fields in your `Settings.model_config` dict (`vault_secret_id` can be a `str` or a `SecretStr`)\n\nYou can also mix-and-match, e.g. write the role ID in your `Settings.model_config` dict and retrieve the secret ID from the environment at runtime.\n\nExample:\n```python\nfrom pydantic import Field, SecretStr\nfrom pydantic_settings import BaseSettings, PydanticBaseSettingsSource\nfrom pydantic_vault import VaultSettingsSource\n\n\nclass Settings(BaseSettings):\n username: str = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"path/to/secret\",\n \"vault_secret_key\": \"my_user\",\n },\n )\n password: SecretStr = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"path/to/secret\",\n \"vault_secret_key\": \"my_password\",\n },\n )\n\n model_config = {\n \"vault_url\": \"https://vault.tld\",\n \"vault_role_id\": \"my-role-id\",\n \"vault_secret_id\": SecretStr(\"my-secret-id\"),\n }\n\n @classmethod\n def settings_customise_sources(\n cls,\n settings_cls: type[BaseSettings],\n init_settings: PydanticBaseSettingsSource,\n env_settings: PydanticBaseSettingsSource,\n dotenv_settings: PydanticBaseSettingsSource,\n file_secret_settings: PydanticBaseSettingsSource,\n ) -> tuple[PydanticBaseSettingsSource, ...]:\n return (\n init_settings,\n env_settings,\n dotenv_settings,\n VaultSettingsSource(settings_cls),\n file_secret_settings,\n )\n```\n\n#### Kubernetes\n\nTo authenticate using the [Kubernetes auth method][vault-auth-kubernetes], you need to pass a role to your Settings class.\n\npydantic-settings-vault reads this information from the following sources (in descending order of priority):\n - the `VAULT_KUBERNETES_ROLE` environment variable\n - the `vault_kubernetes_role` configuration field in your `Settings.model_config` dict, which must be a `str`\n\nThe Kubernetes service account token will be read from the file at `/var/run/secrets/kubernetes.io/serviceaccount/token`.\n\nExample:\n```python\nfrom pydantic import Field, SecretStr\nfrom pydantic_settings import BaseSettings, PydanticBaseSettingsSource\nfrom pydantic_vault import VaultSettingsSource\n\n\nclass Settings(BaseSettings):\n username: str = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"path/to/secret\",\n \"vault_secret_key\": \"my_user\",\n },\n )\n password: SecretStr = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"path/to/secret\",\n \"vault_secret_key\": \"my_password\",\n },\n )\n\n model_config = {\n \"vault_url\": \"https://vault.tld\",\n \"vault_kubernetes_role\": \"my-role\",\n }\n\n @classmethod\n def settings_customise_sources(\n cls,\n settings_cls: type[BaseSettings],\n init_settings: PydanticBaseSettingsSource,\n env_settings: PydanticBaseSettingsSource,\n dotenv_settings: PydanticBaseSettingsSource,\n file_secret_settings: PydanticBaseSettingsSource,\n ) -> tuple[PydanticBaseSettingsSource, ...]:\n return (\n init_settings,\n env_settings,\n dotenv_settings,\n VaultSettingsSource(settings_cls),\n file_secret_settings,\n )\n```\n\n#### Vault token\n\nTo authenticate using the [Token auth method][vault-auth-token], you need to pass a Vault token to your `Settings` class.\n\npydantic-settings-vault reads this token from the following sources (in descending order of priority):\n - the `VAULT_TOKEN` environment variable\n - the `~/.vault-token` file (so you can use the `vault` CLI to login locally, pydantic-settings-vault will transparently reuse its token)\n - the `vault_token` configuration field in your `Settings.model_config` dict, which can be a `str` or a `SecretStr`\n\nExample:\n```python\nfrom pydantic import Field, SecretStr\nfrom pydantic_settings import BaseSettings, PydanticBaseSettingsSource\nfrom pydantic_vault import VaultSettingsSource\n\n\nclass Settings(BaseSettings):\n username: str = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"path/to/secret\",\n \"vault_secret_key\": \"my_user\",\n },\n )\n password: SecretStr = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"path/to/secret\",\n \"vault_secret_key\": \"my_password\",\n },\n )\n\n model_config = {\n \"vault_url\": \"https://vault.tld\",\n \"vault_token\": SecretStr(\"my-secret-token\"),\n }\n\n @classmethod\n def settings_customise_sources(\n cls,\n settings_cls: type[BaseSettings],\n init_settings: PydanticBaseSettingsSource,\n env_settings: PydanticBaseSettingsSource,\n dotenv_settings: PydanticBaseSettingsSource,\n file_secret_settings: PydanticBaseSettingsSource,\n ) -> tuple[PydanticBaseSettingsSource, ...]:\n return (\n init_settings,\n env_settings,\n dotenv_settings,\n VaultSettingsSource(settings_cls),\n file_secret_settings,\n )\n```\n\n#### JWT/OIDC\n\nTo authenticate using the [JWT/OIDC method][vault-auth-jwt-oidc], you need to pass \na token role and a token itself to your Settings class.\n\npydantic-settings-vault reads this information from the following sources (in descending order of priority):\n\n- the `VAULT_JWT_ROLE` and `VAULT_JWT_TOKEN` environment variables\n- the `vault_jwt_role` and `vault_jwt_token` configuration fields in your \n `Settings.model_config` class (`vault_jwt_token` can be a `str` or a `SecretStr`)\n\nYou can also mix and match, for example, write the role in your `Settings.model_config` \nclass and retrieve the token from the environment at runtime.\n\nExample:\n```python\nfrom pydantic import Field, SecretStr\nfrom pydantic_settings import BaseSettings, PydanticBaseSettingsSource\nfrom pydantic_vault import VaultSettingsSource\n\n\nclass Settings(BaseSettings):\n username: str = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"path/to/secret\",\n \"vault_secret_key\": \"my_user\",\n },\n )\n password: SecretStr = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"path/to/secret\",\n \"vault_secret_key\": \"my_password\",\n },\n )\n\n model_config = {\n \"vault_url\": \"https://vault.tld\",\n \"vault_jwt_role\": \"my-role\",\n \"vault_jwt_token\": SecretStr(\"my-token\"),\n }\n\n @classmethod\n def settings_customise_sources(\n cls,\n settings_cls: type[BaseSettings],\n init_settings: PydanticBaseSettingsSource,\n env_settings: PydanticBaseSettingsSource,\n dotenv_settings: PydanticBaseSettingsSource,\n file_secret_settings: PydanticBaseSettingsSource,\n ) -> tuple[PydanticBaseSettingsSource, ...]:\n return (\n init_settings,\n env_settings,\n dotenv_settings,\n VaultSettingsSource(settings_cls),\n file_secret_settings,\n )\n```\n\n### Order of priority\n\nYou can customize settings sources and choose the order of priority you want.\n\nHere are some examples:\n```python\nfrom pydantic_settings import BaseSettings, PydanticBaseSettingsSource\nfrom pydantic_vault import VaultSettingsSource\n\n\nclass Settings(BaseSettings):\n \"\"\"\n In descending order of priority:\n - arguments passed to the `Settings` class initializer\n - environment variables\n - Vault variables\n - variables loaded from the secrets directory, such as Docker Secrets\n - the default field values for the `Settings` model\n \"\"\"\n\n @classmethod\n def settings_customise_sources(\n cls,\n settings_cls: type[BaseSettings],\n init_settings: PydanticBaseSettingsSource,\n env_settings: PydanticBaseSettingsSource,\n dotenv_settings: PydanticBaseSettingsSource,\n file_secret_settings: PydanticBaseSettingsSource,\n ) -> tuple[PydanticBaseSettingsSource, ...]:\n return (\n init_settings,\n env_settings,\n dotenv_settings,\n VaultSettingsSource(settings_cls),\n file_secret_settings,\n )\n\n\nclass Settings(BaseSettings):\n \"\"\"\n In descending order of priority:\n - Vault variables\n - environment variables\n - variables loaded from the secrets directory, such as Docker Secrets\n - the default field values for the `Settings` model\n Here we chose to remove the \"init arguments\" source,\n and move the Vault source up before the environment source\n \"\"\"\n\n @classmethod\n def settings_customise_sources(\n cls,\n settings_cls: type[BaseSettings],\n init_settings: PydanticBaseSettingsSource,\n env_settings: PydanticBaseSettingsSource,\n dotenv_settings: PydanticBaseSettingsSource,\n file_secret_settings: PydanticBaseSettingsSource,\n ) -> tuple[PydanticBaseSettingsSource, ...]:\n return (\n VaultSettingsSource(settings_cls),\n env_settings,\n dotenv_settings,\n file_secret_settings,\n )\n```\n\n## Logging\n\nThe library exports a logger called `pydantic-vault`.\n\nTo help debugging you can change the log level. A simple way to do that if you do not have a custom log setup is:\n```py\n# At the beginning of your main file or entrypoint\nimport logging\n\nlogging.basicConfig()\nlogging.getLogger(\"pydantic-vault\").setLevel(logging.DEBUG) # Change the log level here\n```\n\n## Examples\n\nAll examples use the following structure, so we will omit the imports and the `model_config` dict:\n```python\nfrom pydantic import Field\nfrom pydantic_settings import BaseSettings, PydanticBaseSettingsSource\nfrom pydantic_vault import VaultSettingsSource\n\n\nclass Settings(BaseSettings):\n ###############################################\n # THIS PART CHANGES IN THE DIFFERENT EXAMPLES #\n username: str = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"secret/data/path/to/secret\",\n \"vault_secret_key\": \"my_user\",\n },\n )\n ###############################################\n\n model_config = {\"vault_url\": \"https://vault.tld\"}\n\n @classmethod\n def settings_customise_sources(\n cls,\n settings_cls: type[BaseSettings],\n init_settings: PydanticBaseSettingsSource,\n env_settings: PydanticBaseSettingsSource,\n dotenv_settings: PydanticBaseSettingsSource,\n file_secret_settings: PydanticBaseSettingsSource,\n ) -> tuple[PydanticBaseSettingsSource, ...]:\n return (\n init_settings,\n env_settings,\n dotenv_settings,\n VaultSettingsSource(settings_cls),\n file_secret_settings,\n )\n```\n\n### Retrieve a secret from a KV v2 secret engine\n\nSuppose your secret is at `my-api/prod` and looks like this:\n```\nKey Value\n--- -----\nroot_user root\nroot_password a_v3ry_s3cur3_p4ssw0rd\n```\n\nYour settings class would be:\n```python\nclass Settings(BaseSettings):\n # The `vault_secret_path` is the full path (with mount point included) to the secret.\n # For a KV v2 secret engine, there is always a `data/` sub-path between the mount point and\n # the secret actual path, eg. if your mount point is `secret/` (the default) and your secret\n # path is `my-api/prod`, the full path to use is `secret/data/my-api/prod`.\n # The `vault_secret_key` is the specific key to extract from a secret.\n username: str = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"secret/data/my-api/prod\",\n \"vault_secret_key\": \"root_user\",\n },\n )\n password: SecretStr = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"secret/data/my-api/prod\",\n \"vault_secret_key\": \"root_password\",\n },\n )\n\n\nsettings = Settings()\n\nsettings.username # \"root\"\nsettings.password.get_secret_value() # \"a_v3ry_s3cur3_p4ssw0rd\"\n```\n\n### Retrieve a whole secret at once\n\nIf you omit the `vault_secret_key` parameter in your `Field`, pydantic-settings-vault will load\nthe whole secret in your class field.\n\nWith the same secret as before, located at `my-api/prod` and with this data:\n```\nKey Value\n--- -----\nroot_user root\nroot_password a_v3ry_s3cur3_p4ssw0rd\n```\n\nYou could use a settings class like this to retrieve everything in the secret:\n```python\nclass Settings(BaseSettings):\n # The `vault_secret_path` is the full path (with mount point included) to the secret.\n # For a KV v2 secret engine, there is always a `data/` sub-path between the mount point and\n # the secret actual path, eg. if your mount point is `secret/` (the default) and your secret\n # path is `my-api/prod`, the full path to use is `secret/data/my-api/prod`.\n # We don't pass a `vault_secret_key` here so that pydantic-settings-vault fetches all fields at once.\n credentials: dict = Field(\n ..., json_schema_extra={\"vault_secret_path\": \"secret/data/my-api/prod\"}\n )\n\n\nsettings = Settings()\nsettings.credentials # { \"root_user\": \"root\", \"root_password\": \"a_v3ry_s3cur3_p4ssw0rd\" }\n```\n\nYou can also use a Pydantic `BaseModel` class to parse and validate the incoming secret:\n```python\nclass Credentials(BaseModel):\n root_user: str\n root_password: SecretStr\n\n\nclass Settings(BaseSettings):\n # The `vault_secret_path` is the full path (with mount point included) to the secret.\n # For a KV v2 secret engine, there is always a `data/` sub-path between the mount point and\n # the secret actual path, eg. if your mount point is `secret/` (the default) and your secret\n # path is `my-api/prod`, the full path to use is `secret/data/my-api/prod`.\n # We don't pass a `vault_secret_key` here so that pydantic-settings-vault fetches all fields at once.\n credentials: Credentials = Field(\n ..., json_schema_extra={\"vault_secret_path\": \"secret/data/my-api/prod\"}\n )\n\n\nsettings = Settings()\nsettings.credentials.root_user # \"root\"\nsettings.credentials.root_password.get_secret_value() # \"a_v3ry_s3cur3_p4ssw0rd\"\n```\n\n### Retrieve a secret from a KV v1 secret engine\n\nSuppose your secret is at `my-api/prod` and looks like this:\n```\nKey Value\n--- -----\nroot_user root\nroot_password a_v3ry_s3cur3_p4ssw0rd\n```\n\nYour settings class would be:\n```python\nclass Settings(BaseSettings):\n # The `vault_secret_path` is the full path (with mount point included) to the secret.\n # For a KV v1 secret engine, the secret path is directly appended to the mount point,\n # eg. if your mount point is `kv/` (the default) and your secret path is `my-api/prod`,\n # the full path to use is `kv/my-api/prod` (unlike with KV v2 secret engines).\n # The `vault_secret_key` is the specific key to extract from a secret.\n username: str = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"kv/my-api/prod\",\n \"vault_secret_key\": \"root_user\",\n },\n )\n password: SecretStr = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": \"kv/my-api/prod\",\n \"vault_secret_key\": \"root_password\",\n },\n )\n\n\nsettings = Settings()\n\nsettings.username # \"root\"\nsettings.password.get_secret_value() # \"a_v3ry_s3cur3_p4ssw0rd\"\n```\n\n\u26a0 Beware of the [known limitations](#known-limitations) on KV v1 secrets!\n\n### Retrieve a secret from a database secret engine\n\nDatabase secrets can be \"dynamic\", generated by Vault every time you request access.\nBecause every call to Vault will create a new database account, you cannot store the username\nand password in two different fields in your settings class, or you would get the username of the\n*first* generated account and the password of the *second* account. This means that you must *not*\npass a `vault_secret_key`, so that pydantic-settings-vault retrieves the whole secret at once.\n\nYou can store the credentials in a dict or in a custom `BaseModel` class:\n```python\nclass DbCredentials(BaseModel):\n username: str\n password: SecretStr\n\n\nclass Settings(BaseSettings):\n # The `vault_secret_path` is the full path (with mount point included) to the secret.\n # For a database secret engine, the secret path is `<mount point>/creds/<role name>`.\n # For example if your mount point is `database/` (the default) and your role name is\n # `my-db-prod`, the full path to use is `database/creds/my-db-prod`. You will receive\n # `username` and `password` fields in response.\n # You must *not* pass a `vault_secret_key` so that pydantic-settings-vault fetches both fields at once.\n db_creds: DbCredentials = Field(\n ..., json_schema_extra={\"vault_secret_path\": \"database/creds/my-db-prod\"}\n )\n db_creds_in_dict: dict = Field(\n ..., json_schema_extra={\"vault_secret_path\": \"database/creds/my-db-prod\"}\n )\n\n\nsettings = Settings()\n\nsettings.db_creds.username # \"generated-username-1\"\nsettings.db_creds.password.get_secret_value() # \"generated-password-for-username-1\"\nsettings.db_creds_in_dict[\"username\"] # \"generated-username-2\"\nsettings.db_creds_in_dict[\"password\"] # \"generated-password-for-username-2\"\n```\n\n### Use a dynamic path to retrieve secrets\n\nIf you have different paths for your secrets (for example if you have different environments) you can use string formatting\nto dynamically generate the paths depending on an environment variable.\n\n```python\nimport os\n\n# You will need to specify the environment in an environment variable, but by\n# default it falls back to \"dev\"\nENV = os.getenv(\"ENV\", \"dev\")\n\n\nclass Settings(BaseSettings):\n # This will load different secrets depending on the value of the ENV environment variable\n username: str = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": f\"kv/my-api/{ENV}\",\n \"vault_secret_key\": \"root_user\",\n },\n )\n password: SecretStr = Field(\n ...,\n json_schema_extra={\n \"vault_secret_path\": f\"kv/my-api/{ENV}\",\n \"vault_secret_key\": \"root_password\",\n },\n )\n\n\nsettings = Settings()\n\nsettings.username # \"root\"\nsettings.password.get_secret_value() # \"a_v3ry_s3cur3_p4ssw0rd\"\n```\n\n## Known limitations\n\n- On KV v1 secret engines, if your secret has a `data` key and you do not specify a `vault_secret_key`\nto load the whole secret at once, pydantic-settings-vault will only load the content of the `data` key.\n For example, with a secret `kv/my-secret`\n ```\n Key Value\n --- -----\n user root\n password a_v3ry_s3cur3_p4ssw0rd\n data a very important piece of data\n ```\n and the settings class\n ```python\n class Settings(BaseSettings):\n my_secret: dict = Field(\n ..., json_schema_extra={\"vault_secret_path\": \"kv/my-secret\"}\n )\n ```\n pydantic-settings-vault will try to load only the `data` value (`a very important piece of data`) in\n `my_secret`, which will fail validation from Pydantic because it is not a dict.\n\n **Workaround:** Rename the `data` key in your secret \ud83d\ude05\n\n **Workaround:** Migrate to KV v2\n\n## Inspirations\n\n- [Ansible `hashi_vault` lookup plugin][ansible hashi_vault] for the API and some code\n- [Hashicorp's Vault GitHub Action][vault-action] for the API\n\n## License\n\npydantic-settings-vault is available under the [MIT license](./LICENSE).\n\n## Development\n\n### Debugging with a real Vault server\n\nYou can use a real Vault server to debug this project. To make this process\neasier, this project includes a `docker-compose.yml` file that can run a \nready-to-use Vault server.\n\nTo run the server and set it up, run the following commands:\n\n```shell\ndocker-compose up\nmake setup-vault\n```\n\nAfter that, you will have a Vault server running at `http://localhost:8200`, where you can authorize in two ways:\n\n- using the root token (which is `token`)\n- using the JWT method (role=jwt_role, token=[link](./configs/vault/jwt_token.txt))\n- using the AppRole method (the values of role_id and secret_id can be found in the logs of the `make setup-vault` command).\n\n[ansible hashi_vault]: https://docs.ansible.com/ansible/latest/collections/community/hashi_vault/hashi_vault_lookup.html\n[hvac-private-ca]: https://hvac.readthedocs.io/en/stable/advanced_usage.html#making-use-of-private-ca\n[pydantic]: https://docs.pydantic.dev/latest/\n[pydantic-basesettings]: https://docs.pydantic.dev/latest/usage/pydantic_settings/\n[pydantic-basesettings-customsource]: https://docs.pydantic.dev/latest/usage/pydantic_settings/#adding-sources\n[vault]: https://www.vaultproject.io/\n[vault-action]: https://github.com/hashicorp/vault-action\n[vault-auth-approle]: https://www.vaultproject.io/docs/auth/approle\n[vault-auth-kubernetes]: https://www.vaultproject.io/docs/auth/kubernetes\n[vault-auth-token]: https://www.vaultproject.io/docs/auth/token\n[vault-auth-jwt-oidc]: https://developer.hashicorp.com/vault/docs/auth/jwt\n[vault-kv-v2]: https://www.vaultproject.io/docs/secrets/kv/kv-v2/\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "A simple extension to pydantic-settings that can retrieve secrets from Hashicorp Vault",
"version": "2.1.0",
"project_urls": {
"Bug Tracker": "https://github.com/aleksey925/pydantic-settings-vault/issues",
"Changelog": "https://github.com/aleksey925/pydantic-settings-vault/blob/master/CHANGELOG.md",
"Documentation": "https://github.com/aleksey925/pydantic-settings-vault/",
"Homepage": "https://github.com/aleksey925/pydantic-settings-vault/",
"Repository": "https://github.com/aleksey925/pydantic-settings-vault/"
},
"split_keywords": [
"hashicorp",
" vault",
" hvac",
" pydantic"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "205cc67685c2dc807b33c68bbe71eac5c2c637fe0645b06557c0b2b8a3184e05",
"md5": "1af74b08e2bd576afb02ced8151c07ed",
"sha256": "548d5702d717d1c1626a7154a9bb1f4c73fc5cc6c8960af09eddd7fd422eadce"
},
"downloads": -1,
"filename": "pydantic_settings_vault-2.1.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1af74b08e2bd576afb02ced8151c07ed",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 12826,
"upload_time": "2024-04-13T11:08:15",
"upload_time_iso_8601": "2024-04-13T11:08:15.417796Z",
"url": "https://files.pythonhosted.org/packages/20/5c/c67685c2dc807b33c68bbe71eac5c2c637fe0645b06557c0b2b8a3184e05/pydantic_settings_vault-2.1.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "e9f5ed68fbdb8ff006fc2b6d64c6fb62eaa9550f1b47fa426ef6af03760f4a75",
"md5": "e76dc62de0dcdabc083ab95a9251650e",
"sha256": "08ef235014190e220d60512174b36f586840aa7086cc745a89680410ce243706"
},
"downloads": -1,
"filename": "pydantic_settings_vault-2.1.0.tar.gz",
"has_sig": false,
"md5_digest": "e76dc62de0dcdabc083ab95a9251650e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8",
"size": 17559,
"upload_time": "2024-04-13T11:08:16",
"upload_time_iso_8601": "2024-04-13T11:08:16.531956Z",
"url": "https://files.pythonhosted.org/packages/e9/f5/ed68fbdb8ff006fc2b6d64c6fb62eaa9550f1b47fa426ef6af03760f4a75/pydantic_settings_vault-2.1.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-13 11:08:16",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "aleksey925",
"github_project": "pydantic-settings-vault",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pydantic-settings-vault"
}
JSON
