.. image:: https://raw.githubusercontent.com/rmlibre/aiootp/main/logo.png
:target: https://raw.githubusercontent.com/rmlibre/aiootp/main/logo.png
:alt: aiootp python package logo
aiootp - Asynchronous pseudo one-time pad based crypto and anonymity library.
=============================================================================
``aiootp`` is an asynchronous library providing access to cryptographic
primatives and abstractions, transparently encrypted / decrypted file
I/O and databases, as well as powerful, pythonic utilities that
simplify data processing & cryptographic procedures in python code.
This library's online, salt reuse / misuse resistant, tweakable AEAD cipher, called
``Chunky2048``, is an implementation of the **pseudo one-time pad**. The
aim is to create a simple, standard, efficient implementation that's
indistinguishable from the unbreakable one-time pad cipher; to give
users and applications access to user-friendly cryptographic tools; and,
to increase the overall security, privacy, and anonymity on the web, and
in the digital world. Users will find ``aiootp`` to be easy to write,
easy to read, and fun.
Important Disclaimer
--------------------
``aiootp`` is experimental software that works with Python 3.6+.
It's a work in progress. The programming API could change with
future updates, and it isn't bug free. ``aiootp`` provides powerful
security tools and misc utilities that're designed to be
developer-friendly and privacy preserving.
As a security tool, ``aiootp`` needs to be tested and reviewed
extensively by the programming and cryptography communities to
ensure its implementations are sound. We provide no guarantees.
This software hasn't yet been audited by third-party security
professionals.
.. image:: https://img.shields.io/pypi/v/aiootp
:target: https://img.shields.io/pypi/v/aiootp
:alt: version
.. image:: https://img.shields.io/pypi/pyversions/aiootp?color=black
:target: https://img.shields.io/pypi/pyversions/aiootp?color=black
:alt: python-versions
.. image:: https://img.shields.io/badge/License-AGPL%20v3-red.svg
:target: https://img.shields.io/badge/License-AGPL%20v3-red.svg
:alt: license
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://img.shields.io/badge/code%20style-black-000000.svg
:alt: code-style
.. image:: https://github.com/rmlibre/aiootp/actions/workflows/linux-python-app.yml/badge.svg
:target: https://github.com/rmlibre/aiootp/actions/workflows/linux-python-app.yml/badge.svg
:alt: linux-build-status
.. image:: https://github.com/rmlibre/aiootp/actions/workflows/windows-python-app.yml/badge.svg
:target: https://github.com/rmlibre/aiootp/actions/workflows/windows-python-app.yml/badge.svg
:alt: windows-build-status
.. image:: https://github.com/rmlibre/aiootp/actions/workflows/macos-python-app.yml/badge.svg
:target: https://github.com/rmlibre/aiootp/actions/workflows/macos-python-app.yml/badge.svg
:alt: macos-build-status
Quick Install
-------------
.. code-block:: shell
$ sudo apt-get install python3-setuptools python3-pip
$ pip3 install --user --upgrade pip typing aiootp
Run Tests
---------
.. code-block:: shell
$ cd ~/aiootp/tests
$ coverage run --source aiootp -m pytest -vv test_aiootp.py
_`Table Of Contents`
--------------------
- `Transparently Encrypted Databases`_
a) `Ideal Initialization`_
b) `User Profiles`_
c) `Tags`_
d) `Metatags`_
e) `Basic Management`_
f) `Mirrors`_
g) `Public Cryptographic Functions`_
I. `Encrypt / Decrypt`_
II. `HMACs`_
- `Chunky2048 Cipher`_
a) `High-level Functions`_
b) `High-level Generators`_
- `Passcrypt`_
a) `Hashing & Verifying Passphrases`_
b) `Passcrypt Algorithm Overview`_
- `X25519 & Ed25519`_
a) `X25519`_
b) `Ed25519`_
- `Comprende`_
a) `Synchronous Generators`_
b) `Asynchronous Generators`_
- `Module Overview`_
- `FAQ`_
- `Changelog`_
- `Known Issues`_
_`Transparently Encrypted Databases` .............. `Table Of Contents`_
------------------------------------------------------------------------
The package's ``AsyncDatabase`` & ``Database`` classes are very powerful data persistence utilities. They automatically handle encryption & decryption of user data & metadata, providing a pythonic interface for storing & retrieving any bytes or JSON serializable objects. They're designed to seamlessly bring encrypted bytes at rest to users as dynamic objects in use.
_`Ideal Initialization` ........................... `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Make a new user key with a fast, cryptographically secure pseudo-random number generator. Then this strong 64-byte key can be used to create a database object.
.. code-block:: python
from aiootp import acsprng, AsyncDatabase
key = await acsprng()
db = await AsyncDatabase(key)
_`User Profiles` .................................. `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
With User Profiles, passphrases may be used instead to open a database. Often, passwords & passphrases contain very little entropy. So, they aren't recommended for that reason. However, profiles provide a succinct way to use passphrases more safely. They do this by deriving strong keys from low entropy user input using the memory/cpu hard passcrypt algorithm, & a secret salt which is automatically generated & stored on the user's filesystem.
.. code-block:: python
# Automatically convert any available user credentials into
# cryptographic tokens which help to safely open databases ->
db = await AsyncDatabase.agenerate_profile(
b"server-url.com", # Here an unlimited number of bytes-type
# arguments can be passed as additional
b"address@email.net", # optional credentials.
username=b"username",
passphrase=b"passphrase",
salt=b"optional salt keyword argument",
# Optional passcrypt configuration:
mb=256, # The memory cost in Mibibytes (MiB)
cpu=2, # The computational complexity & number of iterations
cores=8, # How many parallel processes passcrypt will utilize
)
_`Tags` ........................................... `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Data within databases are primarily organized by Tags. Tags are simply string labels, and the data stored under them can be any bytes or JSON serializable objects.
.. code-block:: python
async with db:
# Using bracketed assignment adds tags to the cache
db["tag"] = {"data": "can be any JSON serializable object"}
db["hobby"] = b"fash smasher"
db["bitcoin"] = "0bb6eee10d2f8f45f8a"
db["lawyer"] = {"#": "555-555-1000", "$": 13000.50}
db["safehouses"] = ["Dublin Forgery", "NY Insurrection"]
# Changes in the cache are saved to disk when the context closes.
# View an instance's tags ->
db.tags
>>> {'tag', 'hobby', 'bitcoin', 'lawyer', 'safehouses'}
# View the filenames that locate the data for each tag ->
db.filenames
>>> {'0z0l10btu_yd-n4quc8tsj9baqu8xmrxz87ix',
'197ulmqmxg15lebm26zaahpqnabwr8sprojuh',
'248piaop3j9tmcvqach60qk146mt5xu6kjc-u',
'2enwc3crove2cnrx7ks963d8_se25k6cdn6q9',
'5dm-60yspq8yhah4ywxcp52kztq_9toj0owm2'}
# There are various ways of working with tags ->
await db.aset_tag("new_tag", ["data", "goes", "here"]) # stored only in cache
await db.aquery_tag("new_tag") # reads from disk if not in the cache
>>> ['data', 'goes', 'here']
tag_path = db.path / await db.afilename("new_tag")
"new_tag" in db
>>> True
tag_path.is_file() # the tag is saved in the cache, not to disk yet
>>> False
await db.asave_tag("new_tag")
tag_path.is_file() # now it's saved to disk
>>> True
# This removes the tag from cache, & any of its unsaved changes ->
await db.arollback_tag("new_tag")
# Or, the user can take the tag out of the database & the filesystem ->
await db.apop_tag("new_tag")
>>> ['data', 'goes', 'here']
"new_tag" in db
>>> False
tag_path.is_file()
>>> False
Access to data is open to the user, so care must be taken not to let external API calls touch the database without accounting for how that can go wrong.
_`Metatags` ....................................... `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Metatags are used to organize data by string names & domain separate cryptographic material. They are fully-fledged databases all on their own, with their own distinct key material too. They're accessible from the parent through an attribute that's added to the parent instance with the same name as the metatag. When the parent is saved, or deleted, then their descendants are also.
.. code-block:: python
# Create a metatag database ->
molly = await db.ametatag("molly")
# They can contain their own sets of tags (and metatags) ->
molly["hobbies"] = ["skipping", "punching"]
molly["hobbies"].append("reading")
# The returned metatag & the reference in the parent are the same ->
assert molly["hobbies"] is db.molly["hobbies"]
assert isinstance(molly, AsyncDatabase)
# All of an instance's metatags are viewable ->
db.metatags
>>> {'molly'}
# Delete a metatag from an instance ->
await db.adelete_metatag("molly")
db.metatags
>>> set()
assert not hasattr(db, "molly")
_`Basic Management` ............................... `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
There's a few settings & public methods on databases for users to manage their instances & data. This includes general utilities for saving & deleting databases to & from the filesystem, as well as fine-grained controls for how data is handled.
.. code-block:: python
# The path attribute is set within the instance's __init__
# using a keyword-only argument. It's the directory where the
# instance will store all of its files.
db.path
>>> PosixPath('site-packages/aiootp/aiootp/databases')
# Write database changes to disk with transparent encryption ->
await db.asave_database()
# Entering the instance's context also saves data to disk ->
async with db:
print("Saving to disk...")
# Delete a database from the filesystem ->
await db.adelete_database()
As databases grow in the number of tags, metatags & the size of data within, it becomes desireable to load data from them as needed, instead of all at once into the cache during initialization. This is why the ``preload`` boolean keyword-only argument is set to ``False`` by default.
.. code-block:: python
# Let's create some test values to show the impact preloading has ->
async with (await AsyncDatabase(key, preload=True)) as db:
db["favorite_foods"] = ["justice", "community"]
await db.ametatag("exercise_routines")
db.exercise_routines["gardening"] = {"days": ["moday", "wednesday"]}
db.exercise_routines["swimming"] = {"days": ["thursday", "saturday"]}
# Again, preloading into the cache is toggled off by default ->
uncached_db = await AsyncDatabase(key)
# To retrieve elements, ``aquery_tag`` isn't necessary when
# preloading is used, since the tag is already in the cache ->
async with uncached_db:
db["favorite_foods"]
>>> ["justice", "community"]
uncached_db["favorite_foods"]
>>> None
value = await uncached_db.aquery_tag("favorite_foods", cache=True)
assert value == ["justice", "community"]
assert uncached_db["favorite_foods"] == ["justice", "community"]
# Metatags will be loaded, but their tags won't be ->
assert type(uncached_db.exercise_routines) == AsyncDatabase
uncached_db.exercise_routines["gardening"]
>>> None
await uncached_db.exercise_routines.aquery_tag("gardening", cache=True)
>>> {"days": ["moday", "wednesday"]}
uncached_db.exercise_routines["gardening"]
>>> {"days": ["moday", "wednesday"]}
# But, tags can also be queried without caching their values,
value = await uncached_db.exercise_routines.aquery_tag("swimming")
value
>>> {"days": ["thursday", "saturday"]}
uncached_db.exercise_routines["swimming"]
>>> None
# However, changes to mutable values won't be transmitted to the
# database if they aren't retrieved from the cache ->
value["days"].append("sunday")
value
>>> {"days": ["thursday", "saturday", "sunday"]}
await uncached_db.exercise_routines.aquery_tag("swimming")
>>> {"days": ["thursday", "saturday"]}
_`Mirrors` ........................................ `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Database mirrors allow users to make copies of all files within a database under new encryption keys. This is useful if users simply want to make backups, or if they'd like to update / change their database keys.
.. code-block:: python
# A unique login key / credentials are needed to create a new
# database ->
new_key = await acsprng()
new_db = await AsyncDatabase(new_key)
# Mirroring an existing database is done like this ->
await new_db.amirror_database(db)
assert (
await new_db.aquery_tag("favorite_foods")
is await db.aquery_tag("favorite_foods")
)
# If the user is just updating their database keys, then the old
# database should be deleted ->
await db.adelete_database()
# Now, the new database can be saved to disk & given an appropriate
# name ->
async with new_db as db:
pass
_`Public Cryptographic Functions` ................. `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Although databases handle encryption & decryption automatically, users may want to utilize their databases' keys to do custom cryptographic procedures manually. There are a few public functions available to users if they should want such functionality.
_`Encrypt / Decrypt` .............................. `Table Of Contents`_
************************************************************************
.. code-block:: python
# Either JSON serializable or bytes-type data can be encrypted ->
json_plaintext = {"some": "JSON data can go here..."}
bytes_plaintext = b"some bytes plaintext goes here..."
token_plaintext = b"some token data goes here..."
json_ciphertext = await db.ajson_encrypt(json_plaintext)
bytes_ciphertext = await db.abytes_encrypt(bytes_plaintext)
token_ciphertext = await db.amake_token(token_plaintext)
# Those values can just as easily be decrypted ->
assert json_plaintext == await db.ajson_decrypt(json_ciphertext)
assert bytes_plaintext == await db.abytes_decrypt(bytes_ciphertext)
assert token_plaintext == await db.aread_token(token_ciphertext)
# Filenames may be added to classify ciphertexts. They also alter the
# key material used during encryption in such a way, that without the
# correct filename, the data cannot be decrypted ->
filename = "grocery-list"
groceries = ["carrots", "taytoes", "rice", "beans"]
ciphertext = await db.ajson_encrypt(groceries, filename=filename)
assert groceries == await db.ajson_decrypt(ciphertext, filename=filename)
await db.ajson_decrypt(ciphertext, filename="wrong filename")
>>> "InvalidSHMAC: Invalid StreamHMAC hash for the given ciphertext."
# Time-based expiration of ciphertexts is also available for all
# encrypted data this package produces ->
from aiootp.asynchs import asleep
await asleep(6)
await db.ajson_decrypt(json_ciphertext, ttl=1)
>>> "TimestampExpired: Timestamp expired by <5> seconds."
await db.abytes_decrypt(bytes_ciphertext, ttl=1)
>>> "TimestampExpired: Timestamp expired by <5> seconds."
await db.aread_token(token_ciphertext, ttl=1)
>>> "TimestampExpired: Timestamp expired by <5> seconds."
# The number of seconds that are exceeded may be helpful to know. In
# which case, this is how to retrieve that integer value ->
try:
await db.abytes_decrypt(bytes_ciphertext, ttl=1)
except db.TimestampExpired as error:
assert error.expired_by == 5
_`HMACs` .......................................... `Table Of Contents`_
************************************************************************
Besides encryption & decryption, databases can also be used to manually verify the authenticity of bytes-type data with HMACs.
.. code-block:: python
# Creating an HMAC of some data with a database is done this way ->
data = b"validate this data!"
hmac = await db.amake_hmac(data)
await db.atest_hmac(hmac, data) # Runs without incident
# Data that is not the same will be caught ->
altered_data = b"valiZate this data!"
await db.atest_hmac(hmac, altered_data)
>>> "InvalidHMAC: Invalid HMAC hash for the given data."
# Any number of bytes-type arguments can be run thorugh the function,
# the collection of items is canonically encoded automagically ->
arbitrary_data = (b"uid_\x0f\x12", b"session_id_\xa1")
hmac = await db.amake_hmac(*arbitrary_data)
await db.atest_hmac(hmac, *arbitrary_data) # Runs without incident
# Additional qualifying information can be specified with the ``aad``
# keyword argument ->
from time import time
timestamp = int(time()).to_bytes(8, "big")
hmac = await db.amake_hmac(*arbitrary_data, aad=timestamp)
await db.atest_hmac(hmac, *arbitrary_data)
>>> "InvalidHMAC: Invalid HMAC hash for the given data."
await db.atest_hmac(hmac, *arbitrary_data, aad=timestamp) # Runs fine
# This is most helpful for domain separation of the HMAC outputs.
# Each distinct setting & purpose of the HMAC should be specified
# & NEVER MIXED ->
uuid = await db.amake_hmac(user_name, aad=b"uuid")
hmac = await db.amake_hmac(user_data, aad=b"data-authentication")
#
_`Chunky2048 Cipher` .............................. `Table Of Contents`_
------------------------------------------------------------------------
The ``Chunky2048`` cipher is the built from generators & SHA3-based key-derivation functions. It's designed to be easy to use, difficult to misuse & future-proof with large security margins.
_`High-level Functions` .......................... `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
These premade recipes allow for the easiest usage of the cipher.
.. code-block:: python
import aiootp
cipher = aiootp.Chunky2048(key)
# Symmetric encryption of JSON data ->
json_data = {"account": 33817, "names": ["queen b"], "id": None}
encrypted_json_data = cipher.json_encrypt(json_data, aad=b"demo")
decrypted_json_data = cipher.json_decrypt(
encrypted_json_data, aad=b"demo", ttl=120
)
assert decrypted_json_data == json_data
# Symmetric encryption of binary data ->
binary_data = b"some plaintext data..."
encrypted_binary_data = cipher.bytes_encrypt(binary_data, aad=b"demo")
decrypted_binary_data = cipher.bytes_decrypt(
encrypted_binary_data, aad=b"demo", ttl=30
)
assert decrypted_binary_data == binary_data
# encrypted URL-safe Base64 encoded tokens ->
token_data = b"some plaintext token data..."
encrypted_token_data = cipher.make_token(token_data, aad=b"demo")
decrypted_token_data = cipher.read_token(
encrypted_token_data, aad=b"demo", ttl=3600
)
assert decrypted_token_data == token_data
_`High-level Generators` .......................... `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
With these generators, the online nature of the Chunky2048 cipher can be utilized. This means that any arbitrary amount of data can be processed in streams of controllable, buffered chunks. These streaming interfaces automatically handle message padding & depadding, ciphertext validation & detection of out-of-order message blocks.
Encryption:
.. code-block:: python
from aiootp import AsyncCipherStream
# Let's imagine we are serving some data over a network ->
receiver = SomeRemoteConnection(session).connect()
# This will manage encrypting a stream of data ->
stream = await AsyncCipherStream(key, aad=session.transcript)
# We'll have to send the salt & iv in some way ->
receiver.transmit(salt=stream.salt, iv=stream.iv)
# Now we can buffer the plaintext we are going to encrypt ->
for plaintext in receiver.upload.buffer(4 * stream.PACKETSIZE):
await stream.abuffer(plaintext)
# The stream will now produce encrypted blocks of ciphertext
# as well as the block ID which authenticates each block ->
async for block_id, ciphertext in stream:
# The receiver needs both the block ID & ciphertext ->
receiver.send_packet(block_id + ciphertext)
# Once done with buffering-in the plaintext, the ``afinalize``
# method is called so the remaining encrypted data will be
# flushed out of the buffer to the user ->
async for block_id, ciphertext in stream.afinalize():
receiver.send_packet(block_id + ciphertext)
# Here we can give an optional check of further authenticity,
# also cryptographically asserts the stream is finished ->
receiver.transmit(shmac=await stream.shmac.afinalize())
Decryption / Authentication:
.. code-block:: python
from aiootp import AsyncDecipherStream
# Here let's imagine we'll be downloading some data ->
source = SomeRemoteConnection(session).connect()
# The key, salt, aad & iv must be the same for both parties ->
stream = await AsyncDecipherStream(
key, salt=source.salt, aad=session.transcript, iv=source.iv
)
# The downloaded ciphertext will now be buffered & the stream
# object will produce the plaintext ->
for ciphertext in source.download.buffer(4 * stream.PACKETSIZE):
# Here stream.shmac.InvalidBlockID is raised if an invalid or
# out-of-order block is detected within the last 4 packets ->
await stream.abuffer(ciphertext)
# If authentication succeeds, the plaintext is produced ->
async for plaintext in stream:
yield plaintext
# After all the ciphertext is downloaded, ``afinalize`` is called
# to finish processing the stream & flush out the plaintext ->
async for plaintext in stream.afinalize():
yield plaintext
# An optional check for further authenticity which also
# cryptographically asserts the stream is finished ->
await stream.shmac.afinalize()
await stream.shmac.atest_shmac(source.shmac)
#
_`Passcrypt` .............................. `Table Of Contents`_
------------------------------------------------------------------------
The ``Passcrypt`` algorithm is a data independent memory & computationally hard password-based key derivation function. It's built from a single primitive, the SHAKE-128 extendable output function from the SHA-3 family. Its resource costs are measured by three parameters: ``mb``, which represents an integer number of Mibibytes (MiB); ``cpu``, which is a linear integer measure of computational complexity & the number of iterations of the algorithm over the memory cache; and ``cores``, which is an integer which directly assigns the number of separate processes that will be pooled to complete the algorithm. The number of bytes of the output tag are decided by the integer ``tag_size`` parameter. And, the number of bytes of the automatically generated ``salt`` are decided by the integer ``salt_size`` parameter.
_`Hashing & Verifying Passphrases` .......................... `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
By far, the dominating measure of difficulty for ``Passcrypt`` is determined by the ``mb`` Mibibyte memory cost. It's recommended that increases to desired difficulty are first translated into higher ``mb`` values, where resource limitations of the machines executing the algorithm permit. If more difficulty is desired than can be obtained by increasing ``mb``, then increases to the ``cpu`` parameter should be used. The higher this parameter is the less likely an adversary is to benefit from expending less than the intended memory cost, & increases the execution time & complexity of the algorithm. The final option that should be considered, if still more difficulty is desired, is to lower the ``cores`` parallelization parameter, which will just cause each execution to take longer to complete.
.. code-block:: python
from aiootp import Passcrypt, hash_bytes
# The class accepts an optional (but recommended) static "pepper"
# which is applied as additional randomness to all hashes computed
# by the class. It's a secret random bytes value of any size that is
# expected to be stored somewhere inaccessible by the database which
# contains the hashed passphrases ->
with open(SECRET_PEPPER_PATH, "rb") as pepper_file:
Passcrypt.PEPPER = pepper_file.read()
# when preparing to hash passphrases, it's a good idea to use any &
# all of the static data / credentials available which are specific
# to the context of the registration ->
APPLICATION = b"my-application-name"
PRODUCT = b"the-product-being-accessed-by-this-registration"
STATIC_CONTEXT = [APPLICATION, PRODUCT, PUBLIC_CERTIFICATE]
# If the same difficulty settings are going to be used for every
# hash, then a ``Passcrypt`` instance can be initialized to
# automatically pass those static settings ->
pcrypt = Passcrypt(mb=1024, cpu=2, cores=8) # 1 GiB, 8 cores
# Now that the static credentials / settings are ready to go, we
# can start hashing any user information that arrives ->
username = form["username"].encode()
passphrase = form["passphrase"].encode()
email_address = form["email_address"].encode()
# The ``hash_bytes`` function can then be used to automatically
# encode then hash the multi-input data so as to prevent the chance
# of canonicalization (&/or length extension) attacks ->
aad = hash_bytes(*STATIC_CONTEXT, username, email_address)
hashed_passphrase = pcrypt.hash_passphrase(passphrase, aad=aad)
assert type(hashed_passphrase) is bytes
assert len(hashed_passphrase) == 38
# Later, a hashed passphrase can be used to authenticate a user ->
untrusted_username = form["username"].encode()
untrusted_passphrase = form["passphrase"].encode()
untrusted_email_address = form["email_address"].encode()
aad = hash_bytes(
*STATIC_CONTEXT, untrusted_username, untrusted_email_address
)
try:
pcrypt.verify(
hashed_passphrase, untrusted_passphrase, aad=aad, ttl=3600
)
except pcrypt.InvalidPassphrase as auth_fail:
# If the passphrase does not hash to the same value as the
# stored hash, then this exception is raised & can be handled
# by the application ->
app.post_mortem(error=auth_fail)
except pcrypt.TimestampExpired as registration_expired:
# If the timestamp on the stored hash was created more than
# ``ttl`` seconds before the current time, then this exception
# is raised. This is helpful for automating registrations which
# expire after a certain amount of time, which in this case was
# 1 hour ->
app.post_mortem(error=registration_expired)
else:
# If no exception was raised, then the user has been authenticated
# by their passphrase, username, email address & the context of
# the registration ->
app.login_user(username, email_address)
#
_`Passcrypt Algorithm Overview` .......................... `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
By being secret-independent, ``Passcrypt`` is resistant to side-channel attacks. This implementation is also written in pure python. Significant attention was paid to design the algorithm so as to suffer minimally from the performance inefficiencies of python, since doing so would help to equalize the cost of computation between regular users & dedicated attackers with custom hardware / software. Below is a diagram that depicts how an example execution works:
.. code-block:: python
#
___________________ # of rows ___________________
| |
| initial memory cache |
| row # of columns == 2 * max([1, cpu // 2]) |
| | # of rows == ⌈1024*1024*mb/168*columns⌉ |
v v v
column|---'-----------------------------------------'---| the initial cache
column|---'-----------------------------------------'---| of size ~`mb` is
column|---'-----------------------------------------'---| built very quickly
column|---'-----------------------------------------'---| using SHAKE-128.
column|---'-----------------------------------------'---| each (row, column)
column|---'-----------------------------------------'---| coordinate holds
column|---'-----------------------------------------'---| one element of
column|---'-----------------------------------------'---| 168-bytes.
^
|
reflection row
<- |
|--------------------'-------'--------------------| each row is
|--------------------'-------'--------------------| hashed then has
|--------------------'-------'--------------------| a new 168-byte
|--------------------'-------'--------------------| digest overwrite
|--------------------'-------'--------------------| the current pointer
|--------------------'-------'--------------------| in an alternating
|--------------------Xxxxxxxx'xxxxxxxxxxxxxxxxxxxx| sequence, first at
|oooooooooooooooooooo'oooooooO--------------------| the index, then at
| -> its reflection.
index
|--'-------------------------------------------'--| this continues
|--'-------------------------------------------'--| until the entire
|--'-------------------------------------------Xxx| cache has been
|ooO-------------------------------------------'--| overwritten.
|xx'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'xx| a single `shake_128`
|oo'ooooooooooooooooooooooooooooooooooooooooooo'oo| object (H) is used
|xx'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'xx| to do all of the
|oo'ooooooooooooooooooooooooooooooooooooooooooo'oo| hashing.
| -> <- |
index reflection
|xxxxxxxxxxx'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| finally, the whole
|ooooooooooo'ooooooooooooooooooooooooooooooooooooo| cache is quickly
|xxxxxxxxxxx'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| hashed `cpu` + 2
|ooooooooooo'ooooooooooooooooooooooooooooooooooooo| number of times.
|Fxxxxxxxxxx'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| after each pass an
|foooooooooo'ooooooooooooooooooooooooooooooooooooo| 84-byte digest is
|fxxxxxxxxxx'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| inserted into the
|foooooooooo'ooooooooooooooooooooooooooooooooooooo| cache, ruling out
| -> hashing state cycles.
| hash cpu + 2 # of times Then a `tag_size`-
v byte tag is output.
H(cache)
tag = H.digest(tag_size)
#
_`X25519 & Ed25519` ............................... `Table Of Contents`_
------------------------------------------------------------------------
Asymmetric curve 25519 tools are available from these high-level interfaces over the ``cryptography`` package.
_`X25519` ......................................... `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Elliptic curve 25519 diffie-hellman exchange protocols.
.. code-block:: python
from aiootp import X25519, DomainKDF, GUID, Domains
# Basic Elliptic Curve Diffie-Hellman ->
guid = GUID().new()
my_ecdhe_key = X25519().generate()
yield guid, my_ecdhe_key.public_bytes # send this to Bob
raw_shared_secret = my_ecdhe_key.exchange(bobs_public_key)
shared_kdf = DomainKDF( # Use this to create secret shared keys
Domains.ECDHE,
guid,
bobs_public_key,
my_ecdhe_key.public_bytes,
key=raw_shared_secret,
)
# Triple ECDH Key Exchange client initialization ->
with ecdhe_key.dh3_client() as exchange:
response = internet.post(exchange())
exchange(response)
clients_kdf = exchange.result()
# Triple ECDH Key Exchange for a receiving peer ->
identity_key, ephemeral_key = client_public_keys = internet.receive()
server = ecdhe_key.dh3_server(identity_key, ephemeral_key)
with server as exchange:
internet.post(exchange.exhaust())
servers_kdf = exchange.result()
# Success! Now both the client & server peers share an identical
# ``DomainKDF`` hashing object to create shared keys ->
assert (
clients_kdf.sha3_512(context=b"test")
== servers_kdf.sha3_512(context=b"test")
)
_`Ed25519` ........................................ `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Edwards curve 25519 signing & verification.
.. code-block:: python
from aiootp import Ed25519
# In a land, long ago ->
alices_key = Ed25519().generate()
internet.send(alices_key.public_bytes)
# Alice wants to sign a document so that Bob can prove she wrote it.
# So, Alice sends the public key bytes of the key she wants to
# associate with her identity, the document & the signature ->
document = b"DesignDocument.cad"
signed_document = alices_key.sign(document)
message = {
"document": document,
"signature": signed_document,
"public_key": alices_key.public_bytes,
}
internet.send(message)
# In a land far away ->
alices_message = internet.receive()
# Bob sees the message from Alice! Bob already knows Alice's public
# key & she has reason believe it is genuinely Alice's. So, she'll
# import Alice's known public key to verify the signed document ->
assert alices_message["public_key"] == alices_public_key
alice_verifier = Ed25519().import_public_key(alices_public_key)
alice_verifier.verify(
alices_message["signature"], alices_message["document"]
)
internet.send(b"Beautiful work, Alice! Thanks ^u^")
The verification didn't throw an exception! So, Bob knows the file was signed by Alice.
_`Comprende` ...................................... `Table Of Contents`_
------------------------------------------------------------------------
This magic with generators is made simple with the ``comprehension`` decorator. It wraps them in ``Comprende`` objects with access to myriad data processing pipeline utilities right out of the box.
_`Synchronous Generators` ......................... `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. code-block:: python
from aiootp.gentools import comprehension
@comprehension()
def gen(x: int, y: int):
z = yield x + y
return x * y * z
# Drive the generator forward with a context manager ->
with gen(x=1, y=2) as example:
z = 5
# Calling the object will send ``None`` into the coroutine by default ->
sum_of_x_y = example()
assert sum_of_x_y == 3
# Passing ``z`` will send it into the coroutine, cause it to reach the
# return statement & exit the context manager ->
example(z)
# The result returned from the generator is now available ->
product_of_x_y_z = example.result()
assert product_of_x_y_z == 10
# Here's another example ->
@comprehension()
def one_byte_numbers():
for number in range(256):
yield number
# Chained ``Comprende`` generators are excellent inline data processors ->
base64_data = one_byte_numbers().int_to_bytes(1).to_base64().list()
# This converted each number to bytes then base64 encoded them into a list.
# We can wrap other iterables to add functionality to them ->
@comprehension()
def unpack(iterable):
for item in iterable:
yield item
# This example just hashes each output then yields them
for digest in unpack(base64_data).sha3_256():
print(digest)
_`Asynchronous Generators` ........................ `Table Of Contents`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Async ``Comprende`` coroutines have almost exactly the same interface as synchronous ones.
.. code-block:: python
from aiootp.asynchs import asleep
from aiootp.gentools import Comprende, comprehension
@comprehension()
async def gen(x: int, y: int):
# Because having a return statement in an async generator is a
# SyntaxError, the return value is expected to be passed into
# Comprende.ReturnValue, and then raised to propagate upstream.
# It's then available from the instance's ``aresult`` method ->
z = yield x + y
raise Comprende.ReturnValue(x * y * z)
# Drive the generator forward.
async with gen(x=1, y=2) as example:
z = 5
# Awaiting the ``__call__`` method will send ``None`` into the
# coroutine by default ->
sum_of_x_y = await example()
assert sum_of_x_y == 3
# Passing ``z`` will send it into the coroutine, cause it to reach the
# raise statement which will exit the context manager gracefully ->
await example(z)
# The result returned from the generator is now available ->
product_of_x_y_z = await example.aresult()
assert product_of_x_y_z == 10
# Let's see some other ways async generators mirror synchronous ones ->
@comprehension()
async def one_byte_numbers():
# It's probably a good idea to pass control to the event loop at
# least once or twice, even if async sleeping after each iteration
# may be excessive when no real work is being demanded by range(256).
# This consideration is more or less significant depending on the
# expectations placed on this generator by the calling code.
await asleep()
for number in range(256):
yield number
await asleep()
# This is asynchronous data processing ->
base64_data = await one_byte_numbers().aint_to_bytes(1).ato_base64().alist()
# This converted each number to bytes then base64 encoded them into a list.
# We can wrap other iterables to add asynchronous functionality to them ->
@comprehension()
async def unpack(iterable):
for item in iterable:
yield item
# Want only the first twenty results? ->
async for digest in unpack(base64_data).asha3_256()[:20]:
# Then you can slice the generator.
print(digest)
# Users can slice generators to receive more complex output rules, like:
# Getting every second result starting from the 4th result to the 50th ->
async for result in unpack(base64_data)[3:50:2]:
print(result)
# Although, negative slice numbers are not supported.
``Comprende`` generators have loads of tooling for users to explore. Play around with it and take a look at the other chainable generator methods in ``aiootp.Comprende.lazy_generators``.
_`Module Overview` ................................ `Table Of Contents`_
------------------------------------------------------------------------
Here's a quick overview of this package's modules:
.. code-block:: python
import aiootp
# Commonly used constants, datasets & functionality across all modules ->
aiootp.commons
# The basic utilities & abstractions of the package's architecture ->
aiootp.generics
# A collection of the package's generator utilities ->
aiootp.gentools
# This module is responsible for providing entropy to the package ->
aiootp.randoms
# The high & low level abstractions used to implement the Chunky2048 cipher ->
aiootp.ciphers
# The higher-level abstractions used to create / manage key material ->
aiootp.keygens
# Common system paths & the ``pathlib.Path`` utility ->
aiootp.paths
# Global async / concurrency functionalities & abstractions ->
aiootp.asynchs
#
_`FAQ` ............................................ `Table Of Contents`_
========================================================================
**Q: What is the one-time pad?**
A: It's a cipher which provides an information theoretic guarantee of confidentiality. It's typically thought to be too cumbersome a cipher for generalized application because it conveys strict, and well, cumbersome, requirements onto its users. The need for its keys to be at least as large as all the messages it's ever used to encrypt is one such requirement. Our goal is to design a cipher which immitates the one-time pad through clever algorithms, in such a way as to minimize its inconveniences & still provide some form of information theoretic confidentiality guarantees or, at a minimum, be able to make non-trivial statements about its security against even computationally unbounded adversaries. In this effort, we've built what we hope to be a candidate cipher, which we've called ``Chunky2048``.
**Q: How fast is this ``Chunky2048`` cipher?**
A: Well, because it relies on ``hashlib.shake_128`` hashing to build key material streams, it's rather efficient. It can process about 24 MB/s on a ~1.5 GHz core for both encrypting & decrypting. This is still slow relative to other stream ciphers, but this package is written in pure Python & without hardware optimizations. Using SHA3 ASICs, specific chipset instructions, or a lower-level language implementation, could make this algorithm competitively fast.
**Q: What size keys does the ``Chunky2048`` cipher use?**
A: It's been designed to work with any size of key >= 64 bytes.
**Q: What's up with the ``AsyncDatabase`` / ``Database``?**
A: The idea is to create an intuitive, pythonic interface to a transparently encrypted and decrypted persistence tool that also cryptographically obscures metadata. It's designed to persist raw bytes or JSON serializable data, which gives it native support for some of the most important basic python datatypes. It's still a work in progress, albeit a very nifty one.
**Q: Why are the modules transformed into ``Namespace`` objects?**
A: We overwrite our modules in this package to have a more fine-grained control over what part of the package's internal state is exposed to users & applications. The goal is make it more difficult for users to inadvertently jeopardize their security tools, & minimize the attack surface available to adversaries. The ``Namespace`` class also makes it easier to coordinate and decide the library's UI/UX across the package.
_`Changelog` ...................................... `Table Of Contents`_
========================================================================
Changes for version 0.22.0
---------------------------
(Major Rewrite: Backwards Incompatible)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Security Advisory:
^^^^^^^^^^^^^^^^^^
- The top-level ``(a)csprng`` functions were found to be unsafe in concurrent code, leading to the possibilty of producing identical outputs from distinct calls if run in quick succession from concurrently running threads & coroutines. The classification of this vulnerability is severe because: 1) users should be able to expect the output of a 64-byte cryptographically secure pseudo-random number generator to always produce unique outputs; and, 2) much of the package utilizes them to produce cryptographic material. This vulnerability does not effect users of the library which are not running it in multiple concurrent threads or coroutines. The vulnerability has been patched & all users are **highly encouraged** to upgrade to v0.22.0+.
Major Changes
^^^^^^^^^^^^^
- Support for python 3.6 was dropped. The package now supports python versions 3.7+.
- **Chunky2048**: A new version of the cipher has been developed which
implements algorithms & interfaces that offer improvements in multiple
regards: smaller size overhead of ciphertexts, faster execution time
for large messages & large keys, more robust salt reuse/misue resistance,
fewer aspects harming deniability & better domain separation.
Many of the changes are described here:
- The ``(a)bytes_keys`` generators were updated to use ``shake_128``-based KDF objects instead of ``sha3_512``, yielding 256-bytes on each iteration instead of 128, now requiring only a single iteration to produce a keystream key for each block, instead of two. This choice was made during the process of analyzing the use of the user's encryption `key` to seed the `seed_kdf` on each iteration. We wanted to stop doing that essentially, because it slowed down the cipher too much when used with large keys. And because it seems like a bad idea to use the same key repeatedly while also not incorporating the uniqueness or entropy from the message's `salt`, `siv` or `aad`.
But still, we somehow wanted to come up with an idea which could efficiently & continually extract entropy from the user `key` if it did happen to be large. An answer came in the form of expanding on an earlier implemented idea which used the key multiple times to create unique seeds during initialization. In this case, however, instead of creating unique seeds with the single `seed_kdf`, each of the three KDFs & the MAC object used by the cipher will be given the whole `key` once at initialization, with proper domain separation, & including the message `salt` & `aad` (The `siv` can't be used because its creation happens after initialization during encryption). This gives each of their (SHA3) 200-byte internal states independent access to the full entropy of the `key`.
Then, the problem was that, by using ``sha3_512`` internally, a maximum of 64-bytes of entropy could be communicated between KDFs at each round (and only 32-bytes from the ``StreamHMAC`` (`shmac`) object's ``sha3_256`` MAC). But the blocksize of each round is 256-bytes. So, the idea became to attempt to *communicate* more entropy between the KDFs & MAC each round than there exists possible messages in the message space of each round. It seems plausible, that by only assuming the independence of each of the KDFs / MAC & that they can indeed `efficiently pass entropy` to one another, that for large keys we could argue the relevant key space is that of the 800-byte internal state of the cipher at each round (which happens to be more than three times the size of the message space of each round). This is to say, we conjecture, that by `efficiently communicating more entropy` from *independent sources* than there exists *possible messages*, & in fact incorporating the entropy of *each message block* into the cipher's state at the start of *each round*, that the entropy of the internal keyspace is continually being refreshed in a way which is negligibly distinguishable from using a fresh random key each round the length of the blocksize. This seems like at least a feasible way to begin the argument that it is possible to meaningfully relate the information theoretic security of the one-time pad to a pseudo one-time pad in a measurable way.
`Efficiently Pass Entropy`: By this we mean, the rate of bits extracted from one state object, to the rate of bits of actual entropy absorbed by a receiveing state object, up to its XORable state size, being different by only a negligible amount. Here, we can conservatively assume the limit of this efficiency is the XORable state size, since we know that in the ideal setting, XORing `n` uniform random bits with an unknown message of <= `n` bits is perfectly hiding, which implies perfectly efficient conveyance of entropy. By using ``shake_128`` as each of the cipher's state objects, & its larger rate of 168-bytes, more than twice the number of bytes can be passed to & extracted from each, per round & per call to their internal `f` permutation, as compared with ``sha3_512``. `If they can efficiently pass entropy`, then any secret state exposed by the `left_kdf` or `right_kdf` in the creation of ciphertext, can then be efficiently displaced by the introduction of new entropy from the other state objects. This follows from the theory that a finite sized pool of entropy which is already maximally filled with entropy, cannot incorporate more entropy without fundamentally erasing internal information. From this we arrived at the new design for ``Chunky2048``. In this new design, the `shmac` feeds 168-bytes to the `seed_kdf`, the `seed_kdf` creates 336-bytes to feed 168-bytes each to the `left_kdf` & `right_kdf`, the `left_kdf` & `right_kdf` each produce 128-byte keys which XOR the 256-byte plaintext, then this ciphertext feeds the `shmac` & the cycle repeats.
More work needs to be done to formalize these definitions & analyze their properties. We would be grateful for any help from those with expertise in formal proofs of security in tearing apart this design as we move closer to the first stable release of the package.
- The ``SyntheticIV`` class' algorithm has been updated as a result of analyzing how we could improve the salt reuse / misuse resistance of the cipher without attesting to plaintext contents in the form of an `siv` attached to ciphertexts. This plaintext attestation worked counter to our goal of wanting to be able to say something non-trivial about the key-deniability of the cipher. It was noticed that the plaintext padding already incorporated an 8-byte timestamp (now reduced to 4-bytes) & 16-bytes of ephemeral randomness as part of the prepended inner-header, & that these values were not at all used to seed the cipher's state during decryption. Instead a keyed-hash was calculated over the first block of plaintext during encryption to create the 24-byte `siv`. But, this is actually `less effective` at producing salt reuse / misuse resistance than using the timestamp & ephemeral randomness directly in seeding the `seed_kdf`, because the timestamp is a unique & global counter that does not suffer from collisions. This understanding came while also trying to find a good use for the initial `primer_key` generated by the keystream generator when sending in the first obligatory `None` value. In the previous version it was used to initialize the `shmac`, but now that the `shmac` would be initialized directly with the user `key`, it was searching for a use. So the idea was to pair them.
The new 256-byte `primer_key` would be XORed with the 256-byte first block of plaintext to mask the inner-header. The unmasked inner-header & 148-bytes of the `shmac`'s digest will seed the keystream, & the freshly seeded keystream output would be truncated to XOR the part of the masked plaintext which doesn't include the inner-header. There's no need now to attach the `siv` to the ciphertext. Instead, during decryption, the decipher algorithm has access to the inner-header, because it has access to the `primer_key` & the masked inner-header. The actual plaintext contents of the first block are only accessible after unmasking the inner-header & seeding the keystream. This combination alone of protection from a timestamp & 16-bytes of randomness should give a salt reuse / misuse resistance of at least `~2 ^ 64 messages` **per second**!
However, even with this new scheme, it would still be problematic to repeat a combination of `key`, `salt` & `aad`, since it would leak the XORs of timestamp information. With all of this in mind, the new formulation would include a 16-byte `salt` & a newly introduced 16-byte `iv`, both of which are attached to ciphertexts. This is a header size reduction of 16-bytes, since prior `salt` & `siv` sizes were 24-bytes each. The difference between the `salt` & `iv` is that the `salt` is available for the user to choose, but the `iv` is **always** generated randomly. Since the `iv` isn't dependent on message data the way that the `siv` was, it too can now be incorporated into all of the state objects during initialization. The `iv` ensures that even if a `key`, `salt` & `aad` tuple repeats, the timestamp is still protected. Below is a diagram of the procedure:
.. code-block:: python
#
_____________________________________
| |
| Algorithm Diagram: Encryption |
|_____________________________________|
------------------------------------------------------------------ #
| inner-header | first block of plaintext | #
| timestamp | siv-key | | #
| 4-bytes | 16-bytes | 236-bytes | #
------------------------------------------------------------------ #
|---------------------- entire first block ------------------------| #
| #
| #
first 256-byte keystream key ----⊕ #
| #
| #
V #
masked plaintext block #
------------------------------------------------------------------ #
| masked inner-header | first block of masked plaintext | #
------------------------------------------------------------------ #
|----- the 236-byte masked plaintext -----| #
| #
| #
siv = inner-header + shmac.digest(148) | #
keystream(siv)[10:246] -----------------------⊕ #
| #
| #
V #
------------------------------------------------------------------ #
| masked inner-header | first block of ciphertext | #
------------------------------------------------------------------ #
_____________________________________
| |
| Algorithm Diagram: Decryption |
|_____________________________________|
------------------------------------------------------------------ #
| masked inner-header | first block of ciphertext | #
------------------------------------------------------------------ #
|---------------------- entire first block ------------------------| #
| #
| #
first 256-byte keystream key ----⊕ #
| #
| #
V #
unmasked ciphertext block #
------------------------------------------------------------------ #
| inner-header | first block of unmasked ciphertext | #
------------------------------------------------------------------ #
|--- the 236-byte unmasked ciphertext ----| #
| #
| #
siv = inner-header + shmac.digest(148) | #
keystream(siv)[10:246] -----------------------⊕ #
| #
| #
V #
------------------------------------------------------------------ #
| inner-header | first block of plaintext | #
| timestamp | siv-key | | #
| 4-bytes | 16-bytes | 236-bytes | #
------------------------------------------------------------------ #
#
- The ``Padding`` class has seen some changes. Firstly, the 8-byte timestamp in the inner-header was reduced to 4-bytes. Furthermore, to get the full 136 years out of the 4-byte timestamps, the epoch used to calculate them was changed to unix timestamp `1672531200` (Sun, 01 Jan 2023 00:00:00 UTC). This is the new default `0` date for the package's timestamps. This saves some space & aims to provided fewer bits of confirmable attestation & correlation in proof games which simulate attacks on the key-deniability of the cipher. To explain: the plaintext padding includes random padding. That padding is intended to leave an adversary which attempts to brute force a ciphertext's encryption `key`, even with unbounded computational resources, in a state where it cannot decide with better accuracy than random chance between the exponentially large number of keys which create the same `shmac` tag (the variable `keyspace` is much larger than the 32-byte tag) with their accompanying exponentially large number of `plausible` plaintexts (any `reasonable` plaintext with any variable length random padding between 16 & 272 bytes), & the actual user `key` & plaintext.
We also got rid of the use of a `padding_key` to indicate the end of a plaintext message. It used to be sliced off the `primer_key`, but the `primer_key` has a new use now. Also, the `padding_key` was another form of plaintext / key attestation harming deniability that we wanted to get rid of. Instead, a simpler method is now employed: The final byte of the final block of padded plaintext is a number which tells the decryptor exactly how many bytes of random padding were added to the plaintext to fill the block. This saves a lot of space, is simpler, minimizes unnecessary key attestation, & eliminates the need for the ``Padding`` class to know anything about user secrets in order to do the padding, which is an improvment all around.
- New ``(Async)CipherStream`` & ``(Async)DecipherStream`` classes were introduced which allow users to utilize the online nature of the ``Chunky2048`` cipher, ciphering & deciphering data in bufferable chunks, without needing to know about or instantiate all of the low-level classes. They automatically handle the required plaintext padding, ciphertext authentication, & detection of out-of-order message blocks. This greatly simplifies the safe usage of ``Chunky2048`` in online mode, provides robustness, & gets rid of the need for users to worry about the dangers of release of unverified plaintexts.
- The ``Passcrypt`` algorithm was redesigned to be data-independent, more efficiently acheive its security goals, & allow for more compact hashes which include its difficulty settings metadata. The `kb` parameter was changed to `mb`, & now measures Mibibytes (MiB). A new `cores` parallelization parameter was added, which indicates the number of parallel processes to use to complete the procedure. And the `cpu` parameter now measures the number of iterations over the memory cache that are done, as well as the computational complexity of the algorithm. ``Passcrypt`` now uses ``shake_128`` instead of ``sha3_512`` internally. This also allows for users to specify a ``tag_size`` number of bytes to produce as an output tag. A ``salt_size`` parameter can now also be supplied to the ``(a)hash_passphrase`` methods. The ``(a)hash_passphrase`` methods now produce raw-bytes outputs & the ``(a)hash_passphrase_raw`` & ``(a)verify_raw`` methods were removed. ``(a)verify`` methods now also accept ``range``-type objects as ``mb_allowed``, ``cpu_allowed``, & ``cores_allowed`` keyword argument inputs. These range objects can be used to specify the exact amount of resources which the user allows for difficulty settings, which can mitigate adversarial (or unintentional) DOS attacks on machines doing hash verification.
- Type annotations were added to most of the library, including return types, which were completely neglected in prior versions. They are still not functioning with mypy, & are serving right now as documentation & auto-complete helpers.
- Many unnecesssary, low-level or badly designed features, functions & classes were either deleted or pulled into private namespaces, along with major reorganization & cleanup of the codebase. The tangled mess of internal module imports was also cleaned up. The goal is to provide access to only the highest level, simplest, & safest by default interfaces which can actually help users in their data processing & cryptographic tasks. These changes aim to improve maintainability, readability, correctness & safety.
- New top-level ``(a)hash_bytes`` functions were added to the package, which accept an unlimited number bytes-type inputs as positional arguments & automatically canonically encode all inputs before being hashed (which aims to prevent canonicalization attacks & length-extension attacks). A ``key`` keyword-only argument can also be supplied to optionally produce keyed hashes.
- A new top-level ``GUID`` class was added. It creates objects which produce variable length, obfuscated, pseudo-random bytes-type globally unique identifiers based on a user-defined integer `node_number`, a user-defined uniform bytes `salt`, a nanosecond `timestamp`, random `entropy` bytes & a 1-byte `counter`. The benefits of its novel design explained: **1)** the namespace separation of user-defined salts (like name-based uuids); **2)** guaranteed output uniqueness for all instances using the same `salt` & `node_number` which occur on a different nanosecond (like time-based uuids, but with higher precision); **3)** guaranteed output uniqueness between all instances which use the same `salt` but a different `node_number`, even if produced on the same nanosecond; **4)** guaranteed output uniqueness for any unique instance using the same `salt` & `node_number` if it produces 256 or fewer outputs every nanosecond; **5)** probabilistic output uniqueness for any unique instance using the same `salt` & `node_number` if it produces >256 outputs per-nanosecond, exponentially proportional to the number of random `entropy` bytes (which in turn are proportional to the output size of the GUIDs); **6)** output invertability, meaning outputs can be unmasked & sorted according to `timestamp`, `node_number` & `counter`; **7)** random-appearing outputs, with the marginal amount of privacy which can be afforded by obfuscated affine-group operations. Admittedly, point **7)** still *leaves much room for improvement*, as the privacy of the design could instead be ensured by strong hardness assumptions given by other types of invertible permutations or group operations. The goal was to create something efficient (below 3µs per guid), which met the above criterion, & that produced output bit sequences which passed basic randomness tests. We'd be excited to accept pull requests which use strong invertable permutations or group operations that are also about as efficient, & that for `n`-byte declared output sizes, outputs do not repeat for fewer than ~256 ** `n` sequential input values.
- The top-level ``DomainKDF`` class now also creates KDF objects which automatically canonically encode all inputs.
- The ``X25519`` protocols now return ``DomainKDF`` results instead of plain ``sha3_512`` objects.
- The ``(Base)Comprende`` classes were greatly simplified, & the caching & ``messages`` features were removed.
- The top-level ``(a)mnemonic`` functions now return lists of bytes-type words, instead of str-type, & can now be used to quickly generate lists of randomly selected words without providing a (now optional) passphrase.
- The ``(Async)Database`` classes' ``(a)generate_profile`` methods no longer require tokens to first be created by the user. That is now handled internally, & the external API accepts raw bytes inputs for credentials from the user.
- The ``PackageSigner`` & ``PackageVerifier`` now use ``sha384`` for digests instead of ``sha512``. The verifier now by default recomputes & verifies the digests of files from the filesystem using the ``path`` keyword argument to the constructor as the root directory for the relative filepaths declared in the "checksums" entry of the signature summary.
Minor Changes
^^^^^^^^^^^^^
- A new ``Clock`` class was added to the ``generics.py`` module which provides a very intuitive API for handling time & timestamp functionalities for various time units.
- The test suite was reorganized, cleaned up & extended significantly, & now also utilizes ``pytest-asyncio`` to run async tests. This led to many found & fixed bugs in code that was not being tested. There's still a substantial amount of tests that need to be written. We would greatly appreciate contributions which extend our test coverage.
- Many improvements to the correctness, completeness & aesthetic beauty of the code documentation with the addition of visual aides, diagrams & usage examples.
- A top-level ``report_security_issue`` function was added, which provides a terminal application for users to automatically encrypt security reports to us using our new X25519 public key.
- We lost access to our signing keys in encrypted drives which were damaged in flooding. So we decided to shred them & start fresh. Our new Ed25519 signing key is "70d1740f2a439da98243c43a4d7ef1cf993b87a75f3bb0851ae79de675af5b3b". Contact us via email or twitter if you'd like to confirm that the key you are seeing is really ours.
Changes for version 0.21.1
--------------------------
Minor Changes
^^^^^^^^^^^^^
- Fix usage of the wrong package signing key.
Changes for version 0.21.0
--------------------------
Major Changes
^^^^^^^^^^^^^
- Non-backwards compatible changes:
- Altered the ``Chunky2048`` cipher's key derivation to continuously extract
entropy from users' main encryption key. The design goal of the cipher
is to be as close as possible to a one-time pad, but because we use
key derivations to mix together all the relevant values used by the
cipher, there's a limited amount of entropy that can be extracted
from the main key no matter how large it is. The changes feed the
main key into the internal seed KDF multiple times when creating the
cipher's initial seeds, & once on every iteration of the ``(a)bytes_keys``
generators.
- Merged two internal KDFs used by the cipher into the one seed KDF. This
also now means that using the ``(a)update_key`` methods of the ``StreamHMAC``
class updates the KDF used to ratchet the encryption keystream.
- Use ``sha3_512`` instead of ``sha3_256`` for the ``StreamHMAC`` final HMAC
& slice the first bytes designated by the package's ``commons.py`` module.
This allows the HMAC length to be specified & changed easily. It's
**highly discouraged** to use anything less than 32-bytes.
Minor Changes
^^^^^^^^^^^^^
- Internal refactorings.
- Updates to tests.
Changes for version 0.20.7
--------------------------
Major Changes
^^^^^^^^^^^^^
- Changed the way the ``Padding.(a)end_padding`` methods calculate the
required padding length. The change causes the methods to now assume
that the plaintext has already been prepended with the start padding.
- The various ``test_*`` & ``verify_*`` functions/methods throughout the
package have been changed to return ``None`` on successful validation
instead of ``True``, which more closely matches the convention for
exception-raising validators.
- The default ``block_id`` length was changed from 16-bytes to 24-bytes.
Minor Changes
^^^^^^^^^^^^^
- Make the ``(a)end_padding`` methods of the ``Padding`` class assume the
supplied data has already been prepended with the start padding. This
better integrates with streams of plaintext (online usage).
- Small internal refactorings.
- Documentation fixes.
Changes for version 0.20.6
--------------------------
Major Changes
^^^^^^^^^^^^^
- The ``(Async)Database`` classes now support storing raw ``bytes`` type
tag entries! This is a huge boon to time/space efficiency when needing
to store large binary files, since they don't need to be converted to
& from base64. This feature was made possible with only very minor
changes to the classes, & they're fully backwards-compatible! Older
versions will not be able handle raw ``bytes`` entries, but old JSON
serializable entries work the same way they did.
Minor Changes
^^^^^^^^^^^^^
- Docfixes.
- Small refactorings.
- Add new tests & make existing tests complete faster.
- Support empty strings to be passed to the ``(Async)Database`` constructors'
``directory`` kwarg, signifying the current directory. Now ``None`` is
the only falsey value which triggers the constructors to use the default
database directory.
- Fixed a bug in the ``AsyncDatabase`` class' ``aset_tag`` method, which
would throw an attribute error when passed the ``cache=False`` flag.
- Add Windows support to the CI tests.
Changes for version 0.20.5
--------------------------
Minor Changes
^^^^^^^^^^^^^
- Include the missing changelog entries for ``v0.20.4``.
Changes for version 0.20.4
--------------------------
Major Changes
^^^^^^^^^^^^^
- Add ``python3.10`` support by copying the ``async_lru`` package's main module
from their more up-to-date github repository instead of from PyPI.
Minor Changes
^^^^^^^^^^^^^
- Small refactorings & code cleanups.
- Documentation updates.
- Type-hinting updates.
- Cleanups to the package's module API.
- Improve CI & extend to ``python3.10``.
Changes for version 0.20.3
--------------------------
Minor Changes
^^^^^^^^^^^^^
- Small refactorings.
- Documentation updates.
- Type-hinting updates.
- Additional tests.
Changes for version 0.20.2
--------------------------
Major Changes
^^^^^^^^^^^^^
- Changed the ``Padding`` class' ``(a)check_timestamp`` methods to
``(a)test_timestamp``, to better match the naming convention in the
rest of the package.
- Removed the ``(a)sum_sha3__(256/512)`` chainable generator methods from
the ``Comprende`` class.
- Removed the ``os.urandom`` based functions in the ``randoms.py`` module.
Minor Changes
^^^^^^^^^^^^^
- Fixes & improvements to out of date documentation.
- Small fixes to type-hints.
- Small refactorings.
- Add ``(a)generate_key`` functions to the package & ``(Async)Keys`` classes.
- Fix some exception messages.
Changes for version 0.20.1
--------------------------
Minor Changes
^^^^^^^^^^^^^
- Small fixes & improvements to documentation.
- Small fixes & improvements to tests.
- Small fixes to type-hints.
- Small re-organization of source file contents.
- Small bug fixes.
Changes for version 0.20.0 (Backwards incompatible updates)
-----------------------------------------------------------
Major Changes
^^^^^^^^^^^^^
- The ``(a)json_(en/de)crypt`` & ``(a)bytes_(en/de)crypt`` functions &
methods now only expect to work with ``bytes`` type ciphertext. And,
the low-level cipher generators expect iterables of bytes where they
used to expect iterables of integers.
- The ``pid`` keyword-only argument throughout the package was changed
to ``aad`` to more clearly communicate its purpose as authenticated
additional data.
- The ``key``, ``salt`` & ``aad`` values throughout the package are now
expected to be ``bytes`` type values.
- The ``key`` must now be at least 32-bytes for use within the ``Chunky2048``
cipher & its interfaces.
- The ``salt``, for use in the ``Chunky2048`` cipher & its interfaces,
was decreased from needing to be 32-bytes to 24-bytes.
- The ``siv``, for use in the ``Chunky2048`` cipher & its interfaces, was
increased from needing to be 16-bytes to 24-bytes.
- The new ``KeyAADBundle`` class was created as the primary interface
for consuming ``key``, ``salt``, ``aad`` & ``siv`` values. This class'
objects are the only ones that are used to pass around these values
in low-level ``Chunky2048`` cipher functionalities. The higher-level
cipher functions are the only public interfaces that still receive
these ``key``, ``salt``, & ``aad`` values.
- The ``KeyAADBundle`` now manages the new initial key derivation of the
``Chunky2048`` cipher. This new algorithm is much more efficient,
utilizing the output of the keystream's first priming call instead of
throwing it away, removing the need for several other previously used
hashing calls.
- The ``bytes_keys`` & ``abytes_keys`` keystream generator algorithms
were improved & made more efficient. They also now only receive ``bytes``
type coroutine values or ``None``.
- The ``StreamHMAC`` algorithms were improved & made more efficient.
- The ``Chunky2048`` class now creates instance's that initialize, & who's
methods are callable, much more efficiently by reducing its previously
dynamic structure. Its now reasonable to use these instances in code
that has strict performance requirements.
- The ``Keys`` & ``AsyncKeys`` classes were trimmed of all instance
behaviour. They are now strictly namespaces which contain static or
class methods.
- All instance's of the word `password` throughout the package have been
replaced with the word `passphrase`. The ``Passcrypt`` class now only
accepts ``bytes`` type ``passphrase`` & ``salt`` values. The returned
hashes are also now always ``bytes``.
- The ``Padding`` & ``BytesIO`` classes' functionalities were made more
efficient & cleaned up their implementations.
- New ``PackageSigner`` & ``PackageVerifier`` classes were added to the
``keygens.py`` module to provide an intuituve API for users to sign their
own packages. This package now also uses these classes to sign itself.
- The new ``gentools.py`` module was created to organize the generator
utilities that were previously scattered throughout the package's
top-level namespaces.
- The new ``_exceptions.py`` module was created to help organize the
exceptions raised throughout the package, improving readability
& maintainability.
- The new ``_typing.py`` module was added to assist in the long process
of adding functional type-hinting throughout the package. For now,
the type hints that have been added primarily function as documentation.
- A new ``Slots`` base class was added to the ``commons.py`` module to
simplify the creation of more memory efficient & performant container
classes. The new ``_containers.py`` module was made for such classes
for use throughout the package. And, most classes throughout the
package were given ``__slots__`` attributes.
- A new ``OpenNamespace`` class was added, which is a subclass of ``Namespace``,
with the only difference being that instances do not omit attributes
from their repr's.
- The new ``(a)bytes_are_equal`` functions, which are pointers to
``hmac.compare_digest`` from the standard library, have replaced the
``(a)time_safe_equality`` functions.
- The ``(a)sha_256(_hmac)`` & ``(a)sha_512(_hmac)`` functions have had
their names changed to ``(a)sha3__256(_hmac)`` & ``(a)sha3__512(_hmac)``.
This was done to communicate that they are actually SHA3 functions,
but the double underscore is to keep them differentiable from the
standard library's ``hashlib`` objects. They can now also return
``bytes`` instead of hex strings if their ``hex`` keyword argument is truthy.
- The base functionality of the ``Comprende`` class was refactored out into a
``BaseComprende`` class. The chainable data processor generator methods
remain in the ``Comprende`` class. Their endpoint methods (such as ``(a)list``
& ``(a)join``) have also been changed so they don't cache results by default.
- The ``Passcrypt`` class' ``kb`` & ``hardness`` can now be set to values
independently from one another. The algorithm runs on the new
``(a)bytes_keys`` coroutines, & a slightly more effective cache building
procedure.
- The databases classes now don't preload their values by default. And,
various methods which work with tags & metatags have been given a
``cache`` keyword-only argument to toggle on/off the control of using
the cache for each operation.
- New method additions/changes to the database classes:
- ``(a)rollback_tag``, ``(a)clear_cache``, & a ``filenames`` property
were added.
- ``(a)hmac`` was changed to ``(a)make_hmac``, & now returns ``bytes`` hashes.
- ``(a)save`` was changed to ``(a)save_database``.
- ``(a)query`` was changed to ``(a)query_tag``.
- ``(a)set`` was changed to ``(a)set_tag``.
- ``(a)pop`` was changed to ``(a)pop_tag``.
- The ``tags``, ``metatags`` & ``filenames`` properties now return sets
instead of lists.
- The ``Ropake`` class has been removed from the package pending changes to
the protocol & its implementation.
- The ``(a)generate_salt`` function now returns ``bytes`` type values,
& takes a ``size`` keyword-only argument, with no default, that determines
the number of bytes returned between [8, 64].
- The ``(a)random_512`` & ``(a)random_256`` public functions can now cause
their underlying random number generators to fill their entropy pools
when either the ``rounds`` or ``refresh`` keyword arguments are specified.
- The following variables were removed from the package:
- ``(a)keys``, ``(a)passcrypt``, ``(a)seeder``, ``(a)time_safe_equality``,
``Datastream``, ``bits``, ``(a)seedrange``, ``(a)build_tree``,
``(a)customize_parameters``, ``convert_class_method_to_member``,
``convert_static_method_to_member``, ``(a)xor``, ``(a)padding_key``,
``(a)prime_table``, ``(a)unique_range_gen``, ``(a)non_0_digits``,
``(a)bytes_digits``, ``(a)digits``, ``(a)permute``, ``(a)shuffle``,
``(a)unshuffle``, ``(a)create_namespace``,
(``(a)depad_plaintext``, ``(a)pad_plaintext`` & their generator forms.
Only the non-generator forms remain in the ``Padding`` class), (The
``(a)passcrypt``, ``(a)uuids``, ``(a)into_namespace`` methods from the
database classes), (The ``(a)csprbg`` functions were removed & instead
the ``(a)csprng`` functions produce ``bytes`` type values.)
- Thorough & deep refactorings of modules, classes & methods. Many methods
& functions were made private, cleaning up the APIs of the package,
focusing on bringing the highest-level functionalities to top level
namespaces accessible to users. Some purely private functionalities
were entirely moved to private namespaces not readily accessible to
users.
- Most of the constants which determine the functionalities throughout
the package were refactored out into ``commons.py``. This allows
for easy changes to protocols & data formats.
Minor Changes
^^^^^^^^^^^^^
- Many documentation improvements, fixes, trimmings & updates.
- Added a ``WeakEntropy`` class to the ``randoms.py`` module.
Changes for version 0.19.4
--------------------------
Major Changes
^^^^^^^^^^^^^
- Created a private ``EntropyDaemon`` class to run a thread in the
background which feeds into & extracts entropy from some of the
package's entropy pools. Also moved the separate private ``_cache``
entropy pools from the parameters to the random number generators.
They're now a single private ``_pool`` shared global that's
asynchronously & continuously updated by the background daemon thread.
- Switched the ``random`` portion of function names in the ``randoms.py``
module to read ``unique`` instead. This was done to the functions which
are actually pseudo-random. This should give users a better idea of
which functions do what. The exception is that the ``random_sleep`` &
``arandom_sleep`` functions have kept their names even though they
sleep a pseudo-randomly variable amount of time. Their names may
cause more confusion if they were either ``(a)unique_sleep`` or
``(a)urandom_sleep``. Because they don't use ``os.urandom`` & what
is a ``unique_sleep``? When / if a better name is found these
function names will be updated as well.
Minor Changes
^^^^^^^^^^^^^
- Various docstring / documentation fixes & refactorings.
Changes for version 0.19.3
--------------------------
Major Changes
^^^^^^^^^^^^^
- Removed ``ascii_encipher``, ``ascii_decipher``, ``aascii_encipher`` &
``aascii_decipher`` generators from the ``Chunky2048`` & ``Comprende``
classes, & the package. It was unnecessary, didn't fit well with the
intended use of the ``Padding`` class, & users would be much better
served by converting their ascii to bytes to use the ``bytes_``
generators instead.
- Removed the ``map_encipher``, ``map_decipher``, ``amap_encipher`` &
``amap_decipher`` generators from the ``Chunky2048`` & ``Comprende``
classes, & the package. They were not being used internally to the
package anymore, & their functionality, security & efficiency could
not be guaranteed to track well with the changes in the rest of the
library.
- Added domain specificity to the ``X25519`` protocols' key derivations.
- Renamed the database classes' ``(a)encrypt`` & ``(a)decrypt`` methods
to ``(a)json_encrypt`` & ``(a)json_decrypt`` for clarity & consistency
with the rest of the package. Their signatures, as well as those in
``(a)bytes_encrypt`` & ``(a)bytes_decrypt``, were also altered to
receive plaintext & ciphertext as their only positional arguments.
The ``filename`` argument is now a keyword-only argument with a default
``None`` value. This allows databases to be used more succinctly for
manual encryption & decryption by making the filename tweak optional.
- The ``runs`` keyword argument for the functions in ``randoms.py`` was
renamed to ``rounds``. It seems more clear that it is controlling the
number of rounds are internally run within the ``(a)random_number_generator``
functions when deriving new entropy.
Minor Changes
^^^^^^^^^^^^^
- Fixes to docstrings & tutorials. Rewrite & reorganization of the
``PREADME.rst`` & ``README.rst``. More updates to the readme's are still
on the way.
- Slight fix to the Passcrypt docstring's algorithm diagram.
- Moved the default passcrypt settings to variables in the ``Passcrypt``
class.
- Added the ability to send passcrypt settings into the ``mnemonic`` &
``amnemonic`` coroutines, which call the algorithm internally but
previously could only use the default settings.
- Some code cleanups & refactorings.
Changes for version 0.19.2
--------------------------
Minor Changes
^^^^^^^^^^^^^
- Made the output lengths of the ``Padding`` class' generator functions
uniform. When the footer padding on a stream of plaintext needs to
exceed the 256-byte blocksize (i.e. when the last unpadded plaintext
block's length ``L`` is ``232 < L < 256``), then another full block of
padding is produced. The generators now yield 256-byte blocks
consistently (except during depadding when the last block of plaintext
may be smaller than the blocksize), instead of sometimes producing a
final padded block which is 512 bytes.
Changes for version 0.19.1
--------------------------
Minor Changes
^^^^^^^^^^^^^
- Fixed a bug where database classes were evaluating as falsey when they
didn't have any tags saved in them. They should be considered truthy
if they're instantiated & ready to store data, even if they're
currently empty & not saved to disk. This was reflected in their
``__bool__`` methods. The bug caused empty metatags not to be loaded
when an instance loads, even when ``preload`` is toggled ``True``.
- Removed the coroutine-receiving logic from the ``Padding`` class'
``Comprende`` generators. Since they buffer data, the received values
aren't ever going to coincide with the correct iteration & will be
susceptible to bugs
- Fixed a bug in the ``Padding`` class' ``Comprende`` generators which
cut iteration short because not enough data was available from the
underlying generators upfront. Now, if used correctly to pad/depad
chunks of plaintext 256 bytes at a time, then they work as expected.
- The ``update``, ``aupdate``, ``update_key`` & ``aupdate_key`` methods
in both the ``StreamHMAC`` & ``DomainKDF`` classes now return ``self``
to allow inline updates.
- Added ``acsprng`` & ``csprng`` function pointers to the ``Chunky2048``
class.
- Updates to docstrings which didn't get updated with info on the new
*synthetic IV* feature.
- Some other docstring fixes.
- Some small code cleanups & refactorings.
Changes for version 0.19.0
--------------------------
Major Changes
^^^^^^^^^^^^^
- Security Upgrade: The package's cipher was changed to an online,
authenticated scheme with salt reuse / misuse resistance. This was
acheived through a few backwards incompatible techniques:
1. A synthetic IV (SIV) is calculated from the keyed-hash of the first
256-byte block of plaintext. The SIV is then used to seed the
keystream generator, & is used to update the validator object. This
ensures that if the first block is unique, then the whole ciphertext
will be unique.
2. A 16-byte ephemeral & random SIV-key is also prepended to the
first block of plaintext during message padding. Since this value
is also hashed to derive the SIV, this key gives a strong
guarantee that a given message will produce a globally unique
ciphertext.
3. An 8-byte timestamp is prepended to the first block of plaintext
during padding. Timestamps are inherently sequential, they can be
verified by a user within some bounds, & can also be used to
mitigate replay attacks. Since it's hashed to make the SIV, then
it helps make the entire ciphertext unique.
4. After being updated with each block of ciphertext, the validator's
current state is again fed into the keystream generator as a new
rotating seed. This mitigation is limited to ensuring only that
every following block of ciphertext to a block which is unique
will also be unique. More specifically this means that: **if**
*all* **other mitigations fail to be unique**, or are missing, then
the first block which is unique **will appear the same**, except
for the bits which have changed, **but, all following blocks will
be randomized.** This limitation could be avoided with a linear
expansion in the ciphertext size by generating an SIV for each
block of plaintext. This linear expansion is prohibitive as a
default setting, but the block level secrecy, even when all other
mitigations fail, is enticing. This option may be added in the
future as a type of padding mode on the plaintext.
The SIV-key is by far the most important mitigation, as it isn't
feasibly forgeable by an adversary, & therefore also protects against
attacks using encryption oracles. These changes can be found in the
``SyntheticIV`` class, the (en/de)cipher & xor generators, & the
``StreamHMAC`` class in the ``ciphers.py`` module. The padding
changes can also be found in the new ``Padding`` class in the ``generics.py``
module. The SIV is attached in the clear with ciphertexts & was
designed to function with minimal user interaction. It needs only to
be passed into the ``StreamHMAC`` class during decryption -- during
encryption it's automatically generated & stored in the ``StreamHMAC``
validator object's ``siv`` property attribute.
- Security Patch: The internal ``sha3_512`` kdf's to the ``akeys``, ``keys``,
``abytes_keys`` & ``bytes_keys`` keystream generators are now updated
with 72 bytes of (64 key material + 8 padding), instead of just 64
bytes of key material. 72 bytes is the *bitrate* of the ``sha3_512``
object. This change causes the internal state of the object to be permuted
for each iteration update & before releasing a chunk of key material.
Frequency analysis of ciphertext bytes didn't smooth out to the
cumulative distribution expected for all large ciphertexts prior to
this change. But after the change the distribution does normalize as
expected. This indicates that the key material streams were biased
away from random in a small but measurable way. Although, no
particular byte values seem to have been preferred by this bias, this
is a huge shortcoming with unknown potential impact on the strength
of the package's cipher. This update is strongly recommended & is
backwards incompatible.
- This update gives a name to the package's pseudo-one-time-pad cipher
implementation. It's now called ``Chunky2048``! The ``OneTimePad``
class' name was updated to ``Chunky2048`` to match the change.
- The ``PreemptiveHMACValidation`` class & its related logic in the
``StreamHMAC`` class was removed. The chaining of validator output
into the keystream makes running the validator over the ciphertext
separately or prior to the decryption process very costly. It would
either mean recalculating the full hash of the ciphertext a second
time to reproduce the correct outputs during each block, or a large
linear memory increase to hold all of its digests to be fed in some
time after preemtive validation. It's much simpler to remove that
functionality & potentially replace it with something else that fits
the user's applications better. For instance, the ``current_digest``
& ``acurrent_digest`` methods can produce secure, 32-byte authentication
tags at any arbitrary blocks throughout the cipher's runtime, which
validate the cipehrtext up to that point. Or, the ``next_block_id``
& ``anext_block_id`` methods, which are a more robust option because
each id they produce validates the next ciphertext block before
updating the internal state of the validator. This acts as an
automatic message ordering algorithm, & leaves the deciphering
party's state unharmed by dropped packets or manipulated ciphertext.
- The ``update_key`` & ``aupdate_key`` methods were also added to the
``StreamHMAC`` class. They allow the user to update the validators'
internal key with new entropy or context information during its
runtime.
- The ``Comprende`` class now takes a ``chained`` keyword-only argument
which flags an instance as a chained generator. This flag allows
instances to communicate up & down their generator chain using the
shared ``Namespace`` object accessible by their ``messages`` attribute.
- The chainable ``Comprende`` generator functions had their internals
altered to allow them to receive, & pass down their chain, values
sent from a user using the standard coroutine ``send`` & ``asend``
method syntax.
- ``Comprende`` instances no longer automatically reset themselves every
time they enter their context managers or when they are iterated over.
This makes their interface more closely immitate the behavior of
async/sync generator objects. To get them to reset, the ``areset`` or
``reset`` methods must be used. The message chaining introduced in
this update allows chains of ``Comprende`` async/sync generators to
inform each other when the user instructs one of them to reset.
- The standard library's ``hmac`` module is now used internally to the
``generics.py`` module's ``sha_512_hmac``, ``sha_256_hmac``, ``asha_512_hmac``
& ``asha_256_hmac`` functions. They still allow any type of data to be
hashed, but also now default to hashing ``bytes`` type objects as
they are given.
- The new ``Domains`` class, found in ``generics.py``, is now used to
encode constants into deterministic pseudo-random 8-byte values for
helping turn hash function outputs into domain-specific hashes. Its
use was included throughout the library. This method has an added
benefit with respect to this package's usage of SHA-3. That being, the
*bitrate* for both ``sha3_512`` & ``sha3_256`` are ``(2 * 32 * k) + 8``
bytes, where ``k = 1`` for ``sha3_512`` & ``k = 2`` for ``sha3_256``.
This means that prepending an 8-byte domain string to their inputs
also makes it more efficient to add some multiple of key material
to make the input data precisely equal the *bitrate*. More info on
domain-specific hashing can be found here_.
.. _here: https://eprint.iacr.org/2020/241.pdf
- A new ``DomainsKDF`` class in ``cipehrs.py`` was added to create a
more standard & secure method of key derivation to the library which
also incorporates domain separation. Its use was integrated thoughout
the ``AsyncDatabase`` & ``Database`` classes to mitigate any further
vulnerabilities of their internal key-derivation functions. The
database classes now also use bytes-type keys internally, instead
of hex strings.
- The ``Passcrypt`` class now contains methods which create & validate
passcrypt hashes which have their settings & salt attached to them.
Instances can now also be created with persistent settings that are
automatically sent into instance methods.
Minor Changes
^^^^^^^^^^^^^
- Many fixes of docstrings, typos & tutorials.
- Many refactorings: name changes, extracted classes / functions,
reorderings & moves.
- Various code clean-ups, efficiency & usability improvements.
- Many constants used throughout the library were given names defined
in the ``commons.py`` module.
- Removed extraneous functions throughout the library.
- The asymmetric key generation & exchange functions/protocols were
moved from the ``ciphers.py`` module to ``keygens.py``.
- Add missing modules to the MANIFEST.rst file.
- Added a ``UniformPrimes`` class to the ``__datasets`` module for efficient
access to primes that aren't either mostly 1 or 0 bits, as is the case for
the ``primes`` helper table. These primes are now used in the ``Hasher``
class' ``amask_byte_order`` & ``mask_byte_order`` methods.
- The ``time_safe_equality`` & ``atime_safe_equality`` methods are now
standalone functions available from the ``generics.py`` module.
- Added ``reset_pool`` to the ``Processes`` & ``Threads`` classes. Also
fixed a missing piece of logic in their ``submit`` method.
- Added various conversion values & timing functions to the ``asynchs.py``
module.
- The ``make_uuid`` & ``amake_uuid`` coroutines had their names changed to
``make_uuids`` & ``amake_uuids``.
- Created a new ``Datastream`` class in ``generics.py`` to handle buffering
& resizing iterable streams of data. It enables simplifying logic that
must happen some number of iterations before the end of a stream. It's
utilized in the ``Padding`` class' generator functions available as
chainable ``Comprende`` methods.
- The ``data`` & ``adata`` generators can now produce a precise number of
``size``-length ``blocks`` as specified by a user. This gets rid of the
confusing usage of the old ``stop`` keyword-only argument, which stopped
a stream after *approximately* ``size`` number of elements.
- Improved the efficiency & safety of entropy production in the
``randoms.py`` module.
Changes for version 0.18.1
--------------------------
Major Changes
^^^^^^^^^^^^^
- Security Patch: Deprecated & replaced an internal kdf for saving
database tags due to a vulnerability. If an adversary can get a user
to reveal the value returned by the ``hmac`` method when fed the tag
file's filename & the salt used for that encrypted tag, then they
could deduce the decryption key for the tag. A version check was
added only for backwards compatibility & will be removed on the next
update. All databases should continue functioning as normal, though
all users are advised to **re-save their databases** after upgrading
so the new kdf can be used. This will not overwrite the old files,
so they'll need to be deleted manually.
- Replaced usage of the async ``switch`` coroutine with ``asyncio.sleep``
because it was not allowing tasks to switch as it was designed to.
Many improvements were made related to this change to make the
package behave better in async contexts.
- Removed the private method in the database classes which held a
reference to the root salt. It's now held in a private attribute.
This change simplifies the code a bit & allows instances to be
pickleable.
- The ``atimeout`` & ``timeout`` chainable ``Comprende`` generator
methods can now stop the generators' executions mid-iteration. They
run them in separate async tasks or thread pools, respectively, to
acheive this.
- The ``await_on`` & ``wait_on`` generators now restart their timeout
counters after every successful iteration that detected a new value
in their ``queue``. The ``delay`` keyword argument was changed to
``probe_frequency``, a keyword-only argument.
- Removed the package's dependency on the ``aioitertools`` package.
- Made the ``sympy`` package an optional import. If any of its
functionalities are used by the user, the package is only then
imported & this is done automatically.
- Various streamlining efforts were made to the imports & entropy
initialization to reduce the package's import & startup time.
Minor Changes
^^^^^^^^^^^^^
- Fixes of various typos, docstrings & tutorials.
- Various cleanups, refactorings & efficiency improvements.
- Added new tests for detecting malformed or modified ciphertexts.
- Removed extraneous functions in ``generics.py``.
- Add a ``UNIFORM_PRIME_512`` value to ``__datasets.py`` for use in the
``Hasher.mask_byte_order`` & ``Hasher.amask_byte_order`` methods.
Those methods were also altered to produce more uniform looking
results. The returned masked values are now also 64 bytes by default.
- Added an ``automate_key_use`` keyword-only boolean argument to the init
for the ``OneTimePad``, ``Keys`` & ``AsyncKeys`` classes. It can be toggled to
stop the classes from overwriting class methods so they
automatically read the instance's key attribute. This optionally
speeds up instantiation by an order of magnitude at the cost of
convenience.
- Fixed ``asynchs.Threads`` class' wrongful use of a ``multiprocessing``
``Manager.list`` object instead of a regular list.
- Changed the ``_delay`` keyword-only argument in ``Processes`` & ``Threads``
classes' methods to ``probe_freqeuncy`` so users can specify how often
results will be checked for after firing off a process, thread, or
associated pool submission.
- Now the ``asubmit`` & ``submit`` methods in ``Processes`` & ``Threads``
can accept keyword arguments.
- Added ``agather`` & ``gather`` methods to the ``Threads`` & ``Processes``
classes. They receive any number of functions, & ``args`` &/or ``kwargs`` to
pass to those functions when submitting them to their associated
pools.
- Changed the ``runsum`` instance IDs from hex strings to bytes & cleaned
up the instance caching & cleaning logic.
- Altered & made private the ``asalted_multiply`` & ``salted_multiply``
functions in the ``randoms.py`` module.
- Started a new event loop specific to the ``randoms.py`` module which
should prevent the ``RuntimeError`` when ``random_number_generator``
is called from within the user's running event loop.
- Added a ``ValueError`` check to the ``(a)cspr(b/n)g`` functions in
``randoms.py``. This will allow simultaneously running tasks to
request entropy from the function by returning a result from a
newly instantiated generator object.
- Added checks in the ``*_encipher`` & ``*_decipher`` generators to
help assure users correctly declare the mode for their StreamHMAC
validator instances.
- Fixed the ``__len__`` function in the database classes to count the
number of tags in the database & exclude their internal maintenaince
files.
- The ``TimeoutError`` raised after decrypting a ciphertext with an
expired timestamp now contains the seconds it has exceeded the ``ttl``
in a ``value`` attribute.
- The timestamp used to sign the package now displays the day of
signing instead of the second of signing.
- The ``(a)sum_sha_*`` & ``(a)sum_passcrypt`` generators were altered to
reapply the supplied ``salt`` on every iteration.
- Stabilized the usability of the ``stop`` keyword-only argument in the
``adata`` & ``data`` generators. It now directly decides the total
number of elements in a ``sequence`` allowed to be yielded.
Changes for version 0.18.0
--------------------------
Major Changes
^^^^^^^^^^^^^
- Security Patch: Rewrote the HMAC-like creation & authentication
process for all of the package's ciphers. Now, the ``*_encipher``
& ``*_decipher`` ``Comprende`` generators must be passed a validator
object to hash the ciphertext as it's being created / decrypted.
The ``StreamHMAC`` class was created for this purpose. It's initalized
with the user's long-term key, the ephemeral salt & the pid value.
The pid value can now effectively be used to validate additional data.
These changes force the package's cipher to be used as an AEAD cipher.
- Security Patch: The package's ``*_hmac`` hash functions & the ``Comprende``
class' hash generators were rewritten to prepend salts & keys to data
prior to hashing instead of appending. This is better for several
important reasons, such as: reducing the amortizability of costs in
trying to brute-force hashes, & more closely following the reasoning
behind the HMAC spec even though sha3 has a different security profile.
- Algorithm Patch: The ``akeys``, ``keys``, ``abytes_keys``, & ``bytes_keys``
algorithms have been patched to differentiate each iteration's two
sha3_512 hashes from one another in perpetuity. They contained a design
flaw which would, if both sha3_512 objects landed upon the same
1600-bit internal state, then they would produce the same keystreams
from then on. This change in backwards incompatible. This flaw is
infeasible to exploit in practice, but since the package's hashes &
ciphertext validations were already channging this release, there was
no reason to not fix this flaw so that it's self-healing if they ever
do land on the same internal states.
- The ``Passcrypt`` class & its algorithm were made more efficient to
better equalize the cost for users & adversaries & simplifies the
algorithm. Any inefficiencies in an implementation would likely cause
the adversary to be able to construct optimized implementations to
put users at an even greater disadvantage at protecting their inputs
to the passcrypt algorithm. It used the ``sum_sha_256`` hash function
internally, & since it was also changing in a non-backwards
compatible way with this update, it was the best time to clean up
the implementation.
- Updated the package's description & its docstrings that refer to
the package's cipher as an implementation of the one-time-pad. It's
not accurate since the package uses pseudo-random hash functions to
produce key material. Instead, the package's goal is to create a
pseudo-one-time-pad that's indistinguishable from a one-time-pad.
The ``OneTimePad`` class will keep its name for succinctness.
- New ``amake_token``, ``make_token``, ``aread_token`` & ``read_token``
class & instance methods added to the ``OneTimePad`` class. These
tokens are urlsafe base64 encoded, are encrypted, authenticated &
contain timestamps that can enforce a time-to-live for each token.
- Non-backwards compatible changes to the database classes' filenames,
encryption keys & HMACs. The ``*_hmac`` hash functions that the
databases rely on were changing with this update, so additionally the
filenames table used to encode the filenames was switched from the
``BASE_36_TABLE`` to the ``BASE_38_TABLE``. Both tables are safe for
uri's across all platforms, but the new table can encode information
slightly more efficiently.
- Major refactorings & signature changes across the package to make
passing keys & salts to ``*_hmac`` functions & the ``Comprende``
class' hash generators explicit.
- Removed the ``of`` keyword argument from all of the ``Comprende``
class' generators. It was overly complicating the code, & was not
entirely clear or useful for settings outside of the ``tags`` &
``atags`` generators.
- Removed ``pybase64`` from the package & its dependencies list. The
built-in python ``base64`` module works just fine.
- Sorted the ``WORDS_LIST``, ``ASCII_ALPHANUMERIC``, & ``BASE_64_TABLE``
datasets.
- The ``salt`` & ``asalt`` functions have been renamed to ``generate_salt``
& ``agenerate_salt`` for clarity's sake, & to reduce naming
collisions.
- Added another redundancy to the ``arandom_number_generator`` &
``random_number_generator`` functions. Now the async tasks it prepares
into a list are pseudo-randomly shuffled before being passed into
``asyncio.gather``.
Minor Changes
^^^^^^^^^^^^^
- Added a logo image to the package.
- Separated the FAQ section from ``PREADME.rst``.
- The ``primes`` & ``bits`` datasets are now represented in hex in the
source code.
- Added a ``BASE_38_TABLE`` dataset to the package.
- The database classes now fill an ephemeral dictionary of filenames
that couldn't be used to successfully load a tag file, available from
within the ``_corrupted_files`` attribute.
- The ``Comprende`` class' ``acache_check`` & ``cache_check`` context
manager methods are now called ``aauto_cache`` & ``auto_cache``.
- Added new ``bytes_count`` & ``abytes_count`` generators to ``generics.py``
module which increment each iteration & yield the results as bytes.
- Removed the ``akeypair`` & ``keypair`` functions from the package.
Their successors are the ``asingle_use_key`` & ``single_use_key`` methods
in the ``AsyncKeys`` & ``Keys`` classes. The attempt is to clarify &
put constraints on the interface for creating a bundle of key
material that has a single-use-only salt attached, as well as the pid
value.
- Moved ciphertext encoding functions into the ``BytesIO`` class from
the global ``generics.py`` module.
- Split ``PrimeGroups`` into two classes, one higher-level class by the
same name & a ``BasePrimeGroups`` class. The former also has some
added functionality for masking the order of bytes in a sequence
using an modular exponentiation.
- The ``Hasher`` class now has functionality added to mask the order
of a bytes sequence with a modular multiplication.
- Fixed the name of the project in the attribution lines in several
source files.
- Reconciled tests with the major changes in this release.
- The old identity key for the package that was signed by the gnupg
identity key was shredded & replaced with a new signed key.
- Several bug fixes to the ``setup.py`` automated code signing.
Changes for version 0.17.0
--------------------------
Major Changes
^^^^^^^^^^^^^
- Security Patch: The HMAC verifiers on ciphertexts did not include
the ``salt`` or ``pid`` values when deriving the HMAC. This
associated data can therefore be changed to cause a party to
decrypt a past ciphertext with a salt or pid of an attacker's
choosing. This is a critical vulnerability & it is highly recommended
all users update. The fix is to hash the ciphertext, ``salt``
& ``pid`` together & sending that hash into the validator to have
the HMAC created / tested. This change will cause all prior
ciphertexts to be marked invalid by the validator.
- Refactored the names of the Comprende cipher methods to better
communicate their intended use as lower level tools that cannot be
used on their own to obtain authenticated, CCA or CPA secure
encryption.
- Added more comprehensive tests for ``X25519`` & ``Ed25519`` classes,
as well as the protocols that utilize the ``X25519`` ecdh exchange.
Fixed some bugs in the process.
- ``X25519`` instances that contain a secret key now have access to
protocol methods which automatically pass their key in as a keyword
argument. This simplifies their usage further.
- Incorporated the new ``Hasher`` class into the package's random
number generator to improve its entropy production.
Minor Changes
^^^^^^^^^^^^^
- Various fixes to typos, docstrings & tutorials.
- New tutorials & docs added.
- Changed the default table in ``ByteIO`` 's ``json_to_ascii``, ``ajson_to_ascii``,
``ascii_to_json`` & ``aascii_to_json`` to the ``URL_SAFE_TABLE`` to
facilitate the creation of urlsafe_tokens.
- Removed all code in the ``Ropake`` class that was used to create a default
database to store a default salt for users. All of that functionality
is expected to be handled by the database classes' token & profile
creation tools.
- Fixed bug in package signing script that called hex from a string.
- Updated the package signing script to include these metadata in the
signatures of the ephemeral keys: name of the package, version, the
date in seconds.
- Added metadata to the ``setup.cfg`` file.
- Make passcrypt objects available from the ``keygens`` module.
- Add more consistent ability within ``Ropake`` class to specify a
time-to-live for protocol messages.
- Added check to make sure instances of ``X25519`` & ``Ed25519`` are
not trying to import a new secret key once they already have one.
This won't be allowed in favor of creating a new object for a new
secret key.
- Fixed bug in database classes' bytes ciphers which called themselves
recursively instead of calling the global functions of the same name.
Changes for version 0.16.0
--------------------------
Major Changes
^^^^^^^^^^^^^
- All ``Database`` & ``AsyncDatabase`` filenames have been converted to
base36 to aid in making the manifest files & the databases as a whole
more space efficient. These changes are not backwards compatible.
- More work was done to clean up the databases & make them more
efficient, as well as equalize the sizes of the database files to
mitigate leaking metadata about what they might contain.
- Added new ``X25519`` & ``Ed25519`` classes that greatly simplify the
usage of the cryptography module's 25519 based tools. They also help
organize the codebase better -- where ``Ropake`` was holding onto
all of the asymmetric tooling even though those tools were not part
of the Ropake protocol.
- New base & helper ``Asymmetric25519`` & ``BaseEllipticCurve`` classes
were added as well to facilitate the reorganization.
- Many methods in ``Ropake`` were turned private to simplify & clean up
the interface so its intended use as a protocol is more clear for users.
- Added the time-to-live functionality to ``Ropake`` decryption functions.
The ``TIMEOUT`` attribute on the class can also be changed to import
a global time-to-live for all ``Ropake`` ciphertexts.
- Removed all ``nc_`` hash functions from the package/generics.py module.
- The ``Namespace`` class now has a ``keys`` method so that namespaces
can be unpacked using star-star syntax.
- Because of the ongoing failures of gnupg, we are moving away from
signing our packages with gnupg. Our new Ed25519 keys will be from
the cryptography package, & we'll sign those with our gnupg key as a
secondary form of attestation. Our package signing will be automated
in the setup.py file & the methods we use will be transparent in the
code. The new signatures for each package version will be placed in
the file ``SIGNATURES.txt``.
Minor Changes
^^^^^^^^^^^^^
- Many fixes & additions to docstrings & tutorials.
- Massive refactorings, cleanups & typo fixes across the library,
especially in the database classes, ``Ropake`` & the ``ciphers`` module.
- Added comprehensive functional tests for the Ropake class.
- Added ``BASE_36_TABLE`` to the ``commons`` module.
- Fixed metadata issues in setup.py that caused upload issues to pypi.
- The ``generate_profile``, ``load_profile``, ``agenerate_profile`` &
``aload_profile`` database methods now accept arbitrary keyword arguments
that get passed into the database's __init__ constructor.
- ``username`` & ``password`` are now required keyword-only arguments
to the ``agenerate_profile_tokens`` & ``generate_profile_tokens``
classmethods.
- The ``aload`` & ``load`` database methods now take a ``manifest`` kwarg
that when toggled ``True`` will also refresh the manifest file from
disk.
- Now when a database object is ordered to delete itself, the entirety
of the instance's caches & attribute values are cleared & deleted.
- Filled out the references to strong key generators & protocols in the
``keygens`` module.
Changes for version 0.15.0
--------------------------
Major Changes
^^^^^^^^^^^^^
- Security Patch: The previous update left the default salt stored by
the ``Ropake`` class on the user filesystem as an empty string for
new files that were created since the ``asalt`` & ``salt`` functions
were switched to producing 256-bit values instead of 512-bits. This
bug has now been fixed.
- An 8 byte timestamp is now prepended to each plaintext during the
padding step. The decryption functions now take a ``ttl`` kwarg which
will measure & enforce a time-to-live for ciphertexts under threat of
``TimeoutError``.
- Added new profile feature to the database classes. This standardizes
& simplifies the process for users to open databases using only
low-entropy "profile" information such as ``username``, ``password``,
``*credentials`` & an optional ``salt`` a user may have access to.
The new ``agenerate_profile_tokens``, ``generate_profile_tokens``,
``agenerate_profile``, ``generate_profile``, ``aprofile_exists``,
``profile_exists``, ``aload_profile``, ``load_profile``, ``adelete_profile``
& ``delete_profile`` functions are the public part of this new feature.
- Some more database class attributes have been turned private to clean
up the api.
- Fixed typo in ``__exit__`` method of ``Database`` class which referenced
a method which had its name refactored, leading to a crash.
- Shifted the values in the ``primes`` dictionary such that the key for
each element in the dictionary is the exclusive maximum of each prime
in that element. Ex: primes[512][-1].to_bytes(64, "big") is now valid.
Whereas before, primes[512] was filled with primes that were 64 bytes
and 1 bit long, making them 65 byte primes. This changes some of the
values of constants in the package & therefore some values derived
from those constants.
- Slimmed down the number of elements in the ``primes`` & ``bits``
dictionaries, reducing the size of the package a great deal. ``primes``
now contains two primes in each element, the first is the minimum
prime of that bit length, the latter the maximum.
- Added ``URLSAFE_TABLE`` to the package.
- Made ``salt`` & ``pid`` & ``ttl`` keyword only arguments in key
generators & encryption / decryption functions, further tighening up
the api.
Minor Changes
^^^^^^^^^^^^^
- Added ``this_second`` function to ``asynchs`` module for integer time.
- Added ``apadding_key``, ``padding_key``, ``aplaintext_stream`` &
``plaintext_stream`` functions to the ``ciphers`` module.
- Added ``apadding_key``, ``padding_key`` to the ``keygens`` module &
``AsyncKeys`` & ``Keys`` classes.
- Added ``axi_mix``, ``xi_mix``, ``acheck_timestamp``, ``check_timestamp``,
to the ``generics`` module.
- Added ``acsprbg``, ``csprbg``, ``asalt``, ``salt``, ``apadding_key``,
``padding_key``, ``aplaintext_stream`` & ``plaintext_stream`` functions
to OneTimePad class as ``staticmethod`` & instance methods.
- Added ``acheck_timestamp`` & ``check_timestamp`` functions to the
``BytesIO`` class.
- Added ``adeniable_filename`` & ``deniable_filename`` to the ``paths``
module.
- Removed check for falsey data in encryption functions. Empty data is
& should be treated as valid plaintext.
- Various refactorings, docstring fixes & efficiency improvements.
- Added some new tests for database profiles.
Changes for version 0.14.0
--------------------------
Major Changes
^^^^^^^^^^^^^
- Security patch: The ``apad_bytes``, ``pad_bytes``, ``adepad_bytes`` &
``depad_bytes`` functions were changed internally to execute in a
more constant time. The variations were small for 256-byte buffers
(the default), but can grow very wide with larger buffers. The salt
in the package's encryption utilities is now used to derive the
plaintext's padding, making each padding unique.
- Unified the types of encodings the library's encryption functions
utilize for producing ciphertext. This includes databases. They now
all use the ``LIST_ENCODING``. This greatly increases the efficiency
of the databases' encryption/decryption, save/load times. And this
encoding is more space efficient. This change is backwards
incompatible.
- The ``LIST_ENCODING`` specification was also changed to produce
smaller ciphertexts. The salt is no longer encrypted & included as
the first 256 byte chunk of ciphertext. It is now packaged along with
ciphertext in the clear & is restricted to being a 256-bit hex
string.
- The interfaces for the ``Database`` & ``AsyncDatabase`` were cleaned
up. Many attributes & functions that were not intended as the public
interface of the classes were made "private". Also, the no longer
used utilities for encrypting & decrypting under the MAP_ENCODING
were removed.
- Updated the ``abytes_xor``, ``bytes_xor``, ``axor`` & ``xor`` generators
to shrink the size of the ``seed`` that's fed into the ``keystream``. This
allows the one-time-pad cipher to be more cpu efficient.
Minor Changes
^^^^^^^^^^^^^
- Fixed various typos, docstrings & tutorials that have no kept up
with the pace of changes.
- Various refactorings throughout.
- The ``akeypair`` & ``keypair`` functions now produce a ``Namespace``
populated with a 512-bit hex key & a 256-bit hex salt to be more
consistent with their intended use-case with the one-time-pad cipher.
- Removed ``aencode_salt``, ``encode_salt``, ``adecode_salt`` &
``decode_salt`` functions since they are no longer used in conjunction
with LIST_ENCODING ciphertexts.
- Updated tests to recognize these changes.
- Gave the ``OneTimePad`` class access to a ``BytesIO`` object under a
new ``io`` attribute.
Changes for version 0.13.0
--------------------------
Major Changes
^^^^^^^^^^^^^
- Security Patch: ``xor`` & ``axor`` functions that define the
one-time-pad cipher had a vulnerability fixed that can leak <1-bit of
plaintext. The issue was in the way keys were built, where the
multiplicative products of two key segments were xor'd together. This
lead to keys being slightly more likely to be positive integers,
meaning the final bit had a greater than 1/2 probability of being a
``0``. The fix is accompanied with an overhaul of the one-time-pad
cipher which is more efficient, faster, & designed with a better
understanding of the way bytes are processed & represented. The key
chunks now do not, & must not, surpass 256 bytes & neither should
any chunk of plaintext output. Making each chunk deterministically
256 bytes allows for reversibly formatting ciphertext to & from
bytes-like strings. These changes are backwards incompatible with
prior versions of this package & are strongly recommended.
- Added ``bytes_xor`` & ``abytes_xor`` functions which take in key
generators which produce key segments of type bytes instead of hex
strings.
- ``AsyncDatabase`` & ``Database`` now save files in bytes format,
making them much more efficient on disk space. They use the new
``BytesIO`` class in the ``generics`` module to transparently convert
to & from json & bytes. This change is also not backwards compatible.
- Removed ``acipher``, ``cipher``, ``adecipher``, ``decipher``,
``aorganize_encryption_streams``, ``organize_encryption_streams``,
``aorganize_decryption_streams``, ``organize_decryption_streams``,
``aencrypt``, ``encrypt``, ``adecrypt``, ``decrypt``, ``asubkeys`` &
``subkeys`` generators from the ``ciphers`` module & package to slim
down the code, remove repetition & focus on the cipher tools that
include hmac authentication.
- Removed deprecated diffie-hellman methods in ``Ropake`` class.
- Removed the static ``power10`` dictionary from the package.
- The default secret salt for the ``Ropake`` class is now derived from the
contents of a file that's in the databases directory which is chmod'd to
0o000 unless needed.
- Made ``aclient_message_key``, ``client_message_key``, ``aserver_message_key``,
& ``server_message_key`` ``Ropake`` class methods to help distinguish
client-to-server & server-to-client message keys which prevents replay
attacks on the one-message ROPAKE protocol.
- Added protocol coroutines to the ``Ropake`` class which allow for easily
engaging in 2DH & 3DH elliptic curve exchanges for servers & clients.
- Efficiency improvements to the ``aseeder`` & ``seeder`` generator functions
in the ``randoms`` module. This affects the ``acsprng`` & ``csprng`` objects
& all the areas in the library that utilize those objects.
- Changed the repr behavior of ``Comprende`` instances to redact all args &
kwargs by default to protect cryptographic material from unintentionally
being displayed on user systems. The repr can display full contents by
calling the ``enable_debugging`` method of the ``DebugControl`` class.
- All generator functions decorated with ``comprehension`` are now given
a ``root`` attribute. This allows direct access to the function without
needing to instantiate or run it as a ``Comprende`` object. This saves
a good deal of cpu & time in the overhead that would otherwise be
incurred by the class. This is specifically more helpful in tight &/or
lower-level looping.
Minor Changes
^^^^^^^^^^^^^
- Various refactorings across the library.
- Fixed various typos, bugs & inaccurate docstrings throughout the library.
- Add ``chown`` & ``chmod`` functions to the ``asynchs.aos`` module.
- Now makes new ``multiprocessing.Manager`` objects in the ``asynchs.Processes``
& ``asynchs.Threads`` classes to avoid errors that occur when using a stale
object whose socket connections are closed.
- Changed ``Ropake`` class' ``adb_login`` & ``db_login`` methods to
``adatabase_login_key`` & ``database_login_key``. Also, fix a crash bug in
those methods.
- Changed ``Ropake`` class' ``aec25519_pub``, ``ec25519_pub``, ``aec25519_priv``
& ``ec25519_priv`` methods to ``aec25519_public_bytes``, ``ec25519_public_bytes``,
``aec25519_private_bytes`` & ``ec25519_private_bytes``.
- Added low-level private methods to ``Ropake`` class which do derivation
& querying of the default class key & salt.
- Behavior changes to the ``ainverse_int`` & ``inverse_int`` functions in the
``generics`` module to allow handling bases represented in ``str`` or ``bytes``
type strings.
- Behavior & name changes to the ``abinary_tree`` & ``binary_tree`` functions in the
``generics`` module to ``abuild_tree`` & ``build_tree``. They now allow making
uniform trees of any width & depth, limited only by the memory in a
user's machine.
- Provided new ``acsprbg`` & ``csprbg`` objects to the library that return 512-bits
of cryptographically secure pseudo-random ``bytes`` type strings. They are
made by the new ``abytes_seeder`` & ``bytes_seeder`` generators.
- The ``csprng``, ``acsprng``, ``csprbg`` & ``acsprbg`` objects were
wrapped in functions that automatically restart the generators if they're
stalled / interrupted during a call. This keeps the package from melting
down if it can no longer call the CSPRNGs for new entropy.
- Cleaned up & simplified ``table_key`` functions in the ``keygens`` module.
- Changed the output of ``asafe_symm_keypair`` & ``safe_symm_keypair`` functions
to contain bytes values not their hex-only representation. Also removed
these functions from the main imports of the package since they are slow
& their main contribution is calling ``arandom_number_generator`` &
``random_number_generator`` to utilize a large entropy pool when starting
CSPRNGs.
- Added new values to the ``bits`` dictionary.
- Added ``apad_bytes``, ``pad_bytes``, ``adepad_bytes`` & ``depad_bytes``
functions which use ``shake_256`` to pad/depad plaintext bytes to & from
multiples of 256 bytes. They take in a key to create the padding.
This method is intended to also aid in protecting against padding
oracle attacks.
Changes for version 0.12.0
--------------------------
Major Changes
^^^^^^^^^^^^^
- The OPAKE protocol was renamed to ROPAKE, an acronym for Ratcheting
Opaque Password Authenticated Key Exchange. This change was necessary
since OPAKE is already a name for an existing PAKE protocol. This change
also means the ``Opake`` class name was changed to ``Ropake``.
- The ``Ropake`` class' registration algorithm was slightly modified to
use the generated Curve25519 ``shared_key`` an extra time in the key
derivation process. This shouldn't break any currently authenticated
sessions.
- The ``asyncio_contextmanager`` package is no longer a listed dependency
in ``setup.py``. The main file from that package was copied over into the
``/aiootp`` directory in order to remove the piece of code that caused
warnings to crop up when return values were retrieved from async
generators. This change will put an end to this whack-a-mole process of
trying to stop the warnings with try blocks scattered about the codebase.
- Added ``asave_tag``, ``save_tag``, ``asave_file`` & ``save_file`` methods
to the database classes so that specific entries can be saved to disk
without having to save the entire database which is much more costly. The
manifest file isn't saved to disk when these methods are used, so if a
tag file isn't already saved in the database, then the saved files will
not be present in the manifest or in the cache upon subsequent loads of
the database. The saved file will still however be saved on the
filesystem, though unbeknownst to the database instance.
- The ``Namespace`` class now redacts all obvious key material in instance
repr's, which is any 64+ hex character string, or any number with 64+
decimal digits.
- Removed the experimental recursive value retrieval within ``Comprende``'s
``__aexamine_sent_exceptions`` & ``__examine_sent_exceptions`` methods.
This change leads to more reliable & faster code, in exchange for an
unnecessary feature being removed.
- Bug fix of the ``auuids`` & ``uuids`` methods by editing the code in
the ``asyncio_contextmanager`` dependency & using the patched package
instead of the ``comprehension`` decorator for the ``arelay`` & ``relay``
methods of ``Comprende``. Their internal algorithms was also updated to
be simpler, but are incompatible with the outputs of past versions of
these methods.
Minor Changes
^^^^^^^^^^^^^
- Various refactorings & documentation additions / modifications throughout
the library.
- Various small bug fixes.
- The shared keys derived from the ``Ropake`` protocol are now returned in
a ``Namespace`` object instead of a raw dictionary, which allows the
values to be retrieved by dotted &/or bracketed lookup.
- The ``atest_hmac`` & ``test_hmac`` algorithms / methods were made more
efficient & were refactored. Now they call ``atime_safe_equality`` &
``time_safe_equality`` internally, which are new methods that can apply
the non-constant time but randomized timing comparisons on any pairs of
values.
Changes for version 0.11.0
--------------------------
Major Changes
^^^^^^^^^^^^^
- The Opake protocol was made greatly more efficient. This was done by
replacing the diffie-hellman verifiers with a hash & xor commit & reveal
system. Most hashing was made more efficient my using quicker & smaller
``sha_512`` function instead of ``nc_512``, & streamlining the protocol.
- The ``Opake.client`` & ``Opake.client_registration`` methods now take
an instantiated client database instead of client credentials which
improves security, efficiency & usability. This change reduces the amount
of exposure received by user passwords & other credentials. It also
simplifies usage of the protocol by only needing to carry around a
database instead of a slew of credentials, which is also faster, since
the credentials are passed through the cpu & memory hard ``passcrypt``
function everytime to open the database.
Minor Changes
^^^^^^^^^^^^^
- Heavy refactorings & documentation additions / modifications of the
``Opake`` class. Removed the ``Opake.ainit_database`` & ``Opake.init_database``
methods, & made the ``salt`` default argument parameter in
``Opake.aclient_database``, ``Opake.client_database``, ``Opake.adb_login`` &
``Opake.db_login`` into a keyword only argument so any extra user defined
``credentials`` are able to be passed without specifying a salt.
- The decorators for the ``Comprende.arelay`` & ``Comprende.relay`` methods
were changed from ``@asyncio_contextmanager.async_contextmanager`` to
``@comprehension()`` to stop that package from raising exceptions when
we retrieve return values from async generators.
Changes for version 0.10.1
--------------------------
Major Changes
^^^^^^^^^^^^^
- Added ``Processes`` & ``Threads`` classes to ``asynchs.py`` which abstract
spawning & getting return values from async & sync functions intended to
be run in threads, processes or pools of the former types. This simplifies
& adds time control to usages of processes & threads throughout the
library.
- Reduced the effectiveness of timing analysis of the modular exponentiation
in the ``Opake`` class' verifiers by making the process return values
only after discrete intervals of time. Timing attacks on that part of the
protocol may still be viable, but should be significantly reduced.
- Bug fix in ``Comprende`` which should take care of warnings raised from
the ``aiocontext`` package when retrieving async generator values by
raising ``UserWarning`` within them.
Minor Changes
^^^^^^^^^^^^^
- Heavy refactorings of the ``Opake`` class.
- Various refactorings & cleanups around the package.
- Further add ``return_exceptions=True`` flag to gather calls in ``ciphers.py``.
- Added ``is_registration`` & ``is_authentication`` which take a client
hello message that begin the ``Opake`` protocol, & return ``False`` if
the message is not either a registration or authentication message,
respectively, & return ``"Maybe"`` otherwise, since these functions can't
determine without running the protocol whether or not the message is
valid.
Changes for version 0.10.0
--------------------------
Major Changes
^^^^^^^^^^^^^
- Added a new oblivious, one-message, password authenticated key exchange
protocol class in ``aiootp.ciphers.Opake``. It is a first attempt at the
protocol, which works rather well, but may be changed or cleaned up in a
future update.
- Added the ``cryptography`` package as a dependency for elliptic curve
25519 diffie-hellman key exchange in the ``Opake`` protocol.
- Fix buggy data processing functions in ``generics.py`` module.
- Added ``silent`` flag to ``AsyncDatabase`` & ``Database`` methods, which
allows their instances to finish initializing even if a file is missing
from the filesystem, normally causing a ``FileNotFoundError``. This makes
trouble-shooting corrupted databases easier.
- Added new ``aiootp.paths.SecurePath`` function which returns the path to
a unique directory within the database's default directory. The name of
the returned directory is a cryptographic value used to create & open the
default database used by the ``Opake`` class to store the cryptographic
salt that secures the class' client passwords. It's highly recommended
to override this default database by instantiating the Opake class with
a custom user-defined key. The instance doesn't need to be saved, since
all the class' methods are either class or static methods. The ``__init__``
method only changes the class' default database to one opened with the
user-defined ``key`` &/or ``directory`` kwargs, & should really only be
done once at the beginning of an application.
Minor Changes
^^^^^^^^^^^^^
- Various refactorings & cleanups around the package.
- Added ``Comprende`` class feature to return the values from even the
generators within an instance's arguments. This change better returns
values to the caller from chains of ``Comprende`` generators.
- Fixed ``commons.BYTES_TABLE`` missing values.
- Added ``commons.DH_PRIME_4096_BIT_GROUP_16`` & ``commons.DH_GENERATOR_4096_BIT_GROUP_16``
constants for use in the ``Opake`` protocol's public key verifiers.
- Added other values to the ``commons.py`` module.
- Added new very large no-collision hash functions to the ``generics.py``
module used to xor with diffie-hellman public keys in the ``Opake`` class.
- Added new ``wait_on`` & ``await_on`` ``Comprende`` generators to ``generics.py``
which waits for a queue or container to be populated & yields it whenever
it isn't empty.
Changes for version 0.9.3
-------------------------
Major Changes
^^^^^^^^^^^^^
- Speed & efficiency improvements in the ``Comprende`` class & ``azip``.
Minor Changes
^^^^^^^^^^^^^
- Various refactorings & code cleanups.
- Added ``apop`` & ``pop`` ``Comprende`` generators to the library.
- Switched the default character table in the ``ato_base``, ``to_base``,
``afrom_base``, & ``from_base`` chainable generator methods from the 62
character ``ASCII_ALPHANUMERIC`` table, to the 95 character ``ASCII_TABLE``.
- Made the digits generators in ``randoms.py`` automatically create a new
cryptographically secure key if a key isn't passed by a user.
- Some extra data processing functions added to ``generics.py``.
Changes for version 0.9.2
-------------------------
Major Changes
^^^^^^^^^^^^^
- Added ``passcrypt`` & ``apasscrypt`` instance methods to ``OneTimePad``,
``Keys``, & ``AsyncKeys`` classes. They produce password hashes that are
not just secured by the salt & passcrypt algorithm settings, but also by
their main symmetric instance keys. This makes passwords infeasible to
crack without also compromising the instance's 512-bit key.
Minor Changes
^^^^^^^^^^^^^
- Further improvements to the random number generator in ``randoms.py``.
Made its internals less sequential thereby raising the bar of work needed
by an attacker to successfully carry out an order prediction attack.
- Added checks in the ``Passcrypt`` class to make sure both a salt &
password were passed into the algorithm.
- Switched ``PermissionError`` exceptions in ``Passcrypt._validate_args``
to ``ValueError`` to be more consistent with the rest of the class.
- Documentation updates / fixes.
Changes for version 0.9.1
-------------------------
Minor Changes
^^^^^^^^^^^^^
- Now any falsey values for the ``salt`` keyword argument in the library's
``keys``, ``akeys``, ``bytes_keys``, ``abytes_keys``, ``subkeys``, &
``asubkeys`` infinite keystream generators, & other functions around the
library, will cause them to generate a new cryptographically secure
pseudo-random value for the salt. It formerly only did this when ``salt``
was ``None``.
- The ``seeder`` & ``aseeder`` generators have been updated to introduce
512 new bits of entropy from ``secrets.token_bytes`` on every iteration
to ensure that the CSPRNG will produce secure outputs even if its
internal state is somehow discovered. This also allows for simply calling
the CSPRNG is enough, there's no longer a strong reason to pass new
entropy into it manually, except to add even more entropy as desired.
- Made ``size`` the last keywordCHECKSUMS.txt argument in ``encrypt`` &
``aencrypt`` to better mirror the signatures for rest of the library.
- Added ``token_bits`` & ``atoken_bits`` functions to ``randoms.py`` which
are renamings of ``secrets.randbits``.
- Refactored & improved the security og ``randoms.py``'s random number
generator.
Changes for version 0.9.0
-------------------------
Major Changes
^^^^^^^^^^^^^
- Added hmac codes to ciphertext for the following functions: ``json_encrypt``,
``ajson_encrypt``, ``bytes_encrypt``, ``abytes_encrypt``,
``Database.encrypt`` & ``AsyncDatabase.aencrypt``. This change greatly
increases the security of ciphertext by ensuring it hasn't been modified
or tampered with maliciously. One-time pad ciphertext is maleable, so
without hmac validation it can be changed to successfully allow
decryption but return the wrong plaintext. These functions are the
highest level abstractions of the library for encryption/decryption,
which made them excellent targets for this important security update.
As well, it isn't easily possible for the library to provide hmac codes
for generators that produce ciphertext, because the end of a stream of
ciphertext isn't known until after the results have left the scope
of library code. So users will need to produce their own hmac codes for
generator ciphertext unless we find an elegant solution to this issue.
These functions now all return dictionaries with the associated hmac
stored in the ``"hmac"`` entry. The bytes functions formerly returned
lists, now their ciphertext is available from the ``"ciphertext"`` entry.
And, all database files will have an hmac attached to them now. These
changes were designed to still be compatible with old ciphertexts but
they'll likely be made incompatible by the v0.11.x major release.
- Only truthy values are now valid ``key`` keyword arguments in the
library's ``keys``, ``akeys``, ``bytes_keys``, ``abytes_keys``, ``subkeys``,
& ``asubkeys`` infinite keystream generators. Also now seeding extra entropy
into ``csprng`` & ``acsprng`` when ``salt`` is falsey within them.
- Only truthy values are now valid for ``password`` & ``salt`` arguments in
``apasscrypt``, ``passcrypt`` & their variants.
Minor Changes
^^^^^^^^^^^^^
- Updates to documentation & ``README.rst`` tutorials.
- The ``kb``, ``cpu``, & ``hardness`` arguments in ``sum_passcrypt`` &
``asum_passcrypt`` chainable generator methods were switched to keyword
only arguments.
Changes for version 0.8.1
-------------------------
Major Changes
^^^^^^^^^^^^^
- Added ``sum_passcrypt`` & ``asum_passcrypt`` chainable generator methods
to ``Comprende`` class. They cumulatively apply the passcrypt algorithm
to each yielded value from an underlying generator with the passcrypt'd
prior yielded result used as a salt. This allows making proofs of work,
memory & space-time out of iterations of the passcrypt algorithm very
simple.
Minor Changes
^^^^^^^^^^^^^
- Various inaccurate docstrings fixed.
- Various refactorings of the codebase.
- Made ``kb``, ``cpu``, & ``hardness`` arguments into keyword only arguments
in ``AsyncDatabase`` & ``Database`` classes.
- The ``length`` keyword argument in functions around the library was
changed to ``size`` to be consistent across the whole package. Reducing
the cognitive burden of memorizing more than one name for the same concept.
- Various efficiency boosts.
- Edits to ``README.rst``.
- Added ``encode_salt``, ``aencode_salt``, ``decode_salt`` & ``adecode_salt``
functions to the library, which gives access to the procedure used to
encrypt & decrypt the random salt which is often the first element
produced in one-time pad ciphertexts.
- Added cryptographically secure pseudo-random values as default keys in
encryption functions to safeguard against users accidentally encrypting
data without specifying a key. This way, such mistakes will produce
ciphertext with an unrecoverable key, instead of without a key at all.
Changes for version 0.8.0
-------------------------
Major Changes
^^^^^^^^^^^^^
- Fix ``test_hmac``, ``atest_hmac`` functions in the keys & database
classes. The new non-constant-time algorithm needs a random salt to be
added before doing the secondary hmac to prevent some potential exotic
forms of chosen plaintext/ciphertext attacks on the algorithm. The last
version of the algorithm should not be used.
- The ``Keys`` & ``AsyncKeys`` interfaces were overhauled to remove the
persistance of instance salts. They were intended to be updated by users
with the ``reset`` & ``areset`` methods, but that cannot be guaranteed
easily through the class, so it is an inappropriate interface since
reusing salts for encryption is completely insecure. The instances do
still maintain state of their main encryption key, & new stateful methods
for key generation, like ``mnemonic`` & ``table_key``, have been added.
The ``state`` & ``astate`` methods have been removed.
- Gave ``OneTimePad`` instances new stateful methods from the ``ciphers.py``
module & ``keygens.py`` keys classes. Its instances now remember the main
symmetric key behind the ``key`` property & automatically passes it as a
keyword argument to the methods in ``OneTimePad.instance_methods``.
Minor Changes
^^^^^^^^^^^^^
- Update ``CHANGES.rst`` file with the updates that were not logged for
v0.7.1.
- ``BYTES_TABLE`` was turned into a list so that the byte characters can
be retrieved instead of their ordinal numbers.
Changes for version 0.7.1
-------------------------
Major Changes
^^^^^^^^^^^^^
- Fix a mistake in the signatures of ``passcrypt`` & ``apasscrypt. The args
``kb``, ``cpu`` & ``hardness`` were changed into keyword only arguments
to mitigate user mistakes, but the internal calls to those functions were
still using positional function calls, which broke the api. This issue
is now fixed.
Changes for version 0.7.0
-------------------------
Major Changes
^^^^^^^^^^^^^
- Replaced usage of bare ``random`` module functions, to usage of an
instance of ``random.Random`` to keep from messing with user's settings
on that module.
- Finalized the algorithm for the ``passcrypt`` & ``apasscrypt`` functions.
The algorithm is now provably memory & cpu hard with a wide security
margin with adequate settings. The algorithm isn't likely change with
upcoming versions unless a major flaw is found.
- The default value for the ``cpu`` argument in ``passcrypt`` & ``apasscrypt``
is now ``3`` & now directly determines how many hash iterations are done
for each element in the memory cache. This provides much more
responsiveness to users & increases the capacity to impact resource cost
with less tinkering.
- Switched the ``AsyncKeys.atest_hmac`` & ``Keys.test_hmac`` methods to a
scheme which is not constant time, but which instead does not leak useful
information. It does this by not comparing the hmacs of the data, but of
a pair of secondary hmacs. The timing analysis itself is now dependant
on knowledge of the key, since any conclusions of such an analysis would
be unable correlate its findings with any supplied hmac without it.
- Added ``test_hmac`` & ``atest_hmac`` to the database classes, & changed
their hmac algorithm from ``sha3_512`` to ``sha3_256``.
Minor Changes
^^^^^^^^^^^^^
- Various code cleanups, refactorings & speedups.
- Several fixes to inaccurate documentation.
- Several fixes to inaccurate function signatures.
- Added ``mnemonic`` & ``amnemonic`` key generators to ``keygens.py`` with
a wordlist 2048 entries long. A custom wordlist can also be passed in.
- Minor changes in ``Comprende`` to track down a bug in the functions that
use the asyncio_contextmanager package. It causes a warning when asking
async generators to return (not yield) values.
- Some refactoring of ``random_number_generator`` & ``arandom_number_generator``.
Changes for version 0.6.0
-------------------------
Major Changes
^^^^^^^^^^^^^
- Replaced the usage of ``os.urandom`` within the package with
``secrets.token_bytes`` to be more reliable across platforms.
- Replaced several usages of ``random.randrange`` within ``randoms.py`` to
calls to ``secrets.token_bytes`` which is faster & more secure. It
now also seeds ``random`` module periodically prior to usage.
- Changed the internal cache sorting algorithm of ``passcrypt`` &
``apasscrypt`` functions. The key function passed to ``list.sort(key=key)``
now not only updates the ``hashlib.sha3_512`` proof object with
each element in the cache, but with it's own current output. This change
is incompatible with previous versions of the functions. The key function
is also trimmed down of unnecessary value checking.
- The default value for the ``cpu`` argument in ``passcrypt`` & ``apasscrypt``
is now ``40_000``. This is right at the edge of when the argument begins
impacting the cpu work needed to comptute the password hash when the ``kb``
argument is the default of ``1024``.
- Switched the ``AsyncKeys.atest_hmac`` & ``Keys.test_hmac`` methods to a
constant time algorithm.
Minor Changes
^^^^^^^^^^^^^
- Various code cleanups, refactorings & speedups.
- Added a ``concurrent.futures.ThreadPoolExecutor`` instance to the ``asynchs``
module for easily spinning off threads. It's available under
``asynchs.thread_pool``.
- Added ``sort`` & ``asort`` chainable generator method to the ``Comprende``
class. They support sorting by a ``key`` sorting function as well.
- Changed the name of ``asynchs.executor_wrapper`` to ``asynchs.wrap_in_executor``.
- Changed the name of ``randoms.non0_digit_stream``, ``randoms.anon0_digit_stream``,
``randoms.digit_stream`` & ``randoms.adigit_stream`` to ``randoms.non_0_digits``,
``randoms.anon_0_digits``, ``randoms.digits`` & ``randoms.adigits``.
- Several fixes to inaccurate documentation.
- ``apasscrypt`` & ``Passcrypt.anew`` now use the synchronous version of the
algorithm internally because it's faster & it doesn't change the
parallelization properties of the function since it's already run
automatically in another process.
- Added ``shuffle``, ``ashuffle``, ``unshuffle``, & ``aunshuffle`` functions
to ``randoms.py`` that reorder sequences pseudo-randomly based on their
``key`` & ``salt`` keyword arguments.
- Fixed bugs in ``AsyncKeys`` & ``debuggers.py``.
- Added ``debugger`` & ``adebugger`` chainable generator methods to the
``Comprende`` class which benchmarks & inspects running generators with
an inline syntax.
Changes for version 0.5.1
-------------------------
Major Changes
^^^^^^^^^^^^^
- Fixed a bug in the methods ``auuids`` & ``uuids`` of the database classes
that assigned to a variable within a closure that was nonlocal but which
wasn't declared non-local. This caused an error which made the methods
unusable.
- Added ``passcrypt`` & ``apasscrypt`` functions which are designed to be
tunably memory & cpu hard password-based key derivation function. It was
inspired by the scrypt protocol but internally uses the library's tools.
It is a first attempt at the protocol, it's internal details will likely
change in future updates.
- Added ``bytes_keys`` & ``abytes_keys`` generators, which are just like
the library's ``keys`` generator, except they yield the concatenated
``sha3_512.digest`` instead of the ``sha3_512.hexdigest``.
- Added new chainable generator methods to the ``Comprende`` class for
processing bytes, integers, & hex strings into one another.
Minor Changes
^^^^^^^^^^^^^
- Various code cleanups.
- New tests added to the test suite for ``passcrypt`` & ``apasscrypt``.
- The ``Comprende`` class' ``alist`` & ``list`` methods can now be passed
a boolean argument to return either a ``mutable`` list directly from the
lru_cache, or a copy of the cached list. This list is used by the
generator itself to yield its values, so wilely magic can be done on the
list to mutate the underlying generator's results.
Changes for version 0.5.0
-------------------------
Major Changes
^^^^^^^^^^^^^
- Added interfaces in ``Database`` & ``AsyncDatabase`` to handle encrypting
& decrypting streams (``Comprende`` generators) instead of just raw json
data. They're methods called ``encrypt_stream``, ``decrypt_stream``,
``aencrypt_stream``, & ``adecrypt_stream``.
- Changed the attribute ``_METATAG`` used by ``Database`` & ``AsyncDatabase``
to name the metatags entry in the database. This name is smaller, cleaner
& is used to prevent naming collisions between user entered values & the
metadata the classes need to organize themselves internally. This change
will break databases from older versions keeping them from accessing their
metatag child databases.
- Added the methods ``auuids`` & ``uuids`` to ``AsyncDatabase`` & ``Database``
which return coroutines that accept potentially sensitive identifiers &
turns them into salted ``size`` length hashes distinguished by a ``salt``
& a ``category``.
Minor Changes
^^^^^^^^^^^^^
- Various code & logic cleanups / speedups.
- Refactorings of the ``Database`` & ``AsyncDatabase`` classes.
- Various inaccurate docstrings fixed.
Changes for version 0.4.0
-------------------------
Major Changes
^^^^^^^^^^^^^
- Fixed bug in ``aiootp.abytes_encrypt`` function which inaccurately called
a synchronous ``Comprende`` end-point method on the underlying async
generator, causing an exception and failure to function.
- Changed the procedures in ``akeys`` & ``keys`` that generate their internal
key derivation functions. They're now slightly faster to initialize &
more theoretically secure since each internal state is fed by a seed
which isn't returned to the user. This encryption algorithm change is
incompatible with the encryption algorithms of past versions.
Minor Changes
^^^^^^^^^^^^^
- Various code cleanups.
- Various inaccurate docstrings fixed.
- Keyword arguments in ``Keys().test_hmac`` & ``AsyncKeys().atest_hmac``
had their order switched to be slightly more friendly to use.
- Added documentation to ``README.rst`` on the inner workings of the
one-time-pad algorithm's implementation.
- Made ``Compende.arandom_sleep`` & ``Compende.random_sleep`` chainable
generator methods.
- Changed the ``Compende.adelimit_resize`` & ``Compende.delimit_resize``
algorithms to not yield inbetween two joined delimiters in a sequence
being resized.
Changes for version 0.3.1
-------------------------
Minor Changes
^^^^^^^^^^^^^
- Fixed bug where a static method in ``AsyncDatabase`` & ``Database`` was
wrongly labelled a class method causing a failure to initialize.
Changes for version 0.3.0
-------------------------
Major Changes
^^^^^^^^^^^^^
- The ``AsyncDatabase`` & ``Database`` now use the more secure ``afilename``
& ``filename`` methods to derive the hashmap name and encryption streams
from a user-defined tag internal to their ``aencrypt`` / ``adecrypt`` /
``encrypt`` / ``decrypt`` methods, as well as, prior to them getting called.
This will break past versions of databases' ability to open their files.
- The package now has built-in functions for using the one-time-pad
algorithm to encrypt & decrypt binary data instead of just strings
or integers. They are available in ``aiootp.abytes_encrypt``,
``aiootp.abytes_decrypt``, ``aiootp.bytes_encrypt`` & ``aiootp.bytes_decrypt``.
- The ``Comprende`` class now has generators that do encryption & decryption
of binary data as well. They are available from any ``Comprende`` generator
by the ``abytes_encrypt``, ``abytes_decrypt``, ``bytes_encrypt`` & ``bytes_decrypt``
chainable method calls.
Minor Changes
^^^^^^^^^^^^^
- Fixed typos and inaccuracies in various docstrings.
- Added a ``__ui_coordination.py`` module to handle inserting functionality
from higher-level to lower-level modules and classes.
- Various code clean ups and redundancy eliminations.
- ``AsyncKeys`` & ``Keys`` classes now only update their ``self.salt`` key
by default when their ``areset`` & ``reset`` methods are called. This
aligns more closely with their intended use.
- Added ``arandom_sleep`` & ``random_sleep`` chainable methods to the
``Comprende`` class which yields outputs of generators after a random
sleep for each iteration.
- Added several other chainable methods to the ``Comprende`` class for
string & bytes data processing. They're viewable in ``Comprende.lazy_generators``.
- Added new, initial tests to the test suite.
Changes for version 0.2.0
-------------------------
Major Changes
^^^^^^^^^^^^^
- Added ephemeral salts to the ``AsyncDatabase`` & ``Database`` file
encryption procedures. This is a major security fix, as re-encryption
of files with the same tag in a database with the same open key would
use the same streams of key material each time, breaking encryption if
two different versions of a tag file's ciphertext stored to disk were
available to an adversary. The database methods ``encrypt``, ``decrypt``,
``aencrypt`` & ``adecrypt`` will now produce and decipher true one-time
pad ciphertext with these ephemeral salts.
- The ``aiootp.subkeys`` & ``aiootp.asubkeys`` generators were revamped
to use the ``keys`` & ``akeys`` generators internally instead of using
their own, slower algorithm.
- ``AsyncDatabase`` file deletion is now asynchronous by running the
``builtins.os.remove`` function in an async thread executor. The
decorator which does the magic is available at ``aiootp.asynchs.executor_wrapper``.
Minor Changes
^^^^^^^^^^^^^
- Fix typos in ``__root_salt`` & ``__aroot_salt`` docstrings. Also replaced
the ``hash(self)`` argument for their ``lru_cache`` & ``alru_cache``
with a secure hmac instead.
- add ``gi_frame``, ``gi_running``, ``gi_code``, ``gi_yieldfrom``,
``ag_frame``, ``ag_running``, ``ag_code`` & ``ag_await`` properties to
``Comprende`` class to mirror async/sync generators more closely.
- Remove ``ajson_encrypt``, ``ajson_decrypt``, ``json_encrypt``,
``json_decrypt`` functions' internal creation of dicts to contain the
plaintext. It was unnecessary & therefore wasteful.
- Fix docstrings in ``OneTimePad`` methods mentioning ``parent`` kwarg which
is a reference to deleted, refactored code.
- Fix incorrect docstrings in databases ``namestream`` & ``anamestream``
methods.
- Added ``ASYNC_GEN_THROWN`` constant to ``Comprende`` class to try to stop
an infrequent & difficult to debug ``RuntimeError`` when async generators
do not stop after receiving an ``athrow``.
- Database tags are now fully loaded when they're copied using the methods
``into_namespace`` & ``ainto_namespace``.
- Updated inaccurate docstrings in ``map_encrypt``, ``amap_encrypt``,
``map_decrypt`` & ``amap_decrypt`` ``OneTimePad`` methods.
- Added ``acustomize_parameters`` async function to ``aiootp.generics``
module.
- Various code clean ups.
Changes for version 0.1.0
-------------------------
Minor Changes
^^^^^^^^^^^^^
- Initial version.
Major Changes
^^^^^^^^^^^^^
- Initial version.
_`Known Issues` ................................... `Table Of Contents`_
========================================================================
- The test suite for this software is under construction, & what tests
have been published are currently inadequate to the needs of
cryptography software.
- This package is currently in beta testing & active development,
meaning major changes are still possible when there are really good
reasons to do so. Contributions are welcome. Send us a message if
you spot a bug or security vulnerability:
- gonzo.development@protonmail.ch
- rmlibre@riseup.net
- ed25519-key: 70d1740f2a439da98243c43a4d7ef1cf993b87a75f3bb0851ae79de675af5b3b
- x25519-key: 4457276dbcae91cc5b69f1aed4384b9eb6f933343bb44d9ed8a80e2ce438a450
Raw data
{
"_id": null,
"home_page": "https://twitter.com/aiootp",
"name": "aiootp",
"maintainer": "Gonzo Investigative Journalism Agency, LLC",
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": "gonzo.development@protonmail.ch",
"keywords": "xor key salt pepper nonce aad iv siv resuse misuse AEAD auth authenticated authentication shmac hmac nmac mac digest integrity infosec opsec appsec stream cipher chunky2048 chunky encrypt plaintext decrypt ciphertext passcrypt passphrase password based derivation function 3dh 2dh 25519 x25519 ed25519 curve25519 diffie hellman sign signature verify verification db database store user uuid unique guid global transparent encryption decryption Chunky2048 indistinguishable pseudo one time pad onetimepad domain-specific kdf separation bits 64 128 256 512 1024 2048 4096 hash sha sha3 sha-3 keccak ephemeral byte entropy PRF PRG PRP RNG PRNG CSPRNG cryptographically secure random number generator bitwise operations information cyber security chosen attack resistance resistant tweak tweakable anonymous anonymity pseudonymous symmetric asymmetric communications utilities simple clean code crypto cryptology cryptography beta testing data science processing await async asyncio parallel concurrency coroutine coroutines comprehension",
"author": "Gonzo Investigative Journalism Agency, LLC",
"author_email": "gonzo.development@protonmail.ch",
"download_url": "https://files.pythonhosted.org/packages/12/f1/7efec630717d776118e302c51bb842b4ba3c06e639f680663d4a8456703d/aiootp-0.22.0.tar.gz",
"platform": null,
"description": ".. image:: https://raw.githubusercontent.com/rmlibre/aiootp/main/logo.png\n :target: https://raw.githubusercontent.com/rmlibre/aiootp/main/logo.png\n :alt: aiootp python package logo\n\n\n\n\naiootp - Asynchronous pseudo one-time pad based crypto and anonymity library.\n=============================================================================\n\n``aiootp`` is an asynchronous library providing access to cryptographic \nprimatives and abstractions, transparently encrypted / decrypted file \nI/O and databases, as well as powerful, pythonic utilities that \nsimplify data processing & cryptographic procedures in python code. \nThis library's online, salt reuse / misuse resistant, tweakable AEAD cipher, called \n``Chunky2048``, is an implementation of the **pseudo one-time pad**. The \naim is to create a simple, standard, efficient implementation that's \nindistinguishable from the unbreakable one-time pad cipher; to give \nusers and applications access to user-friendly cryptographic tools; and, \nto increase the overall security, privacy, and anonymity on the web, and \nin the digital world. Users will find ``aiootp`` to be easy to write, \neasy to read, and fun. \n\n\n\n\nImportant Disclaimer\n--------------------\n\n``aiootp`` is experimental software that works with Python 3.6+. \nIt's a work in progress. The programming API could change with \nfuture updates, and it isn't bug free. ``aiootp`` provides powerful \nsecurity tools and misc utilities that're designed to be \ndeveloper-friendly and privacy preserving. \nAs a security tool, ``aiootp`` needs to be tested and reviewed \nextensively by the programming and cryptography communities to \nensure its implementations are sound. We provide no guarantees. \nThis software hasn't yet been audited by third-party security \nprofessionals.\n\n\n\n\n.. image:: https://img.shields.io/pypi/v/aiootp\n :target: https://img.shields.io/pypi/v/aiootp\n :alt: version\n\n.. image:: https://img.shields.io/pypi/pyversions/aiootp?color=black\n :target: https://img.shields.io/pypi/pyversions/aiootp?color=black\n :alt: python-versions\n\n.. image:: https://img.shields.io/badge/License-AGPL%20v3-red.svg\n :target: https://img.shields.io/badge/License-AGPL%20v3-red.svg\n :alt: license\n\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg\n :target: https://img.shields.io/badge/code%20style-black-000000.svg\n :alt: code-style\n\n.. image:: https://github.com/rmlibre/aiootp/actions/workflows/linux-python-app.yml/badge.svg\n :target: https://github.com/rmlibre/aiootp/actions/workflows/linux-python-app.yml/badge.svg\n :alt: linux-build-status\n\n.. image:: https://github.com/rmlibre/aiootp/actions/workflows/windows-python-app.yml/badge.svg\n :target: https://github.com/rmlibre/aiootp/actions/workflows/windows-python-app.yml/badge.svg\n :alt: windows-build-status\n\n.. image:: https://github.com/rmlibre/aiootp/actions/workflows/macos-python-app.yml/badge.svg\n :target: https://github.com/rmlibre/aiootp/actions/workflows/macos-python-app.yml/badge.svg\n :alt: macos-build-status\n\n\n\n\nQuick Install\n-------------\n\n.. code-block:: shell\n\n $ sudo apt-get install python3-setuptools python3-pip\n\n $ pip3 install --user --upgrade pip typing aiootp\n\n\n\n\nRun Tests\n---------\n\n.. code-block:: shell\n\n $ cd ~/aiootp/tests\n\n $ coverage run --source aiootp -m pytest -vv test_aiootp.py\n\n\n\n\n_`Table Of Contents`\n--------------------\n\n- `Transparently Encrypted Databases`_\n\n a) `Ideal Initialization`_\n \n b) `User Profiles`_\n \n c) `Tags`_\n \n d) `Metatags`_\n \n e) `Basic Management`_\n \n f) `Mirrors`_\n \n g) `Public Cryptographic Functions`_\n\n I. `Encrypt / Decrypt`_\n \n II. `HMACs`_\n \n\n- `Chunky2048 Cipher`_\n \n a) `High-level Functions`_\n \n b) `High-level Generators`_\n \n\n- `Passcrypt`_\n\n a) `Hashing & Verifying Passphrases`_\n\n b) `Passcrypt Algorithm Overview`_\n\n\n- `X25519 & Ed25519`_\n \n a) `X25519`_\n \n b) `Ed25519`_\n \n\n- `Comprende`_\n \n a) `Synchronous Generators`_\n \n b) `Asynchronous Generators`_\n \n\n- `Module Overview`_\n \n\n- `FAQ`_\n \n\n- `Changelog`_\n \n\n- `Known Issues`_\n\n\n\n\n_`Transparently Encrypted Databases` .............. `Table Of Contents`_\n------------------------------------------------------------------------\n\nThe package's ``AsyncDatabase`` & ``Database`` classes are very powerful data persistence utilities. They automatically handle encryption & decryption of user data & metadata, providing a pythonic interface for storing & retrieving any bytes or JSON serializable objects. They're designed to seamlessly bring encrypted bytes at rest to users as dynamic objects in use.\n\n\n_`Ideal Initialization` ........................... `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nMake a new user key with a fast, cryptographically secure pseudo-random number generator. Then this strong 64-byte key can be used to create a database object.\n\n.. code-block:: python\n\n from aiootp import acsprng, AsyncDatabase\n \n \n key = await acsprng()\n\n db = await AsyncDatabase(key)\n \n\n_`User Profiles` .................................. `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nWith User Profiles, passphrases may be used instead to open a database. Often, passwords & passphrases contain very little entropy. So, they aren't recommended for that reason. However, profiles provide a succinct way to use passphrases more safely. They do this by deriving strong keys from low entropy user input using the memory/cpu hard passcrypt algorithm, & a secret salt which is automatically generated & stored on the user's filesystem.\n\n.. code-block:: python\n\n # Automatically convert any available user credentials into \n\n # cryptographic tokens which help to safely open databases ->\n\n db = await AsyncDatabase.agenerate_profile(\n \n b\"server-url.com\", # Here an unlimited number of bytes-type\n # arguments can be passed as additional\n b\"address@email.net\", # optional credentials.\n \n username=b\"username\",\n \n passphrase=b\"passphrase\",\n \n salt=b\"optional salt keyword argument\",\n # Optional passcrypt configuration:\n mb=256, # The memory cost in Mibibytes (MiB)\n\n cpu=2, # The computational complexity & number of iterations\n\n cores=8, # How many parallel processes passcrypt will utilize\n \n )\n\n\n_`Tags` ........................................... `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nData within databases are primarily organized by Tags. Tags are simply string labels, and the data stored under them can be any bytes or JSON serializable objects.\n\n.. code-block:: python\n\n async with db:\n \n # Using bracketed assignment adds tags to the cache\n \n db[\"tag\"] = {\"data\": \"can be any JSON serializable object\"}\n \n db[\"hobby\"] = b\"fash smasher\"\n \n db[\"bitcoin\"] = \"0bb6eee10d2f8f45f8a\"\n \n db[\"lawyer\"] = {\"#\": \"555-555-1000\", \"$\": 13000.50}\n \n db[\"safehouses\"] = [\"Dublin Forgery\", \"NY Insurrection\"]\n \n # Changes in the cache are saved to disk when the context closes.\n \n \n # View an instance's tags ->\n\n db.tags\n >>> {'tag', 'hobby', 'bitcoin', 'lawyer', 'safehouses'}\n\n\n # View the filenames that locate the data for each tag ->\n \n db.filenames\n >>> {'0z0l10btu_yd-n4quc8tsj9baqu8xmrxz87ix',\n '197ulmqmxg15lebm26zaahpqnabwr8sprojuh',\n '248piaop3j9tmcvqach60qk146mt5xu6kjc-u',\n '2enwc3crove2cnrx7ks963d8_se25k6cdn6q9',\n '5dm-60yspq8yhah4ywxcp52kztq_9toj0owm2'}\n\n\n # There are various ways of working with tags ->\n\n await db.aset_tag(\"new_tag\", [\"data\", \"goes\", \"here\"]) # stored only in cache\n\n await db.aquery_tag(\"new_tag\") # reads from disk if not in the cache\n >>> ['data', 'goes', 'here']\n\n tag_path = db.path / await db.afilename(\"new_tag\")\n\n \"new_tag\" in db\n >>> True\n\n tag_path.is_file() # the tag is saved in the cache, not to disk yet\n >>> False\n\n await db.asave_tag(\"new_tag\")\n \n tag_path.is_file() # now it's saved to disk\n >>> True\n \n \n # This removes the tag from cache, & any of its unsaved changes ->\n\n await db.arollback_tag(\"new_tag\")\n\n\n # Or, the user can take the tag out of the database & the filesystem ->\n\n await db.apop_tag(\"new_tag\")\n >>> ['data', 'goes', 'here']\n\n \"new_tag\" in db\n >>> False\n\n tag_path.is_file()\n >>> False\n\nAccess to data is open to the user, so care must be taken not to let external API calls touch the database without accounting for how that can go wrong.\n\n\n_`Metatags` ....................................... `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nMetatags are used to organize data by string names & domain separate cryptographic material. They are fully-fledged databases all on their own, with their own distinct key material too. They're accessible from the parent through an attribute that's added to the parent instance with the same name as the metatag. When the parent is saved, or deleted, then their descendants are also.\n\n.. code-block:: python\n\n # Create a metatag database ->\n\n molly = await db.ametatag(\"molly\")\n\n\n # They can contain their own sets of tags (and metatags) ->\n \n molly[\"hobbies\"] = [\"skipping\", \"punching\"]\n \n molly[\"hobbies\"].append(\"reading\")\n\n\n # The returned metatag & the reference in the parent are the same ->\n\n assert molly[\"hobbies\"] is db.molly[\"hobbies\"]\n \n assert isinstance(molly, AsyncDatabase)\n \n\n # All of an instance's metatags are viewable ->\n\n db.metatags\n >>> {'molly'}\n \n\n # Delete a metatag from an instance ->\n\n await db.adelete_metatag(\"molly\")\n \n db.metatags\n >>> set()\n \n assert not hasattr(db, \"molly\")\n\n\n_`Basic Management` ............................... `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nThere's a few settings & public methods on databases for users to manage their instances & data. This includes general utilities for saving & deleting databases to & from the filesystem, as well as fine-grained controls for how data is handled. \n\n.. code-block:: python\n\n # The path attribute is set within the instance's __init__\n\n # using a keyword-only argument. It's the directory where the\n\n # instance will store all of its files.\n\n db.path\n >>> PosixPath('site-packages/aiootp/aiootp/databases')\n \n \n # Write database changes to disk with transparent encryption ->\n \n await db.asave_database()\n\n\n # Entering the instance's context also saves data to disk ->\n\n async with db:\n \n print(\"Saving to disk...\")\n \n\n # Delete a database from the filesystem ->\n \n await db.adelete_database()\n \n \nAs databases grow in the number of tags, metatags & the size of data within, it becomes desireable to load data from them as needed, instead of all at once into the cache during initialization. This is why the ``preload`` boolean keyword-only argument is set to ``False`` by default.\n\n.. code-block:: python\n\n # Let's create some test values to show the impact preloading has ->\n\n async with (await AsyncDatabase(key, preload=True)) as db:\n\n db[\"favorite_foods\"] = [\"justice\", \"community\"]\n \n await db.ametatag(\"exercise_routines\") \n\n db.exercise_routines[\"gardening\"] = {\"days\": [\"moday\", \"wednesday\"]}\n \n db.exercise_routines[\"swimming\"] = {\"days\": [\"thursday\", \"saturday\"]}\n \n\n # Again, preloading into the cache is toggled off by default ->\n\n uncached_db = await AsyncDatabase(key)\n \n \n # To retrieve elements, ``aquery_tag`` isn't necessary when \n\n # preloading is used, since the tag is already in the cache ->\n\n async with uncached_db:\n \n db[\"favorite_foods\"]\n >>> [\"justice\", \"community\"]\n \n uncached_db[\"favorite_foods\"]\n >>> None\n \n value = await uncached_db.aquery_tag(\"favorite_foods\", cache=True)\n \n assert value == [\"justice\", \"community\"]\n \n assert uncached_db[\"favorite_foods\"] == [\"justice\", \"community\"]\n \n \n # Metatags will be loaded, but their tags won't be ->\n \n assert type(uncached_db.exercise_routines) == AsyncDatabase\n \n uncached_db.exercise_routines[\"gardening\"]\n >>> None\n \n await uncached_db.exercise_routines.aquery_tag(\"gardening\", cache=True)\n >>> {\"days\": [\"moday\", \"wednesday\"]}\n \n uncached_db.exercise_routines[\"gardening\"]\n >>> {\"days\": [\"moday\", \"wednesday\"]}\n \n \n # But, tags can also be queried without caching their values, \n \n value = await uncached_db.exercise_routines.aquery_tag(\"swimming\")\n \n value\n >>> {\"days\": [\"thursday\", \"saturday\"]}\n \n uncached_db.exercise_routines[\"swimming\"]\n >>> None\n \n \n # However, changes to mutable values won't be transmitted to the\n \n # database if they aren't retrieved from the cache ->\n \n value[\"days\"].append(\"sunday\")\n \n value\n >>> {\"days\": [\"thursday\", \"saturday\", \"sunday\"]}\n \n await uncached_db.exercise_routines.aquery_tag(\"swimming\")\n >>> {\"days\": [\"thursday\", \"saturday\"]}\n \n \n_`Mirrors` ........................................ `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n \nDatabase mirrors allow users to make copies of all files within a database under new encryption keys. This is useful if users simply want to make backups, or if they'd like to update / change their database keys. \n \n.. code-block:: python\n \n # A unique login key / credentials are needed to create a new \n \n # database ->\n \n new_key = await acsprng()\n \n new_db = await AsyncDatabase(new_key)\n \n \n # Mirroring an existing database is done like this ->\n \n await new_db.amirror_database(db)\n \n assert (\n \n await new_db.aquery_tag(\"favorite_foods\") \n \n is await db.aquery_tag(\"favorite_foods\")\n \n )\n \n \n # If the user is just updating their database keys, then the old\n \n # database should be deleted ->\n \n await db.adelete_database()\n \n \n # Now, the new database can be saved to disk & given an appropriate \n \n # name ->\n \n async with new_db as db:\n \n pass\n \n\n_`Public Cryptographic Functions` ................. `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nAlthough databases handle encryption & decryption automatically, users may want to utilize their databases' keys to do custom cryptographic procedures manually. There are a few public functions available to users if they should want such functionality.\n\n\n_`Encrypt / Decrypt` .............................. `Table Of Contents`_\n************************************************************************\n\n.. code-block:: python\n\n # Either JSON serializable or bytes-type data can be encrypted ->\n\n json_plaintext = {\"some\": \"JSON data can go here...\"}\n \n bytes_plaintext = b\"some bytes plaintext goes here...\"\n \n token_plaintext = b\"some token data goes here...\"\n\n json_ciphertext = await db.ajson_encrypt(json_plaintext)\n\n bytes_ciphertext = await db.abytes_encrypt(bytes_plaintext)\n \n token_ciphertext = await db.amake_token(token_plaintext)\n\n\n # Those values can just as easily be decrypted ->\n\n assert json_plaintext == await db.ajson_decrypt(json_ciphertext)\n\n assert bytes_plaintext == await db.abytes_decrypt(bytes_ciphertext)\n \n assert token_plaintext == await db.aread_token(token_ciphertext)\n\n\n # Filenames may be added to classify ciphertexts. They also alter the \n\n # key material used during encryption in such a way, that without the\n\n # correct filename, the data cannot be decrypted ->\n\n filename = \"grocery-list\"\n\n groceries = [\"carrots\", \"taytoes\", \"rice\", \"beans\"]\n\n ciphertext = await db.ajson_encrypt(groceries, filename=filename)\n\n assert groceries == await db.ajson_decrypt(ciphertext, filename=filename)\n \n await db.ajson_decrypt(ciphertext, filename=\"wrong filename\")\n >>> \"InvalidSHMAC: Invalid StreamHMAC hash for the given ciphertext.\"\n\n\n\n # Time-based expiration of ciphertexts is also available for all \n\n # encrypted data this package produces ->\n\n from aiootp.asynchs import asleep\n\n\n await asleep(6)\n\n await db.ajson_decrypt(json_ciphertext, ttl=1)\n >>> \"TimestampExpired: Timestamp expired by <5> seconds.\"\n\n await db.abytes_decrypt(bytes_ciphertext, ttl=1)\n >>> \"TimestampExpired: Timestamp expired by <5> seconds.\"\n\n await db.aread_token(token_ciphertext, ttl=1)\n >>> \"TimestampExpired: Timestamp expired by <5> seconds.\"\n\n\n # The number of seconds that are exceeded may be helpful to know. In\n\n # which case, this is how to retrieve that integer value ->\n\n try: \n \n await db.abytes_decrypt(bytes_ciphertext, ttl=1)\n\n except db.TimestampExpired as error:\n\n assert error.expired_by == 5\n\n\n_`HMACs` .......................................... `Table Of Contents`_\n************************************************************************\n\nBesides encryption & decryption, databases can also be used to manually verify the authenticity of bytes-type data with HMACs.\n\n.. code-block:: python\n\n # Creating an HMAC of some data with a database is done this way ->\n\n data = b\"validate this data!\"\n\n hmac = await db.amake_hmac(data)\n\n await db.atest_hmac(hmac, data) # Runs without incident\n\n\n # Data that is not the same will be caught ->\n\n altered_data = b\"valiZate this data!\"\n\n await db.atest_hmac(hmac, altered_data)\n >>> \"InvalidHMAC: Invalid HMAC hash for the given data.\"\n \n\n # Any number of bytes-type arguments can be run thorugh the function, \n\n # the collection of items is canonically encoded automagically ->\n\n arbitrary_data = (b\"uid_\\x0f\\x12\", b\"session_id_\\xa1\")\n\n hmac = await db.amake_hmac(*arbitrary_data)\n \n await db.atest_hmac(hmac, *arbitrary_data) # Runs without incident\n\n\n # Additional qualifying information can be specified with the ``aad``\n\n # keyword argument ->\n\n from time import time\n\n timestamp = int(time()).to_bytes(8, \"big\")\n\n hmac = await db.amake_hmac(*arbitrary_data, aad=timestamp)\n \n await db.atest_hmac(hmac, *arbitrary_data)\n >>> \"InvalidHMAC: Invalid HMAC hash for the given data.\"\n\n await db.atest_hmac(hmac, *arbitrary_data, aad=timestamp) # Runs fine\n\n\n # This is most helpful for domain separation of the HMAC outputs.\n\n # Each distinct setting & purpose of the HMAC should be specified\n\n # & NEVER MIXED ->\n\n uuid = await db.amake_hmac(user_name, aad=b\"uuid\")\n\n hmac = await db.amake_hmac(user_data, aad=b\"data-authentication\")\n \n \n #\n\n\n\n\n_`Chunky2048 Cipher` .............................. `Table Of Contents`_\n------------------------------------------------------------------------\n\nThe ``Chunky2048`` cipher is the built from generators & SHA3-based key-derivation functions. It's designed to be easy to use, difficult to misuse & future-proof with large security margins. \n\n\n_`High-level Functions` .......................... `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nThese premade recipes allow for the easiest usage of the cipher.\n\n.. code-block:: python\n\n import aiootp\n \n \n cipher = aiootp.Chunky2048(key)\n \n \n # Symmetric encryption of JSON data ->\n \n json_data = {\"account\": 33817, \"names\": [\"queen b\"], \"id\": None}\n \n encrypted_json_data = cipher.json_encrypt(json_data, aad=b\"demo\")\n \n decrypted_json_data = cipher.json_decrypt(\n \n encrypted_json_data, aad=b\"demo\", ttl=120\n \n )\n \n assert decrypted_json_data == json_data\n \n \n # Symmetric encryption of binary data ->\n \n binary_data = b\"some plaintext data...\"\n \n encrypted_binary_data = cipher.bytes_encrypt(binary_data, aad=b\"demo\")\n \n decrypted_binary_data = cipher.bytes_decrypt(\n \n encrypted_binary_data, aad=b\"demo\", ttl=30\n \n )\n \n assert decrypted_binary_data == binary_data\n \n \n # encrypted URL-safe Base64 encoded tokens ->\n \n token_data = b\"some plaintext token data...\"\n \n encrypted_token_data = cipher.make_token(token_data, aad=b\"demo\")\n \n decrypted_token_data = cipher.read_token(\n \n encrypted_token_data, aad=b\"demo\", ttl=3600\n \n )\n \n assert decrypted_token_data == token_data\n\n\n_`High-level Generators` .......................... `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nWith these generators, the online nature of the Chunky2048 cipher can be utilized. This means that any arbitrary amount of data can be processed in streams of controllable, buffered chunks. These streaming interfaces automatically handle message padding & depadding, ciphertext validation & detection of out-of-order message blocks.\n\nEncryption:\n\n.. code-block:: python\n \n from aiootp import AsyncCipherStream\n \n \n # Let's imagine we are serving some data over a network ->\n\n receiver = SomeRemoteConnection(session).connect()\n\n\n # This will manage encrypting a stream of data ->\n\n stream = await AsyncCipherStream(key, aad=session.transcript)\n\n\n # We'll have to send the salt & iv in some way ->\n\n receiver.transmit(salt=stream.salt, iv=stream.iv)\n\n\n # Now we can buffer the plaintext we are going to encrypt ->\n\n for plaintext in receiver.upload.buffer(4 * stream.PACKETSIZE):\n\n await stream.abuffer(plaintext)\n\n\n # The stream will now produce encrypted blocks of ciphertext\n\n # as well as the block ID which authenticates each block ->\n\n async for block_id, ciphertext in stream:\n\n # The receiver needs both the block ID & ciphertext ->\n\n receiver.send_packet(block_id + ciphertext)\n\n\n # Once done with buffering-in the plaintext, the ``afinalize`` \n\n # method is called so the remaining encrypted data will be \n\n # flushed out of the buffer to the user ->\n\n async for block_id, ciphertext in stream.afinalize():\n\n receiver.send_packet(block_id + ciphertext)\n\n\n # Here we can give an optional check of further authenticity, \n\n # also cryptographically asserts the stream is finished ->\n\n receiver.transmit(shmac=await stream.shmac.afinalize())\n\n\nDecryption / Authentication:\n\n.. code-block:: python\n \n from aiootp import AsyncDecipherStream\n\n \n # Here let's imagine we'll be downloading some data ->\n\n source = SomeRemoteConnection(session).connect()\n\n\n # The key, salt, aad & iv must be the same for both parties ->\n\n stream = await AsyncDecipherStream(\n\n key, salt=source.salt, aad=session.transcript, iv=source.iv\n\n )\n\n # The downloaded ciphertext will now be buffered & the stream\n\n # object will produce the plaintext ->\n\n for ciphertext in source.download.buffer(4 * stream.PACKETSIZE):\n\n # Here stream.shmac.InvalidBlockID is raised if an invalid or\n\n # out-of-order block is detected within the last 4 packets ->\n\n await stream.abuffer(ciphertext) \n\n\n # If authentication succeeds, the plaintext is produced ->\n\n async for plaintext in stream:\n\n yield plaintext\n\n\n # After all the ciphertext is downloaded, ``afinalize`` is called\n\n # to finish processing the stream & flush out the plaintext ->\n\n async for plaintext in stream.afinalize():\n\n yield plaintext\n\n\n # An optional check for further authenticity which also\n\n # cryptographically asserts the stream is finished ->\n\n await stream.shmac.afinalize()\n\n await stream.shmac.atest_shmac(source.shmac)\n\n\n #\n\n\n\n\n_`Passcrypt` .............................. `Table Of Contents`_\n------------------------------------------------------------------------\n\nThe ``Passcrypt`` algorithm is a data independent memory & computationally hard password-based key derivation function. It's built from a single primitive, the SHAKE-128 extendable output function from the SHA-3 family. Its resource costs are measured by three parameters: ``mb``, which represents an integer number of Mibibytes (MiB); ``cpu``, which is a linear integer measure of computational complexity & the number of iterations of the algorithm over the memory cache; and ``cores``, which is an integer which directly assigns the number of separate processes that will be pooled to complete the algorithm. The number of bytes of the output tag are decided by the integer ``tag_size`` parameter. And, the number of bytes of the automatically generated ``salt`` are decided by the integer ``salt_size`` parameter.\n\n\n_`Hashing & Verifying Passphrases` .......................... `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\nBy far, the dominating measure of difficulty for ``Passcrypt`` is determined by the ``mb`` Mibibyte memory cost. It's recommended that increases to desired difficulty are first translated into higher ``mb`` values, where resource limitations of the machines executing the algorithm permit. If more difficulty is desired than can be obtained by increasing ``mb``, then increases to the ``cpu`` parameter should be used. The higher this parameter is the less likely an adversary is to benefit from expending less than the intended memory cost, & increases the execution time & complexity of the algorithm. The final option that should be considered, if still more difficulty is desired, is to lower the ``cores`` parallelization parameter, which will just cause each execution to take longer to complete.\n\n.. code-block:: python\n \n from aiootp import Passcrypt, hash_bytes\n\n\n # The class accepts an optional (but recommended) static \"pepper\"\n\n # which is applied as additional randomness to all hashes computed\n\n # by the class. It's a secret random bytes value of any size that is\n\n # expected to be stored somewhere inaccessible by the database which\n\n # contains the hashed passphrases ->\n\n with open(SECRET_PEPPER_PATH, \"rb\") as pepper_file:\n\n Passcrypt.PEPPER = pepper_file.read()\n\n\n # when preparing to hash passphrases, it's a good idea to use any &\n\n # all of the static data / credentials available which are specific \n\n # to the context of the registration ->\n\n APPLICATION = b\"my-application-name\"\n\n PRODUCT = b\"the-product-being-accessed-by-this-registration\"\n\n STATIC_CONTEXT = [APPLICATION, PRODUCT, PUBLIC_CERTIFICATE]\n\n\n # If the same difficulty settings are going to be used for every \n\n # hash, then a ``Passcrypt`` instance can be initialized to\n\n # automatically pass those static settings ->\n\n pcrypt = Passcrypt(mb=1024, cpu=2, cores=8) # 1 GiB, 8 cores\n\n\n # Now that the static credentials / settings are ready to go, we\n\n # can start hashing any user information that arrives ->\n\n username = form[\"username\"].encode()\n\n passphrase = form[\"passphrase\"].encode()\n\n email_address = form[\"email_address\"].encode()\n\n\n # The ``hash_bytes`` function can then be used to automatically\n\n # encode then hash the multi-input data so as to prevent the chance\n\n # of canonicalization (&/or length extension) attacks ->\n\n aad = hash_bytes(*STATIC_CONTEXT, username, email_address)\n\n hashed_passphrase = pcrypt.hash_passphrase(passphrase, aad=aad)\n\n assert type(hashed_passphrase) is bytes\n\n assert len(hashed_passphrase) == 38\n\n\n # Later, a hashed passphrase can be used to authenticate a user ->\n\n untrusted_username = form[\"username\"].encode()\n\n untrusted_passphrase = form[\"passphrase\"].encode()\n\n untrusted_email_address = form[\"email_address\"].encode()\n\n aad = hash_bytes(\n\n *STATIC_CONTEXT, untrusted_username, untrusted_email_address\n\n )\n\n try:\n\n pcrypt.verify(\n\n hashed_passphrase, untrusted_passphrase, aad=aad, ttl=3600\n\n )\n\n except pcrypt.InvalidPassphrase as auth_fail:\n\n # If the passphrase does not hash to the same value as the \n\n # stored hash, then this exception is raised & can be handled\n\n # by the application ->\n\n app.post_mortem(error=auth_fail)\n\n except pcrypt.TimestampExpired as registration_expired:\n\n # If the timestamp on the stored hash was created more than\n\n # ``ttl`` seconds before the current time, then this exception\n\n # is raised. This is helpful for automating registrations which\n\n # expire after a certain amount of time, which in this case was\n\n # 1 hour ->\n\n app.post_mortem(error=registration_expired)\n\n else:\n\n # If no exception was raised, then the user has been authenticated\n\n # by their passphrase, username, email address & the context of\n\n # the registration ->\n\n app.login_user(username, email_address)\n\n\n # \n\n\n_`Passcrypt Algorithm Overview` .......................... `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nBy being secret-independent, ``Passcrypt`` is resistant to side-channel attacks. This implementation is also written in pure python. Significant attention was paid to design the algorithm so as to suffer minimally from the performance inefficiencies of python, since doing so would help to equalize the cost of computation between regular users & dedicated attackers with custom hardware / software. Below is a diagram that depicts how an example execution works:\n\n.. code-block:: python\n\n #\n ___________________ # of rows ___________________\n | |\n | initial memory cache |\n | row # of columns == 2 * max([1, cpu // 2]) |\n | | # of rows == \u23081024*1024*mb/168*columns\u2309 |\n v v v\n column|---'-----------------------------------------'---| the initial cache\n column|---'-----------------------------------------'---| of size ~`mb` is\n column|---'-----------------------------------------'---| built very quickly\n column|---'-----------------------------------------'---| using SHAKE-128.\n column|---'-----------------------------------------'---| each (row, column)\n column|---'-----------------------------------------'---| coordinate holds\n column|---'-----------------------------------------'---| one element of\n column|---'-----------------------------------------'---| 168-bytes.\n ^\n |\n reflection row\n <- |\n |--------------------'-------'--------------------| each row is\n |--------------------'-------'--------------------| hashed then has\n |--------------------'-------'--------------------| a new 168-byte\n |--------------------'-------'--------------------| digest overwrite\n |--------------------'-------'--------------------| the current pointer\n |--------------------'-------'--------------------| in an alternating\n |--------------------Xxxxxxxx'xxxxxxxxxxxxxxxxxxxx| sequence, first at\n |oooooooooooooooooooo'oooooooO--------------------| the index, then at\n | -> its reflection.\n index\n\n\n |--'-------------------------------------------'--| this continues\n |--'-------------------------------------------'--| until the entire\n |--'-------------------------------------------Xxx| cache has been\n |ooO-------------------------------------------'--| overwritten.\n |xx'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'xx| a single `shake_128`\n |oo'ooooooooooooooooooooooooooooooooooooooooooo'oo| object (H) is used\n |xx'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'xx| to do all of the\n |oo'ooooooooooooooooooooooooooooooooooooooooooo'oo| hashing.\n | -> <- |\n index reflection\n\n\n |xxxxxxxxxxx'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| finally, the whole\n |ooooooooooo'ooooooooooooooooooooooooooooooooooooo| cache is quickly\n |xxxxxxxxxxx'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| hashed `cpu` + 2\n |ooooooooooo'ooooooooooooooooooooooooooooooooooooo| number of times.\n |Fxxxxxxxxxx'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| after each pass an\n |foooooooooo'ooooooooooooooooooooooooooooooooooooo| 84-byte digest is\n |fxxxxxxxxxx'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx| inserted into the\n |foooooooooo'ooooooooooooooooooooooooooooooooooooo| cache, ruling out\n | -> hashing state cycles.\n | hash cpu + 2 # of times Then a `tag_size`-\n v byte tag is output.\n H(cache)\n\n tag = H.digest(tag_size)\n\n #\n\n\n\n\n_`X25519 & Ed25519` ............................... `Table Of Contents`_\n------------------------------------------------------------------------\n\nAsymmetric curve 25519 tools are available from these high-level interfaces over the ``cryptography`` package.\n\n\n_`X25519` ......................................... `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nElliptic curve 25519 diffie-hellman exchange protocols.\n\n.. code-block:: python\n\n from aiootp import X25519, DomainKDF, GUID, Domains\n\n\n # Basic Elliptic Curve Diffie-Hellman ->\n\n guid = GUID().new()\n\n my_ecdhe_key = X25519().generate()\n\n yield guid, my_ecdhe_key.public_bytes # send this to Bob\n\n raw_shared_secret = my_ecdhe_key.exchange(bobs_public_key)\n\n shared_kdf = DomainKDF( # Use this to create secret shared keys\n\n Domains.ECDHE,\n\n guid,\n\n bobs_public_key,\n\n my_ecdhe_key.public_bytes,\n\n key=raw_shared_secret,\n\n )\n \n \n # Triple ECDH Key Exchange client initialization ->\n \n with ecdhe_key.dh3_client() as exchange:\n \n response = internet.post(exchange())\n \n exchange(response)\n \n clients_kdf = exchange.result()\n\n\n # Triple ECDH Key Exchange for a receiving peer ->\n \n identity_key, ephemeral_key = client_public_keys = internet.receive()\n \n server = ecdhe_key.dh3_server(identity_key, ephemeral_key)\n \n with server as exchange:\n \n internet.post(exchange.exhaust())\n \n servers_kdf = exchange.result()\n \n\n # Success! Now both the client & server peers share an identical\n \n # ``DomainKDF`` hashing object to create shared keys ->\n\n assert (\n\n clients_kdf.sha3_512(context=b\"test\") \n\n == servers_kdf.sha3_512(context=b\"test\")\n\n )\n \n \n_`Ed25519` ........................................ `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nEdwards curve 25519 signing & verification.\n\n.. code-block:: python\n\n from aiootp import Ed25519\n \n \n # In a land, long ago ->\n \n alices_key = Ed25519().generate()\n \n internet.send(alices_key.public_bytes)\n \n\n # Alice wants to sign a document so that Bob can prove she wrote it.\n \n # So, Alice sends the public key bytes of the key she wants to\n \n # associate with her identity, the document & the signature ->\n \n document = b\"DesignDocument.cad\"\n \n signed_document = alices_key.sign(document)\n\n message = {\n \"document\": document,\n \"signature\": signed_document,\n \"public_key\": alices_key.public_bytes,\n }\n\n internet.send(message)\n \n\n # In a land far away ->\n \n alices_message = internet.receive()\n\n # Bob sees the message from Alice! Bob already knows Alice's public\n \n # key & she has reason believe it is genuinely Alice's. So, she'll\n \n # import Alice's known public key to verify the signed document ->\n \n assert alices_message[\"public_key\"] == alices_public_key\n \n alice_verifier = Ed25519().import_public_key(alices_public_key)\n \n alice_verifier.verify(\n alices_message[\"signature\"], alices_message[\"document\"]\n )\n \n internet.send(b\"Beautiful work, Alice! Thanks ^u^\")\n\nThe verification didn't throw an exception! So, Bob knows the file was signed by Alice.\n \n \n \n \n_`Comprende` ...................................... `Table Of Contents`_\n------------------------------------------------------------------------\n\nThis magic with generators is made simple with the ``comprehension`` decorator. It wraps them in ``Comprende`` objects with access to myriad data processing pipeline utilities right out of the box.\n\n\n_`Synchronous Generators` ......................... `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n.. code-block:: python\n\n from aiootp.gentools import comprehension\n \n \n @comprehension()\n \n def gen(x: int, y: int):\n \n z = yield x + y\n \n return x * y * z\n \n \n # Drive the generator forward with a context manager ->\n \n with gen(x=1, y=2) as example:\n \n z = 5\n \n \n # Calling the object will send ``None`` into the coroutine by default ->\n \n sum_of_x_y = example()\n \n assert sum_of_x_y == 3\n\n\n # Passing ``z`` will send it into the coroutine, cause it to reach the \n \n # return statement & exit the context manager ->\n \n example(z)\n \n \n # The result returned from the generator is now available ->\n \n product_of_x_y_z = example.result()\n \n assert product_of_x_y_z == 10\n \n \n # Here's another example ->\n \n @comprehension()\n \n def one_byte_numbers():\n \n for number in range(256):\n \n yield number\n \n \n # Chained ``Comprende`` generators are excellent inline data processors ->\n \n base64_data = one_byte_numbers().int_to_bytes(1).to_base64().list()\n \n # This converted each number to bytes then base64 encoded them into a list.\n\n\n # We can wrap other iterables to add functionality to them ->\n\n @comprehension()\n \n def unpack(iterable):\n \n for item in iterable:\n \n yield item\n\n\n # This example just hashes each output then yields them\n\n for digest in unpack(base64_data).sha3_256():\n \n print(digest)\n\n\n_`Asynchronous Generators` ........................ `Table Of Contents`_\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nAsync ``Comprende`` coroutines have almost exactly the same interface as synchronous ones.\n\n.. code-block:: python\n\n from aiootp.asynchs import asleep\n\n from aiootp.gentools import Comprende, comprehension\n\n\n @comprehension()\n \n async def gen(x: int, y: int):\n \n # Because having a return statement in an async generator is a\n \n # SyntaxError, the return value is expected to be passed into\n \n # Comprende.ReturnValue, and then raised to propagate upstream. \n\n # It's then available from the instance's ``aresult`` method ->\n \n z = yield x + y\n \n raise Comprende.ReturnValue(x * y * z)\n \n \n # Drive the generator forward.\n \n async with gen(x=1, y=2) as example:\n \n z = 5\n \n \n # Awaiting the ``__call__`` method will send ``None`` into the\n\n # coroutine by default ->\n \n sum_of_x_y = await example()\n \n assert sum_of_x_y == 3\n\n\n # Passing ``z`` will send it into the coroutine, cause it to reach the\n \n # raise statement which will exit the context manager gracefully ->\n \n await example(z)\n \n \n # The result returned from the generator is now available ->\n \n product_of_x_y_z = await example.aresult()\n \n assert product_of_x_y_z == 10\n \n \n # Let's see some other ways async generators mirror synchronous ones ->\n \n @comprehension()\n \n async def one_byte_numbers():\n\n # It's probably a good idea to pass control to the event loop at\n\n # least once or twice, even if async sleeping after each iteration\n\n # may be excessive when no real work is being demanded by range(256).\n\n # This consideration is more or less significant depending on the \n\n # expectations placed on this generator by the calling code.\n\n await asleep()\n \n for number in range(256):\n \n yield number\n\n await asleep()\n \n \n # This is asynchronous data processing ->\n \n base64_data = await one_byte_numbers().aint_to_bytes(1).ato_base64().alist()\n \n # This converted each number to bytes then base64 encoded them into a list.\n\n\n # We can wrap other iterables to add asynchronous functionality to them ->\n\n @comprehension()\n \n async def unpack(iterable):\n \n for item in iterable:\n \n yield item\n\n\n # Want only the first twenty results? ->\n\n async for digest in unpack(base64_data).asha3_256()[:20]:\n \n # Then you can slice the generator.\n \n print(digest)\n \n \n # Users can slice generators to receive more complex output rules, like:\n \n # Getting every second result starting from the 4th result to the 50th ->\n \n async for result in unpack(base64_data)[3:50:2]:\n \n print(result)\n\n\n # Although, negative slice numbers are not supported.\n\n``Comprende`` generators have loads of tooling for users to explore. Play around with it and take a look at the other chainable generator methods in ``aiootp.Comprende.lazy_generators``.\n\n\n\n\n_`Module Overview` ................................ `Table Of Contents`_\n------------------------------------------------------------------------\n\nHere's a quick overview of this package's modules:\n\n\n.. code-block:: python\n\n import aiootp\n \n \n # Commonly used constants, datasets & functionality across all modules ->\n \n aiootp.commons\n \n \n # The basic utilities & abstractions of the package's architecture ->\n \n aiootp.generics\n \n \n # A collection of the package's generator utilities ->\n \n aiootp.gentools\n \n \n # This module is responsible for providing entropy to the package ->\n \n aiootp.randoms\n \n \n # The high & low level abstractions used to implement the Chunky2048 cipher ->\n \n aiootp.ciphers\n \n \n # The higher-level abstractions used to create / manage key material ->\n \n aiootp.keygens\n \n \n # Common system paths & the ``pathlib.Path`` utility ->\n \n aiootp.paths\n \n \n # Global async / concurrency functionalities & abstractions ->\n \n aiootp.asynchs\n \n \n #\n\n\n\n\n_`FAQ` ............................................ `Table Of Contents`_\n========================================================================\n\n\n**Q: What is the one-time pad?**\n\nA: It's a cipher which provides an information theoretic guarantee of confidentiality. It's typically thought to be too cumbersome a cipher for generalized application because it conveys strict, and well, cumbersome, requirements onto its users. The need for its keys to be at least as large as all the messages it's ever used to encrypt is one such requirement. Our goal is to design a cipher which immitates the one-time pad through clever algorithms, in such a way as to minimize its inconveniences & still provide some form of information theoretic confidentiality guarantees or, at a minimum, be able to make non-trivial statements about its security against even computationally unbounded adversaries. In this effort, we've built what we hope to be a candidate cipher, which we've called ``Chunky2048``.\n\n\n**Q: How fast is this ``Chunky2048`` cipher?** \n\nA: Well, because it relies on ``hashlib.shake_128`` hashing to build key material streams, it's rather efficient. It can process about 24 MB/s on a ~1.5 GHz core for both encrypting & decrypting. This is still slow relative to other stream ciphers, but this package is written in pure Python & without hardware optimizations. Using SHA3 ASICs, specific chipset instructions, or a lower-level language implementation, could make this algorithm competitively fast.\n\n\n**Q: What size keys does the ``Chunky2048`` cipher use?** \n\nA: It's been designed to work with any size of key >= 64 bytes. \n\n\n**Q: What's up with the ``AsyncDatabase`` / ``Database``?**\n\nA: The idea is to create an intuitive, pythonic interface to a transparently encrypted and decrypted persistence tool that also cryptographically obscures metadata. It's designed to persist raw bytes or JSON serializable data, which gives it native support for some of the most important basic python datatypes. It's still a work in progress, albeit a very nifty one.\n\n\n**Q: Why are the modules transformed into ``Namespace`` objects?**\n\nA: We overwrite our modules in this package to have a more fine-grained control over what part of the package's internal state is exposed to users & applications. The goal is make it more difficult for users to inadvertently jeopardize their security tools, & minimize the attack surface available to adversaries. The ``Namespace`` class also makes it easier to coordinate and decide the library's UI/UX across the package.\n\n\n\n\n_`Changelog` ...................................... `Table Of Contents`_\n========================================================================\n\n\n\nChanges for version 0.22.0 \n---------------------------\n\n(Major Rewrite: Backwards Incompatible)\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n\nSecurity Advisory:\n^^^^^^^^^^^^^^^^^^\n\n- The top-level ``(a)csprng`` functions were found to be unsafe in concurrent code, leading to the possibilty of producing identical outputs from distinct calls if run in quick succession from concurrently running threads & coroutines. The classification of this vulnerability is severe because: 1) users should be able to expect the output of a 64-byte cryptographically secure pseudo-random number generator to always produce unique outputs; and, 2) much of the package utilizes them to produce cryptographic material. This vulnerability does not effect users of the library which are not running it in multiple concurrent threads or coroutines. The vulnerability has been patched & all users are **highly encouraged** to upgrade to v0.22.0+.\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Support for python 3.6 was dropped. The package now supports python versions 3.7+.\n- **Chunky2048**: A new version of the cipher has been developed which\n implements algorithms & interfaces that offer improvements in multiple\n regards: smaller size overhead of ciphertexts, faster execution time\n for large messages & large keys, more robust salt reuse/misue resistance,\n fewer aspects harming deniability & better domain separation.\n Many of the changes are described here:\n\n - The ``(a)bytes_keys`` generators were updated to use ``shake_128``-based KDF objects instead of ``sha3_512``, yielding 256-bytes on each iteration instead of 128, now requiring only a single iteration to produce a keystream key for each block, instead of two. This choice was made during the process of analyzing the use of the user's encryption `key` to seed the `seed_kdf` on each iteration. We wanted to stop doing that essentially, because it slowed down the cipher too much when used with large keys. And because it seems like a bad idea to use the same key repeatedly while also not incorporating the uniqueness or entropy from the message's `salt`, `siv` or `aad`. \n\n But still, we somehow wanted to come up with an idea which could efficiently & continually extract entropy from the user `key` if it did happen to be large. An answer came in the form of expanding on an earlier implemented idea which used the key multiple times to create unique seeds during initialization. In this case, however, instead of creating unique seeds with the single `seed_kdf`, each of the three KDFs & the MAC object used by the cipher will be given the whole `key` once at initialization, with proper domain separation, & including the message `salt` & `aad` (The `siv` can't be used because its creation happens after initialization during encryption). This gives each of their (SHA3) 200-byte internal states independent access to the full entropy of the `key`.\n\n Then, the problem was that, by using ``sha3_512`` internally, a maximum of 64-bytes of entropy could be communicated between KDFs at each round (and only 32-bytes from the ``StreamHMAC`` (`shmac`) object's ``sha3_256`` MAC). But the blocksize of each round is 256-bytes. So, the idea became to attempt to *communicate* more entropy between the KDFs & MAC each round than there exists possible messages in the message space of each round. It seems plausible, that by only assuming the independence of each of the KDFs / MAC & that they can indeed `efficiently pass entropy` to one another, that for large keys we could argue the relevant key space is that of the 800-byte internal state of the cipher at each round (which happens to be more than three times the size of the message space of each round). This is to say, we conjecture, that by `efficiently communicating more entropy` from *independent sources* than there exists *possible messages*, & in fact incorporating the entropy of *each message block* into the cipher's state at the start of *each round*, that the entropy of the internal keyspace is continually being refreshed in a way which is negligibly distinguishable from using a fresh random key each round the length of the blocksize. This seems like at least a feasible way to begin the argument that it is possible to meaningfully relate the information theoretic security of the one-time pad to a pseudo one-time pad in a measurable way.\n\n `Efficiently Pass Entropy`: By this we mean, the rate of bits extracted from one state object, to the rate of bits of actual entropy absorbed by a receiveing state object, up to its XORable state size, being different by only a negligible amount. Here, we can conservatively assume the limit of this efficiency is the XORable state size, since we know that in the ideal setting, XORing `n` uniform random bits with an unknown message of <= `n` bits is perfectly hiding, which implies perfectly efficient conveyance of entropy. By using ``shake_128`` as each of the cipher's state objects, & its larger rate of 168-bytes, more than twice the number of bytes can be passed to & extracted from each, per round & per call to their internal `f` permutation, as compared with ``sha3_512``. `If they can efficiently pass entropy`, then any secret state exposed by the `left_kdf` or `right_kdf` in the creation of ciphertext, can then be efficiently displaced by the introduction of new entropy from the other state objects. This follows from the theory that a finite sized pool of entropy which is already maximally filled with entropy, cannot incorporate more entropy without fundamentally erasing internal information. From this we arrived at the new design for ``Chunky2048``. In this new design, the `shmac` feeds 168-bytes to the `seed_kdf`, the `seed_kdf` creates 336-bytes to feed 168-bytes each to the `left_kdf` & `right_kdf`, the `left_kdf` & `right_kdf` each produce 128-byte keys which XOR the 256-byte plaintext, then this ciphertext feeds the `shmac` & the cycle repeats.\n\n More work needs to be done to formalize these definitions & analyze their properties. We would be grateful for any help from those with expertise in formal proofs of security in tearing apart this design as we move closer to the first stable release of the package.\n\n - The ``SyntheticIV`` class' algorithm has been updated as a result of analyzing how we could improve the salt reuse / misuse resistance of the cipher without attesting to plaintext contents in the form of an `siv` attached to ciphertexts. This plaintext attestation worked counter to our goal of wanting to be able to say something non-trivial about the key-deniability of the cipher. It was noticed that the plaintext padding already incorporated an 8-byte timestamp (now reduced to 4-bytes) & 16-bytes of ephemeral randomness as part of the prepended inner-header, & that these values were not at all used to seed the cipher's state during decryption. Instead a keyed-hash was calculated over the first block of plaintext during encryption to create the 24-byte `siv`. But, this is actually `less effective` at producing salt reuse / misuse resistance than using the timestamp & ephemeral randomness directly in seeding the `seed_kdf`, because the timestamp is a unique & global counter that does not suffer from collisions. This understanding came while also trying to find a good use for the initial `primer_key` generated by the keystream generator when sending in the first obligatory `None` value. In the previous version it was used to initialize the `shmac`, but now that the `shmac` would be initialized directly with the user `key`, it was searching for a use. So the idea was to pair them. \n\n The new 256-byte `primer_key` would be XORed with the 256-byte first block of plaintext to mask the inner-header. The unmasked inner-header & 148-bytes of the `shmac`'s digest will seed the keystream, & the freshly seeded keystream output would be truncated to XOR the part of the masked plaintext which doesn't include the inner-header. There's no need now to attach the `siv` to the ciphertext. Instead, during decryption, the decipher algorithm has access to the inner-header, because it has access to the `primer_key` & the masked inner-header. The actual plaintext contents of the first block are only accessible after unmasking the inner-header & seeding the keystream. This combination alone of protection from a timestamp & 16-bytes of randomness should give a salt reuse / misuse resistance of at least `~2 ^ 64 messages` **per second**!\n\n However, even with this new scheme, it would still be problematic to repeat a combination of `key`, `salt` & `aad`, since it would leak the XORs of timestamp information. With all of this in mind, the new formulation would include a 16-byte `salt` & a newly introduced 16-byte `iv`, both of which are attached to ciphertexts. This is a header size reduction of 16-bytes, since prior `salt` & `siv` sizes were 24-bytes each. The difference between the `salt` & `iv` is that the `salt` is available for the user to choose, but the `iv` is **always** generated randomly. Since the `iv` isn't dependent on message data the way that the `siv` was, it too can now be incorporated into all of the state objects during initialization. The `iv` ensures that even if a `key`, `salt` & `aad` tuple repeats, the timestamp is still protected. Below is a diagram of the procedure:\n\n\n .. code-block:: python\n\n #\n _____________________________________\n | |\n | Algorithm Diagram: Encryption |\n |_____________________________________|\n ------------------------------------------------------------------ #\n | inner-header | first block of plaintext | #\n | timestamp | siv-key | | #\n | 4-bytes | 16-bytes | 236-bytes | #\n ------------------------------------------------------------------ #\n |---------------------- entire first block ------------------------| #\n | #\n | #\n first 256-byte keystream key ----\u2295 #\n | #\n | #\n V #\n masked plaintext block #\n ------------------------------------------------------------------ #\n | masked inner-header | first block of masked plaintext | #\n ------------------------------------------------------------------ #\n |----- the 236-byte masked plaintext -----| #\n | #\n | #\n siv = inner-header + shmac.digest(148) | #\n keystream(siv)[10:246] -----------------------\u2295 #\n | #\n | #\n V #\n ------------------------------------------------------------------ #\n | masked inner-header | first block of ciphertext | #\n ------------------------------------------------------------------ #\n\n\n _____________________________________ \n | |\n | Algorithm Diagram: Decryption |\n |_____________________________________|\n ------------------------------------------------------------------ #\n | masked inner-header | first block of ciphertext | #\n ------------------------------------------------------------------ #\n |---------------------- entire first block ------------------------| #\n | #\n | #\n first 256-byte keystream key ----\u2295 #\n | #\n | #\n V #\n unmasked ciphertext block #\n ------------------------------------------------------------------ #\n | inner-header | first block of unmasked ciphertext | #\n ------------------------------------------------------------------ #\n |--- the 236-byte unmasked ciphertext ----| #\n | #\n | #\n siv = inner-header + shmac.digest(148) | #\n keystream(siv)[10:246] -----------------------\u2295 #\n | #\n | #\n V #\n ------------------------------------------------------------------ #\n | inner-header | first block of plaintext | #\n | timestamp | siv-key | | #\n | 4-bytes | 16-bytes | 236-bytes | #\n ------------------------------------------------------------------ #\n \n #\n\n - The ``Padding`` class has seen some changes. Firstly, the 8-byte timestamp in the inner-header was reduced to 4-bytes. Furthermore, to get the full 136 years out of the 4-byte timestamps, the epoch used to calculate them was changed to unix timestamp `1672531200` (Sun, 01 Jan 2023 00:00:00 UTC). This is the new default `0` date for the package's timestamps. This saves some space & aims to provided fewer bits of confirmable attestation & correlation in proof games which simulate attacks on the key-deniability of the cipher. To explain: the plaintext padding includes random padding. That padding is intended to leave an adversary which attempts to brute force a ciphertext's encryption `key`, even with unbounded computational resources, in a state where it cannot decide with better accuracy than random chance between the exponentially large number of keys which create the same `shmac` tag (the variable `keyspace` is much larger than the 32-byte tag) with their accompanying exponentially large number of `plausible` plaintexts (any `reasonable` plaintext with any variable length random padding between 16 & 272 bytes), & the actual user `key` & plaintext.\n\n We also got rid of the use of a `padding_key` to indicate the end of a plaintext message. It used to be sliced off the `primer_key`, but the `primer_key` has a new use now. Also, the `padding_key` was another form of plaintext / key attestation harming deniability that we wanted to get rid of. Instead, a simpler method is now employed: The final byte of the final block of padded plaintext is a number which tells the decryptor exactly how many bytes of random padding were added to the plaintext to fill the block. This saves a lot of space, is simpler, minimizes unnecessary key attestation, & eliminates the need for the ``Padding`` class to know anything about user secrets in order to do the padding, which is an improvment all around.\n\n- New ``(Async)CipherStream`` & ``(Async)DecipherStream`` classes were introduced which allow users to utilize the online nature of the ``Chunky2048`` cipher, ciphering & deciphering data in bufferable chunks, without needing to know about or instantiate all of the low-level classes. They automatically handle the required plaintext padding, ciphertext authentication, & detection of out-of-order message blocks. This greatly simplifies the safe usage of ``Chunky2048`` in online mode, provides robustness, & gets rid of the need for users to worry about the dangers of release of unverified plaintexts.\n\n- The ``Passcrypt`` algorithm was redesigned to be data-independent, more efficiently acheive its security goals, & allow for more compact hashes which include its difficulty settings metadata. The `kb` parameter was changed to `mb`, & now measures Mibibytes (MiB). A new `cores` parallelization parameter was added, which indicates the number of parallel processes to use to complete the procedure. And the `cpu` parameter now measures the number of iterations over the memory cache that are done, as well as the computational complexity of the algorithm. ``Passcrypt`` now uses ``shake_128`` instead of ``sha3_512`` internally. This also allows for users to specify a ``tag_size`` number of bytes to produce as an output tag. A ``salt_size`` parameter can now also be supplied to the ``(a)hash_passphrase`` methods. The ``(a)hash_passphrase`` methods now produce raw-bytes outputs & the ``(a)hash_passphrase_raw`` & ``(a)verify_raw`` methods were removed. ``(a)verify`` methods now also accept ``range``-type objects as ``mb_allowed``, ``cpu_allowed``, & ``cores_allowed`` keyword argument inputs. These range objects can be used to specify the exact amount of resources which the user allows for difficulty settings, which can mitigate adversarial (or unintentional) DOS attacks on machines doing hash verification.\n\n- Type annotations were added to most of the library, including return types, which were completely neglected in prior versions. They are still not functioning with mypy, & are serving right now as documentation & auto-complete helpers.\n\n- Many unnecesssary, low-level or badly designed features, functions & classes were either deleted or pulled into private namespaces, along with major reorganization & cleanup of the codebase. The tangled mess of internal module imports was also cleaned up. The goal is to provide access to only the highest level, simplest, & safest by default interfaces which can actually help users in their data processing & cryptographic tasks. These changes aim to improve maintainability, readability, correctness & safety.\n\n- New top-level ``(a)hash_bytes`` functions were added to the package, which accept an unlimited number bytes-type inputs as positional arguments & automatically canonically encode all inputs before being hashed (which aims to prevent canonicalization attacks & length-extension attacks). A ``key`` keyword-only argument can also be supplied to optionally produce keyed hashes.\n\n- A new top-level ``GUID`` class was added. It creates objects which produce variable length, obfuscated, pseudo-random bytes-type globally unique identifiers based on a user-defined integer `node_number`, a user-defined uniform bytes `salt`, a nanosecond `timestamp`, random `entropy` bytes & a 1-byte `counter`. The benefits of its novel design explained: **1)** the namespace separation of user-defined salts (like name-based uuids); **2)** guaranteed output uniqueness for all instances using the same `salt` & `node_number` which occur on a different nanosecond (like time-based uuids, but with higher precision); **3)** guaranteed output uniqueness between all instances which use the same `salt` but a different `node_number`, even if produced on the same nanosecond; **4)** guaranteed output uniqueness for any unique instance using the same `salt` & `node_number` if it produces 256 or fewer outputs every nanosecond; **5)** probabilistic output uniqueness for any unique instance using the same `salt` & `node_number` if it produces >256 outputs per-nanosecond, exponentially proportional to the number of random `entropy` bytes (which in turn are proportional to the output size of the GUIDs); **6)** output invertability, meaning outputs can be unmasked & sorted according to `timestamp`, `node_number` & `counter`; **7)** random-appearing outputs, with the marginal amount of privacy which can be afforded by obfuscated affine-group operations. Admittedly, point **7)** still *leaves much room for improvement*, as the privacy of the design could instead be ensured by strong hardness assumptions given by other types of invertible permutations or group operations. The goal was to create something efficient (below 3\u00b5s per guid), which met the above criterion, & that produced output bit sequences which passed basic randomness tests. We'd be excited to accept pull requests which use strong invertable permutations or group operations that are also about as efficient, & that for `n`-byte declared output sizes, outputs do not repeat for fewer than ~256 ** `n` sequential input values.\n\n- The top-level ``DomainKDF`` class now also creates KDF objects which automatically canonically encode all inputs.\n\n- The ``X25519`` protocols now return ``DomainKDF`` results instead of plain ``sha3_512`` objects.\n\n- The ``(Base)Comprende`` classes were greatly simplified, & the caching & ``messages`` features were removed.\n\n- The top-level ``(a)mnemonic`` functions now return lists of bytes-type words, instead of str-type, & can now be used to quickly generate lists of randomly selected words without providing a (now optional) passphrase.\n\n- The ``(Async)Database`` classes' ``(a)generate_profile`` methods no longer require tokens to first be created by the user. That is now handled internally, & the external API accepts raw bytes inputs for credentials from the user.\n\n- The ``PackageSigner`` & ``PackageVerifier`` now use ``sha384`` for digests instead of ``sha512``. The verifier now by default recomputes & verifies the digests of files from the filesystem using the ``path`` keyword argument to the constructor as the root directory for the relative filepaths declared in the \"checksums\" entry of the signature summary.\n\n\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- A new ``Clock`` class was added to the ``generics.py`` module which provides a very intuitive API for handling time & timestamp functionalities for various time units.\n\n- The test suite was reorganized, cleaned up & extended significantly, & now also utilizes ``pytest-asyncio`` to run async tests. This led to many found & fixed bugs in code that was not being tested. There's still a substantial amount of tests that need to be written. We would greatly appreciate contributions which extend our test coverage.\n\n- Many improvements to the correctness, completeness & aesthetic beauty of the code documentation with the addition of visual aides, diagrams & usage examples.\n\n- A top-level ``report_security_issue`` function was added, which provides a terminal application for users to automatically encrypt security reports to us using our new X25519 public key.\n\n- We lost access to our signing keys in encrypted drives which were damaged in flooding. So we decided to shred them & start fresh. Our new Ed25519 signing key is \"70d1740f2a439da98243c43a4d7ef1cf993b87a75f3bb0851ae79de675af5b3b\". Contact us via email or twitter if you'd like to confirm that the key you are seeing is really ours.\n\n\n\n\nChanges for version 0.21.1\n--------------------------\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Fix usage of the wrong package signing key.\n\n\n\n\nChanges for version 0.21.0\n--------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Non-backwards compatible changes:\n- Altered the ``Chunky2048`` cipher's key derivation to continuously extract\n entropy from users' main encryption key. The design goal of the cipher\n is to be as close as possible to a one-time pad, but because we use \n key derivations to mix together all the relevant values used by the \n cipher, there's a limited amount of entropy that can be extracted \n from the main key no matter how large it is. The changes feed the \n main key into the internal seed KDF multiple times when creating the \n cipher's initial seeds, & once on every iteration of the ``(a)bytes_keys``\n generators. \n- Merged two internal KDFs used by the cipher into the one seed KDF. This\n also now means that using the ``(a)update_key`` methods of the ``StreamHMAC``\n class updates the KDF used to ratchet the encryption keystream.\n- Use ``sha3_512`` instead of ``sha3_256`` for the ``StreamHMAC`` final HMAC\n & slice the first bytes designated by the package's ``commons.py`` module.\n This allows the HMAC length to be specified & changed easily. It's \n **highly discouraged** to use anything less than 32-bytes.\n \n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Internal refactorings.\n- Updates to tests.\n\n\n\n\nChanges for version 0.20.7\n--------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Changed the way the ``Padding.(a)end_padding`` methods calculate the\n required padding length. The change causes the methods to now assume \n that the plaintext has already been prepended with the start padding.\n- The various ``test_*`` & ``verify_*`` functions/methods throughout the\n package have been changed to return ``None`` on successful validation\n instead of ``True``, which more closely matches the convention for\n exception-raising validators.\n- The default ``block_id`` length was changed from 16-bytes to 24-bytes.\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Make the ``(a)end_padding`` methods of the ``Padding`` class assume the \n supplied data has already been prepended with the start padding. This \n better integrates with streams of plaintext (online usage).\n- Small internal refactorings.\n- Documentation fixes.\n\n\n\n\nChanges for version 0.20.6\n--------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- The ``(Async)Database`` classes now support storing raw ``bytes`` type\n tag entries! This is a huge boon to time/space efficiency when needing\n to store large binary files, since they don't need to be converted to \n & from base64. This feature was made possible with only very minor \n changes to the classes, & they're fully backwards-compatible! Older \n versions will not be able handle raw ``bytes`` entries, but old JSON \n serializable entries work the same way they did.\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Docfixes.\n- Small refactorings.\n- Add new tests & make existing tests complete faster.\n- Support empty strings to be passed to the ``(Async)Database`` constructors'\n ``directory`` kwarg, signifying the current directory. Now ``None`` is\n the only falsey value which triggers the constructors to use the default\n database directory.\n- Fixed a bug in the ``AsyncDatabase`` class' ``aset_tag`` method, which \n would throw an attribute error when passed the ``cache=False`` flag.\n- Add Windows support to the CI tests.\n\n\n\n\nChanges for version 0.20.5\n--------------------------\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Include the missing changelog entries for ``v0.20.4``.\n\n\n\n\nChanges for version 0.20.4\n--------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Add ``python3.10`` support by copying the ``async_lru`` package's main module\n from their more up-to-date github repository instead of from PyPI.\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Small refactorings & code cleanups.\n- Documentation updates.\n- Type-hinting updates.\n- Cleanups to the package's module API.\n- Improve CI & extend to ``python3.10``.\n\n\n\n\nChanges for version 0.20.3\n--------------------------\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Small refactorings.\n- Documentation updates.\n- Type-hinting updates.\n- Additional tests.\n\n\n\n\nChanges for version 0.20.2\n--------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Changed the ``Padding`` class' ``(a)check_timestamp`` methods to\n ``(a)test_timestamp``, to better match the naming convention in the \n rest of the package.\n- Removed the ``(a)sum_sha3__(256/512)`` chainable generator methods from\n the ``Comprende`` class.\n- Removed the ``os.urandom`` based functions in the ``randoms.py`` module.\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Fixes & improvements to out of date documentation.\n- Small fixes to type-hints.\n- Small refactorings.\n- Add ``(a)generate_key`` functions to the package & ``(Async)Keys`` classes.\n- Fix some exception messages.\n\n\n\n\nChanges for version 0.20.1\n--------------------------\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Small fixes & improvements to documentation.\n- Small fixes & improvements to tests.\n- Small fixes to type-hints.\n- Small re-organization of source file contents.\n- Small bug fixes.\n\n\n\n\nChanges for version 0.20.0 (Backwards incompatible updates)\n-----------------------------------------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- The ``(a)json_(en/de)crypt`` & ``(a)bytes_(en/de)crypt`` functions &\n methods now only expect to work with ``bytes`` type ciphertext. And,\n the low-level cipher generators expect iterables of bytes where they\n used to expect iterables of integers.\n- The ``pid`` keyword-only argument throughout the package was changed\n to ``aad`` to more clearly communicate its purpose as authenticated\n additional data.\n- The ``key``, ``salt`` & ``aad`` values throughout the package are now\n expected to be ``bytes`` type values.\n- The ``key`` must now be at least 32-bytes for use within the ``Chunky2048``\n cipher & its interfaces.\n- The ``salt``, for use in the ``Chunky2048`` cipher & its interfaces, \n was decreased from needing to be 32-bytes to 24-bytes.\n- The ``siv``, for use in the ``Chunky2048`` cipher & its interfaces, was\n increased from needing to be 16-bytes to 24-bytes.\n- The new ``KeyAADBundle`` class was created as the primary interface\n for consuming ``key``, ``salt``, ``aad`` & ``siv`` values. This class'\n objects are the only ones that are used to pass around these values\n in low-level ``Chunky2048`` cipher functionalities. The higher-level\n cipher functions are the only public interfaces that still receive\n these ``key``, ``salt``, & ``aad`` values.\n- The ``KeyAADBundle`` now manages the new initial key derivation of the\n ``Chunky2048`` cipher. This new algorithm is much more efficient,\n utilizing the output of the keystream's first priming call instead of\n throwing it away, removing the need for several other previously used\n hashing calls.\n- The ``bytes_keys`` & ``abytes_keys`` keystream generator algorithms\n were improved & made more efficient. They also now only receive ``bytes``\n type coroutine values or ``None``.\n- The ``StreamHMAC`` algorithms were improved & made more efficient.\n- The ``Chunky2048`` class now creates instance's that initialize, & who's\n methods are callable, much more efficiently by reducing its previously\n dynamic structure. Its now reasonable to use these instances in code\n that has strict performance requirements.\n- The ``Keys`` & ``AsyncKeys`` classes were trimmed of all instance\n behaviour. They are now strictly namespaces which contain static or\n class methods.\n- All instance's of the word `password` throughout the package have been\n replaced with the word `passphrase`. The ``Passcrypt`` class now only\n accepts ``bytes`` type ``passphrase`` & ``salt`` values. The returned\n hashes are also now always ``bytes``.\n- The ``Padding`` & ``BytesIO`` classes' functionalities were made more\n efficient & cleaned up their implementations.\n- New ``PackageSigner`` & ``PackageVerifier`` classes were added to the\n ``keygens.py`` module to provide an intuituve API for users to sign their\n own packages. This package now also uses these classes to sign itself.\n- The new ``gentools.py`` module was created to organize the generator\n utilities that were previously scattered throughout the package's\n top-level namespaces.\n- The new ``_exceptions.py`` module was created to help organize the\n exceptions raised throughout the package, improving readability\n & maintainability.\n- The new ``_typing.py`` module was added to assist in the long process\n of adding functional type-hinting throughout the package. For now,\n the type hints that have been added primarily function as documentation.\n- A new ``Slots`` base class was added to the ``commons.py`` module to\n simplify the creation of more memory efficient & performant container\n classes. The new ``_containers.py`` module was made for such classes\n for use throughout the package. And, most classes throughout the\n package were given ``__slots__`` attributes.\n- A new ``OpenNamespace`` class was added, which is a subclass of ``Namespace``,\n with the only difference being that instances do not omit attributes\n from their repr's.\n- The new ``(a)bytes_are_equal`` functions, which are pointers to\n ``hmac.compare_digest`` from the standard library, have replaced the\n ``(a)time_safe_equality`` functions.\n- The ``(a)sha_256(_hmac)`` & ``(a)sha_512(_hmac)`` functions have had\n their names changed to ``(a)sha3__256(_hmac)`` & ``(a)sha3__512(_hmac)``.\n This was done to communicate that they are actually SHA3 functions,\n but the double underscore is to keep them differentiable from the\n standard library's ``hashlib`` objects. They can now also return\n ``bytes`` instead of hex strings if their ``hex`` keyword argument is truthy.\n- The base functionality of the ``Comprende`` class was refactored out into a\n ``BaseComprende`` class. The chainable data processor generator methods\n remain in the ``Comprende`` class. Their endpoint methods (such as ``(a)list``\n & ``(a)join``) have also been changed so they don't cache results by default.\n- The ``Passcrypt`` class' ``kb`` & ``hardness`` can now be set to values\n independently from one another. The algorithm runs on the new\n ``(a)bytes_keys`` coroutines, & a slightly more effective cache building\n procedure.\n- The databases classes now don't preload their values by default. And,\n various methods which work with tags & metatags have been given a\n ``cache`` keyword-only argument to toggle on/off the control of using\n the cache for each operation.\n- New method additions/changes to the database classes:\n\n - ``(a)rollback_tag``, ``(a)clear_cache``, & a ``filenames`` property \n were added.\n - ``(a)hmac`` was changed to ``(a)make_hmac``, & now returns ``bytes`` hashes.\n - ``(a)save`` was changed to ``(a)save_database``.\n - ``(a)query`` was changed to ``(a)query_tag``.\n - ``(a)set`` was changed to ``(a)set_tag``.\n - ``(a)pop`` was changed to ``(a)pop_tag``.\n - The ``tags``, ``metatags`` & ``filenames`` properties now return sets\n instead of lists.\n\n- The ``Ropake`` class has been removed from the package pending changes to\n the protocol & its implementation.\n- The ``(a)generate_salt`` function now returns ``bytes`` type values,\n & takes a ``size`` keyword-only argument, with no default, that determines\n the number of bytes returned between [8, 64].\n- The ``(a)random_512`` & ``(a)random_256`` public functions can now cause\n their underlying random number generators to fill their entropy pools\n when either the ``rounds`` or ``refresh`` keyword arguments are specified.\n- The following variables were removed from the package:\n \n - ``(a)keys``, ``(a)passcrypt``, ``(a)seeder``, ``(a)time_safe_equality``,\n ``Datastream``, ``bits``, ``(a)seedrange``, ``(a)build_tree``,\n ``(a)customize_parameters``, ``convert_class_method_to_member``,\n ``convert_static_method_to_member``, ``(a)xor``, ``(a)padding_key``,\n ``(a)prime_table``, ``(a)unique_range_gen``, ``(a)non_0_digits``,\n ``(a)bytes_digits``, ``(a)digits``, ``(a)permute``, ``(a)shuffle``,\n ``(a)unshuffle``, ``(a)create_namespace``,\n (``(a)depad_plaintext``, ``(a)pad_plaintext`` & their generator forms.\n Only the non-generator forms remain in the ``Padding`` class), (The\n ``(a)passcrypt``, ``(a)uuids``, ``(a)into_namespace`` methods from the\n database classes), (The ``(a)csprbg`` functions were removed & instead\n the ``(a)csprng`` functions produce ``bytes`` type values.)\n \n- Thorough & deep refactorings of modules, classes & methods. Many methods\n & functions were made private, cleaning up the APIs of the package,\n focusing on bringing the highest-level functionalities to top level\n namespaces accessible to users. Some purely private functionalities\n were entirely moved to private namespaces not readily accessible to\n users.\n- Most of the constants which determine the functionalities throughout\n the package were refactored out into ``commons.py``. This allows\n for easy changes to protocols & data formats.\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Many documentation improvements, fixes, trimmings & updates.\n- Added a ``WeakEntropy`` class to the ``randoms.py`` module.\n\n\n\n\nChanges for version 0.19.4 \n-------------------------- \n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Created a private ``EntropyDaemon`` class to run a thread in the \n background which feeds into & extracts entropy from some of the \n package's entropy pools. Also moved the separate private ``_cache`` \n entropy pools from the parameters to the random number generators. \n They're now a single private ``_pool`` shared global that's \n asynchronously & continuously updated by the background daemon thread. \n- Switched the ``random`` portion of function names in the ``randoms.py`` \n module to read ``unique`` instead. This was done to the functions which \n are actually pseudo-random. This should give users a better idea of \n which functions do what. The exception is that the ``random_sleep`` & \n ``arandom_sleep`` functions have kept their names even though they \n sleep a pseudo-randomly variable amount of time. Their names may \n cause more confusion if they were either ``(a)unique_sleep`` or \n ``(a)urandom_sleep``. Because they don't use ``os.urandom`` & what \n is a ``unique_sleep``? When / if a better name is found these \n function names will be updated as well. \n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Various docstring / documentation fixes & refactorings.\n\n\n\n\nChanges for version 0.19.3 \n-------------------------- \n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Removed ``ascii_encipher``, ``ascii_decipher``, ``aascii_encipher`` &\n ``aascii_decipher`` generators from the ``Chunky2048`` & ``Comprende``\n classes, & the package. It was unnecessary, didn't fit well with the\n intended use of the ``Padding`` class, & users would be much better\n served by converting their ascii to bytes to use the ``bytes_``\n generators instead.\n- Removed the ``map_encipher``, ``map_decipher``, ``amap_encipher`` &\n ``amap_decipher`` generators from the ``Chunky2048`` & ``Comprende``\n classes, & the package. They were not being used internally to the \n package anymore, & their functionality, security & efficiency could \n not be guaranteed to track well with the changes in the rest of the \n library.\n- Added domain specificity to the ``X25519`` protocols' key derivations.\n- Renamed the database classes' ``(a)encrypt`` & ``(a)decrypt`` methods\n to ``(a)json_encrypt`` & ``(a)json_decrypt`` for clarity & consistency\n with the rest of the package. Their signatures, as well as those in \n ``(a)bytes_encrypt`` & ``(a)bytes_decrypt``, were also altered to\n receive plaintext & ciphertext as their only positional arguments. \n The ``filename`` argument is now a keyword-only argument with a default\n ``None`` value. This allows databases to be used more succinctly for\n manual encryption & decryption by making the filename tweak optional.\n- The ``runs`` keyword argument for the functions in ``randoms.py`` was\n renamed to ``rounds``. It seems more clear that it is controlling the\n number of rounds are internally run within the ``(a)random_number_generator``\n functions when deriving new entropy. \n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Fixes to docstrings & tutorials. Rewrite & reorganization of the \n ``PREADME.rst`` & ``README.rst``. More updates to the readme's are still\n on the way.\n- Slight fix to the Passcrypt docstring's algorithm diagram.\n- Moved the default passcrypt settings to variables in the ``Passcrypt``\n class.\n- Added the ability to send passcrypt settings into the ``mnemonic`` &\n ``amnemonic`` coroutines, which call the algorithm internally but \n previously could only use the default settings.\n- Some code cleanups & refactorings.\n\n\n\n\nChanges for version 0.19.2 \n-------------------------- \n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Made the output lengths of the ``Padding`` class' generator functions \n uniform. When the footer padding on a stream of plaintext needs to \n exceed the 256-byte blocksize (i.e. when the last unpadded plaintext \n block's length ``L`` is ``232 < L < 256``), then another full block of\n padding is produced. The generators now yield 256-byte blocks \n consistently (except during depadding when the last block of plaintext\n may be smaller than the blocksize), instead of sometimes producing a\n final padded block which is 512 bytes.\n\n\n\n\nChanges for version 0.19.1 \n-------------------------- \n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Fixed a bug where database classes were evaluating as falsey when they\n didn't have any tags saved in them. They should be considered truthy \n if they're instantiated & ready to store data, even if they're \n currently empty & not saved to disk. This was reflected in their \n ``__bool__`` methods. The bug caused empty metatags not to be loaded \n when an instance loads, even when ``preload`` is toggled ``True``.\n- Removed the coroutine-receiving logic from the ``Padding`` class'\n ``Comprende`` generators. Since they buffer data, the received values\n aren't ever going to coincide with the correct iteration & will be\n susceptible to bugs\n- Fixed a bug in the ``Padding`` class' ``Comprende`` generators which \n cut iteration short because not enough data was available from the \n underlying generators upfront. Now, if used correctly to pad/depad \n chunks of plaintext 256 bytes at a time, then they work as expected.\n- The ``update``, ``aupdate``, ``update_key`` & ``aupdate_key`` methods\n in both the ``StreamHMAC`` & ``DomainKDF`` classes now return ``self``\n to allow inline updates.\n- Added ``acsprng`` & ``csprng`` function pointers to the ``Chunky2048``\n class.\n- Updates to docstrings which didn't get updated with info on the new \n *synthetic IV* feature.\n- Some other docstring fixes.\n- Some small code cleanups & refactorings.\n\n\n\n\nChanges for version 0.19.0 \n-------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Security Upgrade: The package's cipher was changed to an online, \n authenticated scheme with salt reuse / misuse resistance. This was \n acheived through a few backwards incompatible techniques: \n \n 1. A synthetic IV (SIV) is calculated from the keyed-hash of the first \n 256-byte block of plaintext. The SIV is then used to seed the \n keystream generator, & is used to update the validator object. This \n ensures that if the first block is unique, then the whole ciphertext \n will be unique.\n 2. A 16-byte ephemeral & random SIV-key is also prepended to the \n first block of plaintext during message padding. Since this value \n is also hashed to derive the SIV, this key gives a strong \n guarantee that a given message will produce a globally unique \n ciphertext.\n 3. An 8-byte timestamp is prepended to the first block of plaintext \n during padding. Timestamps are inherently sequential, they can be \n verified by a user within some bounds, & can also be used to \n mitigate replay attacks. Since it's hashed to make the SIV, then \n it helps make the entire ciphertext unique.\n 4. After being updated with each block of ciphertext, the validator's \n current state is again fed into the keystream generator as a new \n rotating seed. This mitigation is limited to ensuring only that \n every following block of ciphertext to a block which is unique\n will also be unique. More specifically this means that: **if** \n *all* **other mitigations fail to be unique**, or are missing, then \n the first block which is unique **will appear the same**, except \n for the bits which have changed, **but, all following blocks will\n be randomized.** This limitation could be avoided with a linear\n expansion in the ciphertext size by generating an SIV for each\n block of plaintext. This linear expansion is prohibitive as a\n default setting, but the block level secrecy, even when all other \n mitigations fail, is enticing. This option may be added in the \n future as a type of padding mode on the plaintext.\n \n The SIV-key is by far the most important mitigation, as it isn't \n feasibly forgeable by an adversary, & therefore also protects against\n attacks using encryption oracles. These changes can be found in the \n ``SyntheticIV`` class, the (en/de)cipher & xor generators, & the \n ``StreamHMAC`` class in the ``ciphers.py`` module. The padding \n changes can also be found in the new ``Padding`` class in the ``generics.py`` \n module. The SIV is attached in the clear with ciphertexts & was \n designed to function with minimal user interaction. It needs only to \n be passed into the ``StreamHMAC`` class during decryption -- during \n encryption it's automatically generated & stored in the ``StreamHMAC`` \n validator object's ``siv`` property attribute. \n- Security Patch: The internal ``sha3_512`` kdf's to the ``akeys``, ``keys``, \n ``abytes_keys`` & ``bytes_keys`` keystream generators are now updated\n with 72 bytes of (64 key material + 8 padding), instead of just 64 \n bytes of key material. 72 bytes is the *bitrate* of the ``sha3_512`` \n object. This change causes the internal state of the object to be permuted \n for each iteration update & before releasing a chunk of key material. \n Frequency analysis of ciphertext bytes didn't smooth out to the \n cumulative distribution expected for all large ciphertexts prior to \n this change. But after the change the distribution does normalize as\n expected. This indicates that the key material streams were biased \n away from random in a small but measurable way. Although, no \n particular byte values seem to have been preferred by this bias, this \n is a huge shortcoming with unknown potential impact on the strength \n of the package's cipher. This update is strongly recommended & is \n backwards incompatible. \n- This update gives a name to the package's pseudo-one-time-pad cipher \n implementation. It's now called ``Chunky2048``! The ``OneTimePad`` \n class' name was updated to ``Chunky2048`` to match the change.\n- The ``PreemptiveHMACValidation`` class & its related logic in the\n ``StreamHMAC`` class was removed. The chaining of validator output\n into the keystream makes running the validator over the ciphertext \n separately or prior to the decryption process very costly. It would \n either mean recalculating the full hash of the ciphertext a second \n time to reproduce the correct outputs during each block, or a large \n linear memory increase to hold all of its digests to be fed in some \n time after preemtive validation. It's much simpler to remove that \n functionality & potentially replace it with something else that fits\n the user's applications better. For instance, the ``current_digest``\n & ``acurrent_digest`` methods can produce secure, 32-byte authentication\n tags at any arbitrary blocks throughout the cipher's runtime, which\n validate the cipehrtext up to that point. Or, the ``next_block_id`` \n & ``anext_block_id`` methods, which are a more robust option because \n each id they produce validates the next ciphertext block before \n updating the internal state of the validator. This acts as an \n automatic message ordering algorithm, & leaves the deciphering \n party's state unharmed by dropped packets or manipulated ciphertext.\n- The ``update_key`` & ``aupdate_key`` methods were also added to the\n ``StreamHMAC`` class. They allow the user to update the validators'\n internal key with new entropy or context information during its \n runtime. \n- The ``Comprende`` class now takes a ``chained`` keyword-only argument\n which flags an instance as a chained generator. This flag allows \n instances to communicate up & down their generator chain using the \n shared ``Namespace`` object accessible by their ``messages`` attribute.\n- The chainable ``Comprende`` generator functions had their internals\n altered to allow them to receive, & pass down their chain, values \n sent from a user using the standard coroutine ``send`` & ``asend``\n method syntax.\n- ``Comprende`` instances no longer automatically reset themselves every \n time they enter their context managers or when they are iterated over.\n This makes their interface more closely immitate the behavior of \n async/sync generator objects. To get them to reset, the ``areset`` or \n ``reset`` methods must be used. The message chaining introduced in \n this update allows chains of ``Comprende`` async/sync generators to \n inform each other when the user instructs one of them to reset.\n- The standard library's ``hmac`` module is now used internally to the\n ``generics.py`` module's ``sha_512_hmac``, ``sha_256_hmac``, ``asha_512_hmac`` \n & ``asha_256_hmac`` functions. They still allow any type of data to be \n hashed, but also now default to hashing ``bytes`` type objects as \n they are given.\n- The new ``Domains`` class, found in ``generics.py``, is now used to\n encode constants into deterministic pseudo-random 8-byte values for\n helping turn hash function outputs into domain-specific hashes. Its\n use was included throughout the library. This method has an added\n benefit with respect to this package's usage of SHA-3. That being, the\n *bitrate* for both ``sha3_512`` & ``sha3_256`` are ``(2 * 32 * k) + 8``\n bytes, where ``k = 1`` for ``sha3_512`` & ``k = 2`` for ``sha3_256``.\n This means that prepending an 8-byte domain string to their inputs\n also makes it more efficient to add some multiple of key material\n to make the input data precisely equal the *bitrate*. More info on\n domain-specific hashing can be found here_.\n\n.. _here: https://eprint.iacr.org/2020/241.pdf\n\n- A new ``DomainsKDF`` class in ``cipehrs.py`` was added to create a\n more standard & secure method of key derivation to the library which \n also incorporates domain separation. Its use was integrated thoughout \n the ``AsyncDatabase`` & ``Database`` classes to mitigate any further \n vulnerabilities of their internal key-derivation functions. The \n database classes now also use bytes-type keys internally, instead \n of hex strings.\n- The ``Passcrypt`` class now contains methods which create & validate\n passcrypt hashes which have their settings & salt attached to them.\n Instances can now also be created with persistent settings that are \n automatically sent into instance methods.\n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Many fixes of docstrings, typos & tutorials. \n- Many refactorings: name changes, extracted classes / functions, \n reorderings & moves. \n- Various code clean-ups, efficiency & usability improvements.\n- Many constants used throughout the library were given names defined \n in the ``commons.py`` module.\n- Removed extraneous functions throughout the library.\n- The asymmetric key generation & exchange functions/protocols were \n moved from the ``ciphers.py`` module to ``keygens.py``.\n- Add missing modules to the MANIFEST.rst file. \n- Added a ``UniformPrimes`` class to the ``__datasets`` module for efficient \n access to primes that aren't either mostly 1 or 0 bits, as is the case for \n the ``primes`` helper table. These primes are now used in the ``Hasher`` \n class' ``amask_byte_order`` & ``mask_byte_order`` methods. \n- The ``time_safe_equality`` & ``atime_safe_equality`` methods are now \n standalone functions available from the ``generics.py`` module.\n- Added ``reset_pool`` to the ``Processes`` & ``Threads`` classes. Also\n fixed a missing piece of logic in their ``submit`` method.\n- Added various conversion values & timing functions to the ``asynchs.py``\n module.\n- The ``make_uuid`` & ``amake_uuid`` coroutines had their names changed to \n ``make_uuids`` & ``amake_uuids``.\n- Created a new ``Datastream`` class in ``generics.py`` to handle buffering\n & resizing iterable streams of data. It enables simplifying logic that \n must happen some number of iterations before the end of a stream. It's \n utilized in the ``Padding`` class' generator functions available as \n chainable ``Comprende`` methods.\n- The ``data`` & ``adata`` generators can now produce a precise number of\n ``size``-length ``blocks`` as specified by a user. This gets rid of the\n confusing usage of the old ``stop`` keyword-only argument, which stopped \n a stream after *approximately* ``size`` number of elements.\n- Improved the efficiency & safety of entropy production in the \n ``randoms.py`` module.\n\n\n\nChanges for version 0.18.1 \n-------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Security Patch: Deprecated & replaced an internal kdf for saving \n database tags due to a vulnerability. If an adversary can get a user \n to reveal the value returned by the ``hmac`` method when fed the tag \n file's filename & the salt used for that encrypted tag, then they \n could deduce the decryption key for the tag. A version check was \n added only for backwards compatibility & will be removed on the next \n update. All databases should continue functioning as normal, though \n all users are advised to **re-save their databases** after upgrading\n so the new kdf can be used. This will not overwrite the old files,\n so they'll need to be deleted manually.\n- Replaced usage of the async ``switch`` coroutine with ``asyncio.sleep``\n because it was not allowing tasks to switch as it was designed to.\n Many improvements were made related to this change to make the\n package behave better in async contexts.\n- Removed the private method in the database classes which held a \n reference to the root salt. It's now held in a private attribute. \n This change simplifies the code a bit & allows instances to be \n pickleable.\n- The ``atimeout`` & ``timeout`` chainable ``Comprende`` generator\n methods can now stop the generators' executions mid-iteration. They\n run them in separate async tasks or thread pools, respectively, to \n acheive this.\n- The ``await_on`` & ``wait_on`` generators now restart their timeout\n counters after every successful iteration that detected a new value\n in their ``queue``. The ``delay`` keyword argument was changed to \n ``probe_frequency``, a keyword-only argument.\n- Removed the package's dependency on the ``aioitertools`` package.\n- Made the ``sympy`` package an optional import. If any of its\n functionalities are used by the user, the package is only then\n imported & this is done automatically.\n- Various streamlining efforts were made to the imports & entropy\n initialization to reduce the package's import & startup time.\n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Fixes of various typos, docstrings & tutorials.\n- Various cleanups, refactorings & efficiency improvements.\n- Added new tests for detecting malformed or modified ciphertexts.\n- Removed extraneous functions in ``generics.py``.\n- Add a ``UNIFORM_PRIME_512`` value to ``__datasets.py`` for use in the \n ``Hasher.mask_byte_order`` & ``Hasher.amask_byte_order`` methods.\n Those methods were also altered to produce more uniform looking \n results. The returned masked values are now also 64 bytes by default.\n- Added an ``automate_key_use`` keyword-only boolean argument to the init\n for the ``OneTimePad``, ``Keys`` & ``AsyncKeys`` classes. It can be toggled to\n stop the classes from overwriting class methods so they \n automatically read the instance's key attribute. This optionally \n speeds up instantiation by an order of magnitude at the cost of \n convenience.\n- Fixed ``asynchs.Threads`` class' wrongful use of a ``multiprocessing``\n ``Manager.list`` object instead of a regular list.\n- Changed the ``_delay`` keyword-only argument in ``Processes`` & ``Threads``\n classes' methods to ``probe_freqeuncy`` so users can specify how often\n results will be checked for after firing off a process, thread, or\n associated pool submission.\n- Now the ``asubmit`` & ``submit`` methods in ``Processes`` & ``Threads`` \n can accept keyword arguments.\n- Added ``agather`` & ``gather`` methods to the ``Threads`` & ``Processes``\n classes. They receive any number of functions, & ``args`` &/or ``kwargs`` to\n pass to those functions when submitting them to their associated \n pools.\n- Changed the ``runsum`` instance IDs from hex strings to bytes & cleaned \n up the instance caching & cleaning logic.\n- Altered & made private the ``asalted_multiply`` & ``salted_multiply``\n functions in the ``randoms.py`` module.\n- Started a new event loop specific to the ``randoms.py`` module which\n should prevent the ``RuntimeError`` when ``random_number_generator``\n is called from within the user's running event loop.\n- Added a ``ValueError`` check to the ``(a)cspr(b/n)g`` functions in \n ``randoms.py``. This will allow simultaneously running tasks to \n request entropy from the function by returning a result from a \n newly instantiated generator object. \n- Added checks in the ``*_encipher`` & ``*_decipher`` generators to \n help assure users correctly declare the mode for their StreamHMAC \n validator instances. \n- Fixed the ``__len__`` function in the database classes to count the \n number of tags in the database & exclude their internal maintenaince \n files.\n- The ``TimeoutError`` raised after decrypting a ciphertext with an \n expired timestamp now contains the seconds it has exceeded the ``ttl``\n in a ``value`` attribute.\n- The timestamp used to sign the package now displays the day of \n signing instead of the second of signing.\n- The ``(a)sum_sha_*`` & ``(a)sum_passcrypt`` generators were altered to\n reapply the supplied ``salt`` on every iteration. \n- Stabilized the usability of the ``stop`` keyword-only argument in the\n ``adata`` & ``data`` generators. It now directly decides the total\n number of elements in a ``sequence`` allowed to be yielded.\n\n\n\n\nChanges for version 0.18.0 \n-------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Security Patch: Rewrote the HMAC-like creation & authentication \n process for all of the package's ciphers. Now, the ``*_encipher``\n & ``*_decipher`` ``Comprende`` generators must be passed a validator\n object to hash the ciphertext as it's being created / decrypted.\n The ``StreamHMAC`` class was created for this purpose. It's initalized\n with the user's long-term key, the ephemeral salt & the pid value.\n The pid value can now effectively be used to validate additional data.\n These changes force the package's cipher to be used as an AEAD cipher.\n- Security Patch: The package's ``*_hmac`` hash functions & the ``Comprende``\n class' hash generators were rewritten to prepend salts & keys to data\n prior to hashing instead of appending. This is better for several \n important reasons, such as: reducing the amortizability of costs in\n trying to brute-force hashes, & more closely following the reasoning\n behind the HMAC spec even though sha3 has a different security profile. \n- Algorithm Patch: The ``akeys``, ``keys``, ``abytes_keys``, & ``bytes_keys``\n algorithms have been patched to differentiate each iteration's two\n sha3_512 hashes from one another in perpetuity. They contained a design\n flaw which would, if both sha3_512 objects landed upon the same \n 1600-bit internal state, then they would produce the same keystreams \n from then on. This change in backwards incompatible. This flaw is \n infeasible to exploit in practice, but since the package's hashes & \n ciphertext validations were already channging this release, there was \n no reason to not fix this flaw so that it's self-healing if they ever \n do land on the same internal states.\n- The ``Passcrypt`` class & its algorithm were made more efficient to\n better equalize the cost for users & adversaries & simplifies the\n algorithm. Any inefficiencies in an implementation would likely cause\n the adversary to be able to construct optimized implementations to \n put users at an even greater disadvantage at protecting their inputs\n to the passcrypt algorithm. It used the ``sum_sha_256`` hash function \n internally, & since it was also changing in a non-backwards \n compatible way with this update, it was the best time to clean up\n the implementation.\n- Updated the package's description & its docstrings that refer to \n the package's cipher as an implementation of the one-time-pad. It's \n not accurate since the package uses pseudo-random hash functions to \n produce key material. Instead, the package's goal is to create a \n pseudo-one-time-pad that's indistinguishable from a one-time-pad.\n The ``OneTimePad`` class will keep its name for succinctness. \n- New ``amake_token``, ``make_token``, ``aread_token`` & ``read_token``\n class & instance methods added to the ``OneTimePad`` class. These\n tokens are urlsafe base64 encoded, are encrypted, authenticated &\n contain timestamps that can enforce a time-to-live for each token.\n- Non-backwards compatible changes to the database classes' filenames,\n encryption keys & HMACs. The ``*_hmac`` hash functions that the \n databases rely on were changing with this update, so additionally the \n filenames table used to encode the filenames was switched from the \n ``BASE_36_TABLE`` to the ``BASE_38_TABLE``. Both tables are safe for \n uri's across all platforms, but the new table can encode information \n slightly more efficiently.\n- Major refactorings & signature changes across the package to make\n passing keys & salts to ``*_hmac`` functions & the ``Comprende`` \n class' hash generators explicit.\n- Removed the ``of`` keyword argument from all of the ``Comprende`` \n class' generators. It was overly complicating the code, & was not\n entirely clear or useful for settings outside of the ``tags`` & \n ``atags`` generators.\n- Removed ``pybase64`` from the package & its dependencies list. The\n built-in python ``base64`` module works just fine.\n- Sorted the ``WORDS_LIST``, ``ASCII_ALPHANUMERIC``, & ``BASE_64_TABLE``\n datasets.\n- The ``salt`` & ``asalt`` functions have been renamed to ``generate_salt``\n & ``agenerate_salt`` for clarity's sake, & to reduce naming \n collisions.\n- Added another redundancy to the ``arandom_number_generator`` &\n ``random_number_generator`` functions. Now the async tasks it prepares\n into a list are pseudo-randomly shuffled before being passed into \n ``asyncio.gather``.\n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Added a logo image to the package.\n- Separated the FAQ section from ``PREADME.rst``.\n- The ``primes`` & ``bits`` datasets are now represented in hex in the\n source code.\n- Added a ``BASE_38_TABLE`` dataset to the package.\n- The database classes now fill an ephemeral dictionary of filenames\n that couldn't be used to successfully load a tag file, available from \n within the ``_corrupted_files`` attribute.\n- The ``Comprende`` class' ``acache_check`` & ``cache_check`` context\n manager methods are now called ``aauto_cache`` & ``auto_cache``.\n- Added new ``bytes_count`` & ``abytes_count`` generators to ``generics.py``\n module which increment each iteration & yield the results as bytes.\n- Removed the ``akeypair`` & ``keypair`` functions from the package. \n Their successors are the ``asingle_use_key`` & ``single_use_key`` methods\n in the ``AsyncKeys`` & ``Keys`` classes. The attempt is to clarify &\n put constraints on the interface for creating a bundle of key \n material that has a single-use-only salt attached, as well as the pid \n value. \n- Moved ciphertext encoding functions into the ``BytesIO`` class from\n the global ``generics.py`` module.\n- Split ``PrimeGroups`` into two classes, one higher-level class by the\n same name & a ``BasePrimeGroups`` class. The former also has some\n added functionality for masking the order of bytes in a sequence \n using an modular exponentiation.\n- The ``Hasher`` class now has functionality added to mask the order\n of a bytes sequence with a modular multiplication.\n- Fixed the name of the project in the attribution lines in several \n source files.\n- Reconciled tests with the major changes in this release.\n- The old identity key for the package that was signed by the gnupg \n identity key was shredded & replaced with a new signed key.\n- Several bug fixes to the ``setup.py`` automated code signing.\n\n\n\n\nChanges for version 0.17.0 \n-------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Security Patch: The HMAC verifiers on ciphertexts did not include \n the ``salt`` or ``pid`` values when deriving the HMAC. This \n associated data can therefore be changed to cause a party to\n decrypt a past ciphertext with a salt or pid of an attacker's\n choosing. This is a critical vulnerability & it is highly recommended\n all users update. The fix is to hash the ciphertext, ``salt`` \n & ``pid`` together & sending that hash into the validator to have\n the HMAC created / tested. This change will cause all prior \n ciphertexts to be marked invalid by the validator.\n- Refactored the names of the Comprende cipher methods to better \n communicate their intended use as lower level tools that cannot be\n used on their own to obtain authenticated, CCA or CPA secure \n encryption.\n- Added more comprehensive tests for ``X25519`` & ``Ed25519`` classes,\n as well as the protocols that utilize the ``X25519`` ecdh exchange.\n Fixed some bugs in the process.\n- ``X25519`` instances that contain a secret key now have access to\n protocol methods which automatically pass their key in as a keyword\n argument. This simplifies their usage further.\n- Incorporated the new ``Hasher`` class into the package's random\n number generator to improve its entropy production.\n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Various fixes to typos, docstrings & tutorials.\n- New tutorials & docs added.\n- Changed the default table in ``ByteIO`` 's ``json_to_ascii``, ``ajson_to_ascii``,\n ``ascii_to_json`` & ``aascii_to_json`` to the ``URL_SAFE_TABLE`` to \n facilitate the creation of urlsafe_tokens.\n- Removed all code in the ``Ropake`` class that was used to create a default\n database to store a default salt for users. All of that functionality \n is expected to be handled by the database classes' token & profile \n creation tools.\n- Fixed bug in package signing script that called hex from a string.\n- Updated the package signing script to include these metadata in the\n signatures of the ephemeral keys: name of the package, version, the \n date in seconds.\n- Added metadata to the ``setup.cfg`` file.\n- Make passcrypt objects available from the ``keygens`` module.\n- Add more consistent ability within ``Ropake`` class to specify a\n time-to-live for protocol messages.\n- Added check to make sure instances of ``X25519`` & ``Ed25519`` are\n not trying to import a new secret key once they already have one. \n This won't be allowed in favor of creating a new object for a new\n secret key.\n- Fixed bug in database classes' bytes ciphers which called themselves\n recursively instead of calling the global functions of the same name.\n\n\n\n\nChanges for version 0.16.0 \n-------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- All ``Database`` & ``AsyncDatabase`` filenames have been converted to\n base36 to aid in making the manifest files & the databases as a whole \n more space efficient. These changes are not backwards compatible.\n- More work was done to clean up the databases & make them more \n efficient, as well as equalize the sizes of the database files to\n mitigate leaking metadata about what they might contain. \n- Added new ``X25519`` & ``Ed25519`` classes that greatly simplify the\n usage of the cryptography module's 25519 based tools. They also help\n organize the codebase better -- where ``Ropake`` was holding onto\n all of the asymmetric tooling even though those tools were not part\n of the Ropake protocol.\n- New base & helper ``Asymmetric25519`` & ``BaseEllipticCurve`` classes \n were added as well to facilitate the reorganization.\n- Many methods in ``Ropake`` were turned private to simplify & clean up \n the interface so its intended use as a protocol is more clear for users.\n- Added the time-to-live functionality to ``Ropake`` decryption functions.\n The ``TIMEOUT`` attribute on the class can also be changed to import \n a global time-to-live for all ``Ropake`` ciphertexts.\n- Removed all ``nc_`` hash functions from the package/generics.py module.\n- The ``Namespace`` class now has a ``keys`` method so that namespaces\n can be unpacked using star-star syntax.\n- Because of the ongoing failures of gnupg, we are moving away from \n signing our packages with gnupg. Our new Ed25519 keys will be from\n the cryptography package, & we'll sign those with our gnupg key as a\n secondary form of attestation. Our package signing will be automated\n in the setup.py file & the methods we use will be transparent in the \n code. The new signatures for each package version will be placed in \n the file ``SIGNATURES.txt``.\n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Many fixes & additions to docstrings & tutorials.\n- Massive refactorings, cleanups & typo fixes across the library, \n especially in the database classes, ``Ropake`` & the ``ciphers`` module.\n- Added comprehensive functional tests for the Ropake class.\n- Added ``BASE_36_TABLE`` to the ``commons`` module.\n- Fixed metadata issues in setup.py that caused upload issues to pypi.\n- The ``generate_profile``, ``load_profile``, ``agenerate_profile`` &\n ``aload_profile`` database methods now accept arbitrary keyword arguments \n that get passed into the database's __init__ constructor.\n- ``username`` & ``password`` are now required keyword-only arguments\n to the ``agenerate_profile_tokens`` & ``generate_profile_tokens`` \n classmethods.\n- The ``aload`` & ``load`` database methods now take a ``manifest`` kwarg\n that when toggled ``True`` will also refresh the manifest file from \n disk.\n- Now when a database object is ordered to delete itself, the entirety \n of the instance's caches & attribute values are cleared & deleted.\n- Filled out the references to strong key generators & protocols in the\n ``keygens`` module.\n\n\n\n\nChanges for version 0.15.0 \n-------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Security Patch: The previous update left the default salt stored by\n the ``Ropake`` class on the user filesystem as an empty string for\n new files that were created since the ``asalt`` & ``salt`` functions\n were switched to producing 256-bit values instead of 512-bits. This\n bug has now been fixed.\n- An 8 byte timestamp is now prepended to each plaintext during the\n padding step. The decryption functions now take a ``ttl`` kwarg which\n will measure & enforce a time-to-live for ciphertexts under threat of\n ``TimeoutError``.\n- Added new profile feature to the database classes. This standardizes\n & simplifies the process for users to open databases using only \n low-entropy \"profile\" information such as ``username``, ``password``,\n ``*credentials`` & an optional ``salt`` a user may have access to. \n The new ``agenerate_profile_tokens``, ``generate_profile_tokens``, \n ``agenerate_profile``, ``generate_profile``, ``aprofile_exists``, \n ``profile_exists``, ``aload_profile``, ``load_profile``, ``adelete_profile``\n & ``delete_profile`` functions are the public part of this new feature.\n- Some more database class attributes have been turned private to clean\n up the api.\n- Fixed typo in ``__exit__`` method of ``Database`` class which referenced \n a method which had its name refactored, leading to a crash.\n- Shifted the values in the ``primes`` dictionary such that the key for\n each element in the dictionary is the exclusive maximum of each prime\n in that element. Ex: primes[512][-1].to_bytes(64, \"big\") is now valid.\n Whereas before, primes[512] was filled with primes that were 64 bytes\n and 1 bit long, making them 65 byte primes. This changes some of the\n values of constants in the package & therefore some values derived \n from those constants.\n- Slimmed down the number of elements in the ``primes`` & ``bits`` \n dictionaries, reducing the size of the package a great deal. ``primes``\n now contains two primes in each element, the first is the minimum \n prime of that bit length, the latter the maximum.\n- Added ``URLSAFE_TABLE`` to the package.\n- Made ``salt`` & ``pid`` & ``ttl`` keyword only arguments in key \n generators & encryption / decryption functions, further tighening up\n the api.\n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Added ``this_second`` function to ``asynchs`` module for integer time.\n- Added ``apadding_key``, ``padding_key``, ``aplaintext_stream`` & \n ``plaintext_stream`` functions to the ``ciphers`` module.\n- Added ``apadding_key``, ``padding_key`` to the ``keygens`` module &\n ``AsyncKeys`` & ``Keys`` classes.\n- Added ``axi_mix``, ``xi_mix``, ``acheck_timestamp``, ``check_timestamp``,\n to the ``generics`` module.\n- Added ``acsprbg``, ``csprbg``, ``asalt``, ``salt``, ``apadding_key``, \n ``padding_key``, ``aplaintext_stream`` & ``plaintext_stream`` functions\n to OneTimePad class as ``staticmethod`` & instance methods.\n- Added ``acheck_timestamp`` & ``check_timestamp`` functions to the \n ``BytesIO`` class.\n- Added ``adeniable_filename`` & ``deniable_filename`` to the ``paths`` \n module. \n- Removed check for falsey data in encryption functions. Empty data is \n & should be treated as valid plaintext.\n- Various refactorings, docstring fixes & efficiency improvements.\n- Added some new tests for database profiles.\n\n\n\n\nChanges for version 0.14.0 \n-------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Security patch: The ``apad_bytes``, ``pad_bytes``, ``adepad_bytes`` &\n ``depad_bytes`` functions were changed internally to execute in a\n more constant time. The variations were small for 256-byte buffers\n (the default), but can grow very wide with larger buffers. The salt\n in the package's encryption utilities is now used to derive the \n plaintext's padding, making each padding unique. \n- Unified the types of encodings the library's encryption functions\n utilize for producing ciphertext. This includes databases. They now\n all use the ``LIST_ENCODING``. This greatly increases the efficiency\n of the databases' encryption/decryption, save/load times. And this\n encoding is more space efficient. This change is backwards\n incompatible.\n- The ``LIST_ENCODING`` specification was also changed to produce\n smaller ciphertexts. The salt is no longer encrypted & included as\n the first 256 byte chunk of ciphertext. It is now packaged along with\n ciphertext in the clear & is restricted to being a 256-bit hex\n string.\n- The interfaces for the ``Database`` & ``AsyncDatabase`` were cleaned\n up. Many attributes & functions that were not intended as the public\n interface of the classes were made \"private\". Also, the no longer\n used utilities for encrypting & decrypting under the MAP_ENCODING\n were removed.\n- Updated the ``abytes_xor``, ``bytes_xor``, ``axor`` & ``xor`` generators \n to shrink the size of the ``seed`` that's fed into the ``keystream``. This\n allows the one-time-pad cipher to be more cpu efficient.\n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Fixed various typos, docstrings & tutorials that have no kept up\n with the pace of changes.\n- Various refactorings throughout.\n- The ``akeypair`` & ``keypair`` functions now produce a ``Namespace``\n populated with a 512-bit hex key & a 256-bit hex salt to be more\n consistent with their intended use-case with the one-time-pad cipher.\n- Removed ``aencode_salt``, ``encode_salt``, ``adecode_salt`` & \n ``decode_salt`` functions since they are no longer used in conjunction\n with LIST_ENCODING ciphertexts.\n- Updated tests to recognize these changes.\n- Gave the ``OneTimePad`` class access to a ``BytesIO`` object under a\n new ``io`` attribute.\n\n\n\n\nChanges for version 0.13.0 \n-------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Security Patch: ``xor`` & ``axor`` functions that define the \n one-time-pad cipher had a vulnerability fixed that can leak <1-bit of\n plaintext. The issue was in the way keys were built, where the\n multiplicative products of two key segments were xor'd together. This\n lead to keys being slightly more likely to be positive integers, \n meaning the final bit had a greater than 1/2 probability of being a \n ``0``. The fix is accompanied with an overhaul of the one-time-pad \n cipher which is more efficient, faster, & designed with a better \n understanding of the way bytes are processed & represented. The key\n chunks now do not, & must not, surpass 256 bytes & neither should \n any chunk of plaintext output. Making each chunk deterministically \n 256 bytes allows for reversibly formatting ciphertext to & from \n bytes-like strings. These changes are backwards incompatible with \n prior versions of this package & are strongly recommended.\n- Added ``bytes_xor`` & ``abytes_xor`` functions which take in key \n generators which produce key segments of type bytes instead of hex \n strings.\n- ``AsyncDatabase`` & ``Database`` now save files in bytes format,\n making them much more efficient on disk space. They use the new\n ``BytesIO`` class in the ``generics`` module to transparently convert\n to & from json & bytes. This change is also not backwards compatible.\n- Removed ``acipher``, ``cipher``, ``adecipher``, ``decipher``,\n ``aorganize_encryption_streams``, ``organize_encryption_streams``,\n ``aorganize_decryption_streams``, ``organize_decryption_streams``,\n ``aencrypt``, ``encrypt``, ``adecrypt``, ``decrypt``, ``asubkeys`` &\n ``subkeys`` generators from the ``ciphers`` module & package to slim \n down the code, remove repetition & focus on the cipher tools that \n include hmac authentication.\n- Removed deprecated diffie-hellman methods in ``Ropake`` class. \n- Removed the static ``power10`` dictionary from the package.\n- The default secret salt for the ``Ropake`` class is now derived from the \n contents of a file that's in the databases directory which is chmod'd to \n 0o000 unless needed. \n- Made ``aclient_message_key``, ``client_message_key``, ``aserver_message_key``, \n & ``server_message_key`` ``Ropake`` class methods to help distinguish \n client-to-server & server-to-client message keys which prevents replay \n attacks on the one-message ROPAKE protocol. \n- Added protocol coroutines to the ``Ropake`` class which allow for easily\n engaging in 2DH & 3DH elliptic curve exchanges for servers & clients.\n- Efficiency improvements to the ``aseeder`` & ``seeder`` generator functions\n in the ``randoms`` module. This affects the ``acsprng`` & ``csprng`` objects\n & all the areas in the library that utilize those objects.\n- Changed the repr behavior of ``Comprende`` instances to redact all args &\n kwargs by default to protect cryptographic material from unintentionally\n being displayed on user systems. The repr can display full contents by \n calling the ``enable_debugging`` method of the ``DebugControl`` class.\n- All generator functions decorated with ``comprehension`` are now given\n a ``root`` attribute. This allows direct access to the function without\n needing to instantiate or run it as a ``Comprende`` object. This saves \n a good deal of cpu & time in the overhead that would otherwise be \n incurred by the class. This is specifically more helpful in tight &/or\n lower-level looping.\n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Various refactorings across the library. \n- Fixed various typos, bugs & inaccurate docstrings throughout the library.\n- Add ``chown`` & ``chmod`` functions to the ``asynchs.aos`` module. \n- Now makes new ``multiprocessing.Manager`` objects in the ``asynchs.Processes`` \n & ``asynchs.Threads`` classes to avoid errors that occur when using a stale \n object whose socket connections are closed. \n- Changed ``Ropake`` class' ``adb_login`` & ``db_login`` methods to \n ``adatabase_login_key`` & ``database_login_key``. Also, fix a crash bug in \n those methods. \n- Changed ``Ropake`` class' ``aec25519_pub``, ``ec25519_pub``, ``aec25519_priv`` \n & ``ec25519_priv`` methods to ``aec25519_public_bytes``, ``ec25519_public_bytes``, \n ``aec25519_private_bytes`` & ``ec25519_private_bytes``. \n- Added low-level private methods to ``Ropake`` class which do derivation \n & querying of the default class key & salt. \n- Behavior changes to the ``ainverse_int`` & ``inverse_int`` functions in the \n ``generics`` module to allow handling bases represented in ``str`` or ``bytes`` \n type strings. \n- Behavior & name changes to the ``abinary_tree`` & ``binary_tree`` functions in the \n ``generics`` module to ``abuild_tree`` & ``build_tree``. They now allow making \n uniform trees of any width & depth, limited only by the memory in a \n user's machine. \n- Provided new ``acsprbg`` & ``csprbg`` objects to the library that return 512-bits \n of cryptographically secure pseudo-random ``bytes`` type strings. They are \n made by the new ``abytes_seeder`` & ``bytes_seeder`` generators. \n- The ``csprng``, ``acsprng``, ``csprbg`` & ``acsprbg`` objects were \n wrapped in functions that automatically restart the generators if they're\n stalled / interrupted during a call. This keeps the package from melting\n down if it can no longer call the CSPRNGs for new entropy.\n- Cleaned up & simplified ``table_key`` functions in the ``keygens`` module. \n- Changed the output of ``asafe_symm_keypair`` & ``safe_symm_keypair`` functions \n to contain bytes values not their hex-only representation. Also removed \n these functions from the main imports of the package since they are slow \n & their main contribution is calling ``arandom_number_generator`` & \n ``random_number_generator`` to utilize a large entropy pool when starting\n CSPRNGs.\n- Added new values to the ``bits`` dictionary.\n- Added ``apad_bytes``, ``pad_bytes``, ``adepad_bytes`` & ``depad_bytes``\n functions which use ``shake_256`` to pad/depad plaintext bytes to & from\n multiples of 256 bytes. They take in a key to create the padding. \n This method is intended to also aid in protecting against padding\n oracle attacks.\n\n\n\n\nChanges for version 0.12.0 \n-------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- The OPAKE protocol was renamed to ROPAKE, an acronym for Ratcheting \n Opaque Password Authenticated Key Exchange. This change was necessary \n since OPAKE is already a name for an existing PAKE protocol. This change \n also means the ``Opake`` class name was changed to ``Ropake``. \n- The ``Ropake`` class' registration algorithm was slightly modified to \n use the generated Curve25519 ``shared_key`` an extra time in the key \n derivation process. This shouldn't break any currently authenticated \n sessions. \n- The ``asyncio_contextmanager`` package is no longer a listed dependency \n in ``setup.py``. The main file from that package was copied over into the \n ``/aiootp`` directory in order to remove the piece of code that caused \n warnings to crop up when return values were retrieved from async \n generators. This change will put an end to this whack-a-mole process of \n trying to stop the warnings with try blocks scattered about the codebase. \n- Added ``asave_tag``, ``save_tag``, ``asave_file`` & ``save_file`` methods \n to the database classes so that specific entries can be saved to disk \n without having to save the entire database which is much more costly. The \n manifest file isn't saved to disk when these methods are used, so if a \n tag file isn't already saved in the database, then the saved files will \n not be present in the manifest or in the cache upon subsequent loads of \n the database. The saved file will still however be saved on the \n filesystem, though unbeknownst to the database instance.\n- The ``Namespace`` class now redacts all obvious key material in instance \n repr's, which is any 64+ hex character string, or any number with 64+ \n decimal digits. \n- Removed the experimental recursive value retrieval within ``Comprende``'s \n ``__aexamine_sent_exceptions`` & ``__examine_sent_exceptions`` methods. \n This change leads to more reliable & faster code, in exchange for an \n unnecessary feature being removed. \n- Bug fix of the ``auuids`` & ``uuids`` methods by editing the code in \n the ``asyncio_contextmanager`` dependency & using the patched package \n instead of the ``comprehension`` decorator for the ``arelay`` & ``relay`` \n methods of ``Comprende``. Their internal algorithms was also updated to \n be simpler, but are incompatible with the outputs of past versions of \n these methods. \n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Various refactorings & documentation additions / modifications throughout \n the library. \n- Various small bug fixes.\n- The shared keys derived from the ``Ropake`` protocol are now returned in \n a ``Namespace`` object instead of a raw dictionary, which allows the \n values to be retrieved by dotted &/or bracketed lookup. \n- The ``atest_hmac`` & ``test_hmac`` algorithms / methods were made more \n efficient & were refactored. Now they call ``atime_safe_equality`` &\n ``time_safe_equality`` internally, which are new methods that can apply\n the non-constant time but randomized timing comparisons on any pairs of\n values.\n\n\n\n\nChanges for version 0.11.0 \n-------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- The Opake protocol was made greatly more efficient. This was done by \n replacing the diffie-hellman verifiers with a hash & xor commit & reveal\n system. Most hashing was made more efficient my using quicker & smaller\n ``sha_512`` function instead of ``nc_512``, & streamlining the protocol.\n- The ``Opake.client`` & ``Opake.client_registration`` methods now take\n an instantiated client database instead of client credentials which \n improves security, efficiency & usability. This change reduces the amount\n of exposure received by user passwords & other credentials. It also \n simplifies usage of the protocol by only needing to carry around a \n database instead of a slew of credentials, which is also faster, since\n the credentials are passed through the cpu & memory hard ``passcrypt``\n function everytime to open the database.\n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Heavy refactorings & documentation additions / modifications of the \n ``Opake`` class. Removed the ``Opake.ainit_database`` & ``Opake.init_database``\n methods, & made the ``salt`` default argument parameter in \n ``Opake.aclient_database``, ``Opake.client_database``, ``Opake.adb_login`` &\n ``Opake.db_login`` into a keyword only argument so any extra user defined\n ``credentials`` are able to be passed without specifying a salt.\n- The decorators for the ``Comprende.arelay`` & ``Comprende.relay`` methods \n were changed from ``@asyncio_contextmanager.async_contextmanager`` to\n ``@comprehension()`` to stop that package from raising exceptions when\n we retrieve return values from async generators.\n\n\n\n\nChanges for version 0.10.1 \n-------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Added ``Processes`` & ``Threads`` classes to ``asynchs.py`` which abstract \n spawning & getting return values from async & sync functions intended to \n be run in threads, processes or pools of the former types. This simplifies \n & adds time control to usages of processes & threads throughout the \n library. \n- Reduced the effectiveness of timing analysis of the modular exponentiation \n in the ``Opake`` class' verifiers by making the process return values \n only after discrete intervals of time. Timing attacks on that part of the \n protocol may still be viable, but should be significantly reduced. \n- Bug fix in ``Comprende`` which should take care of warnings raised from \n the ``aiocontext`` package when retrieving async generator values by \n raising ``UserWarning`` within them. \n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Heavy refactorings of the ``Opake`` class. \n- Various refactorings & cleanups around the package. \n- Further add ``return_exceptions=True`` flag to gather calls in ``ciphers.py``. \n- Added ``is_registration`` & ``is_authentication`` which take a client \n hello message that begin the ``Opake`` protocol, & return ``False`` if \n the message is not either a registration or authentication message, \n respectively, & return ``\"Maybe\"`` otherwise, since these functions can't \n determine without running the protocol whether or not the message is \n valid. \n\n\n\n\nChanges for version 0.10.0 \n-------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Added a new oblivious, one-message, password authenticated key exchange \n protocol class in ``aiootp.ciphers.Opake``. It is a first attempt at the \n protocol, which works rather well, but may be changed or cleaned up in a \n future update. \n- Added the ``cryptography`` package as a dependency for elliptic curve \n 25519 diffie-hellman key exchange in the ``Opake`` protocol. \n- Fix buggy data processing functions in ``generics.py`` module. \n- Added ``silent`` flag to ``AsyncDatabase`` & ``Database`` methods, which \n allows their instances to finish initializing even if a file is missing \n from the filesystem, normally causing a ``FileNotFoundError``. This makes \n trouble-shooting corrupted databases easier. \n- Added new ``aiootp.paths.SecurePath`` function which returns the path to \n a unique directory within the database's default directory. The name of \n the returned directory is a cryptographic value used to create & open the \n default database used by the ``Opake`` class to store the cryptographic \n salt that secures the class' client passwords. It's highly recommended \n to override this default database by instantiating the Opake class with \n a custom user-defined key. The instance doesn't need to be saved, since \n all the class' methods are either class or static methods. The ``__init__`` \n method only changes the class' default database to one opened with the \n user-defined ``key`` &/or ``directory`` kwargs, & should really only be \n done once at the beginning of an application. \n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Various refactorings & cleanups around the package. \n- Added ``Comprende`` class feature to return the values from even the \n generators within an instance's arguments. This change better returns \n values to the caller from chains of ``Comprende`` generators. \n- Fixed ``commons.BYTES_TABLE`` missing values. \n- Added ``commons.DH_PRIME_4096_BIT_GROUP_16`` & ``commons.DH_GENERATOR_4096_BIT_GROUP_16`` \n constants for use in the ``Opake`` protocol's public key verifiers. \n- Added other values to the ``commons.py`` module. \n- Added new very large no-collision hash functions to the ``generics.py`` \n module used to xor with diffie-hellman public keys in the ``Opake`` class. \n- Added new ``wait_on`` & ``await_on`` ``Comprende`` generators to ``generics.py`` \n which waits for a queue or container to be populated & yields it whenever \n it isn't empty. \n\n\n\n\nChanges for version 0.9.3 \n------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Speed & efficiency improvements in the ``Comprende`` class & ``azip``. \n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Various refactorings & code cleanups.\n- Added ``apop`` & ``pop`` ``Comprende`` generators to the library.\n- Switched the default character table in the ``ato_base``, ``to_base``, \n ``afrom_base``, & ``from_base`` chainable generator methods from the 62\n character ``ASCII_ALPHANUMERIC`` table, to the 95 character ``ASCII_TABLE``.\n- Made the digits generators in ``randoms.py`` automatically create a new\n cryptographically secure key if a key isn't passed by a user.\n- Some extra data processing functions added to ``generics.py``.\n\n\n\n\nChanges for version 0.9.2 \n------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Added ``passcrypt`` & ``apasscrypt`` instance methods to ``OneTimePad``,\n ``Keys``, & ``AsyncKeys`` classes. They produce password hashes that are\n not just secured by the salt & passcrypt algorithm settings, but also by\n their main symmetric instance keys. This makes passwords infeasible to\n crack without also compromising the instance's 512-bit key.\n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Further improvements to the random number generator in ``randoms.py``.\n Made its internals less sequential thereby raising the bar of work needed\n by an attacker to successfully carry out an order prediction attack.\n- Added checks in the ``Passcrypt`` class to make sure both a salt & \n password were passed into the algorithm.\n- Switched ``PermissionError`` exceptions in ``Passcrypt._validate_args``\n to ``ValueError`` to be more consistent with the rest of the class.\n- Documentation updates / fixes.\n\n\n\n\nChanges for version 0.9.1 \n------------------------- \n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Now any falsey values for the ``salt`` keyword argument in the library's \n ``keys``, ``akeys``, ``bytes_keys``, ``abytes_keys``, ``subkeys``, & \n ``asubkeys`` infinite keystream generators, & other functions around the \n library, will cause them to generate a new cryptographically secure \n pseudo-random value for the salt. It formerly only did this when ``salt`` \n was ``None``. \n- The ``seeder`` & ``aseeder`` generators have been updated to introduce \n 512 new bits of entropy from ``secrets.token_bytes`` on every iteration \n to ensure that the CSPRNG will produce secure outputs even if its \n internal state is somehow discovered. This also allows for simply calling \n the CSPRNG is enough, there's no longer a strong reason to pass new \n entropy into it manually, except to add even more entropy as desired.\n- Made ``size`` the last keywordCHECKSUMS.txt argument in ``encrypt`` & \n ``aencrypt`` to better mirror the signatures for rest of the library. \n- Added ``token_bits`` & ``atoken_bits`` functions to ``randoms.py`` which \n are renamings of ``secrets.randbits``. \n- Refactored & improved the security og ``randoms.py``'s random number \n generator. \n\n\n\n\nChanges for version 0.9.0 \n------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Added hmac codes to ciphertext for the following functions: ``json_encrypt``, \n ``ajson_encrypt``, ``bytes_encrypt``, ``abytes_encrypt``, \n ``Database.encrypt`` & ``AsyncDatabase.aencrypt``. This change greatly \n increases the security of ciphertext by ensuring it hasn't been modified \n or tampered with maliciously. One-time pad ciphertext is maleable, so \n without hmac validation it can be changed to successfully allow \n decryption but return the wrong plaintext. These functions are the \n highest level abstractions of the library for encryption/decryption, \n which made them excellent targets for this important security update. \n As well, it isn't easily possible for the library to provide hmac codes \n for generators that produce ciphertext, because the end of a stream of \n ciphertext isn't known until after the results have left the scope \n of library code. So users will need to produce their own hmac codes for \n generator ciphertext unless we find an elegant solution to this issue. \n These functions now all return dictionaries with the associated hmac \n stored in the ``\"hmac\"`` entry. The bytes functions formerly returned \n lists, now their ciphertext is available from the ``\"ciphertext\"`` entry. \n And, all database files will have an hmac attached to them now. These \n changes were designed to still be compatible with old ciphertexts but \n they'll likely be made incompatible by the v0.11.x major release. \n- Only truthy values are now valid ``key`` keyword arguments in the \n library's ``keys``, ``akeys``, ``bytes_keys``, ``abytes_keys``, ``subkeys``, \n & ``asubkeys`` infinite keystream generators. Also now seeding extra entropy \n into ``csprng`` & ``acsprng`` when ``salt`` is falsey within them. \n- Only truthy values are now valid for ``password`` & ``salt`` arguments in \n ``apasscrypt``, ``passcrypt`` & their variants. \n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Updates to documentation & ``README.rst`` tutorials.\n- The ``kb``, ``cpu``, & ``hardness`` arguments in ``sum_passcrypt`` &\n ``asum_passcrypt`` chainable generator methods were switched to keyword\n only arguments.\n\n\n\n\nChanges for version 0.8.1 \n------------------------- \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Added ``sum_passcrypt`` & ``asum_passcrypt`` chainable generator methods \n to ``Comprende`` class. They cumulatively apply the passcrypt algorithm \n to each yielded value from an underlying generator with the passcrypt'd \n prior yielded result used as a salt. This allows making proofs of work, \n memory & space-time out of iterations of the passcrypt algorithm very \n simple. \n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Various inaccurate docstrings fixed. \n- Various refactorings of the codebase. \n- Made ``kb``, ``cpu``, & ``hardness`` arguments into keyword only arguments \n in ``AsyncDatabase`` & ``Database`` classes. \n- The ``length`` keyword argument in functions around the library was \n changed to ``size`` to be consistent across the whole package. Reducing \n the cognitive burden of memorizing more than one name for the same concept. \n- Various efficiency boosts. \n- Edits to ``README.rst``. \n- Added ``encode_salt``, ``aencode_salt``, ``decode_salt`` & ``adecode_salt`` \n functions to the library, which gives access to the procedure used to \n encrypt & decrypt the random salt which is often the first element \n produced in one-time pad ciphertexts. \n- Added cryptographically secure pseudo-random values as default keys in \n encryption functions to safeguard against users accidentally encrypting \n data without specifying a key. This way, such mistakes will produce \n ciphertext with an unrecoverable key, instead of without a key at all. \n\n\n\n\nChanges for version 0.8.0\n-------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Fix ``test_hmac``, ``atest_hmac`` functions in the keys & database \n classes. The new non-constant-time algorithm needs a random salt to be \n added before doing the secondary hmac to prevent some potential exotic \n forms of chosen plaintext/ciphertext attacks on the algorithm. The last \n version of the algorithm should not be used. \n- The ``Keys`` & ``AsyncKeys`` interfaces were overhauled to remove the \n persistance of instance salts. They were intended to be updated by users \n with the ``reset`` & ``areset`` methods, but that cannot be guaranteed \n easily through the class, so it is an inappropriate interface since \n reusing salts for encryption is completely insecure. The instances do\n still maintain state of their main encryption key, & new stateful methods\n for key generation, like ``mnemonic`` & ``table_key``, have been added.\n The ``state`` & ``astate`` methods have been removed.\n- Gave ``OneTimePad`` instances new stateful methods from the ``ciphers.py`` \n module & ``keygens.py`` keys classes. Its instances now remember the main \n symmetric key behind the ``key`` property & automatically passes it as a \n keyword argument to the methods in ``OneTimePad.instance_methods``.\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Update ``CHANGES.rst`` file with the updates that were not logged for\n v0.7.1.\n- ``BYTES_TABLE`` was turned into a list so that the byte characters can \n be retrieved instead of their ordinal numbers.\n\n\n\n\nChanges for version 0.7.1\n-------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Fix a mistake in the signatures of ``passcrypt`` & ``apasscrypt. The args\n ``kb``, ``cpu`` & ``hardness`` were changed into keyword only arguments\n to mitigate user mistakes, but the internal calls to those functions were\n still using positional function calls, which broke the api. This issue\n is now fixed.\n\n\n\n\nChanges for version 0.7.0\n-------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Replaced usage of bare ``random`` module functions, to usage of an \n instance of ``random.Random`` to keep from messing with user's settings \n on that module. \n- Finalized the algorithm for the ``passcrypt`` & ``apasscrypt`` functions. \n The algorithm is now provably memory & cpu hard with a wide security \n margin with adequate settings. The algorithm isn't likely change with \n upcoming versions unless a major flaw is found. \n- The default value for the ``cpu`` argument in ``passcrypt`` & ``apasscrypt`` \n is now ``3`` & now directly determines how many hash iterations are done \n for each element in the memory cache. This provides much more \n responsiveness to users & increases the capacity to impact resource cost\n with less tinkering. \n- Switched the ``AsyncKeys.atest_hmac`` & ``Keys.test_hmac`` methods to a \n scheme which is not constant time, but which instead does not leak useful \n information. It does this by not comparing the hmacs of the data, but of \n a pair of secondary hmacs. The timing analysis itself is now dependant \n on knowledge of the key, since any conclusions of such an analysis would \n be unable correlate its findings with any supplied hmac without it. \n- Added ``test_hmac`` & ``atest_hmac`` to the database classes, & changed \n their hmac algorithm from ``sha3_512`` to ``sha3_256``. \n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Various code cleanups, refactorings & speedups.\n- Several fixes to inaccurate documentation.\n- Several fixes to inaccurate function signatures.\n- Added ``mnemonic`` & ``amnemonic`` key generators to ``keygens.py`` with\n a wordlist 2048 entries long. A custom wordlist can also be passed in.\n- Minor changes in ``Comprende`` to track down a bug in the functions that \n use the asyncio_contextmanager package. It causes a warning when asking\n async generators to return (not yield) values.\n- Some refactoring of ``random_number_generator`` & ``arandom_number_generator``.\n\n\n\n\nChanges for version 0.6.0\n-------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Replaced the usage of ``os.urandom`` within the package with \n ``secrets.token_bytes`` to be more reliable across platforms. \n- Replaced several usages of ``random.randrange`` within ``randoms.py`` to \n calls to ``secrets.token_bytes`` which is faster & more secure. It\n now also seeds ``random`` module periodically prior to usage.\n- Changed the internal cache sorting algorithm of ``passcrypt`` & \n ``apasscrypt`` functions. The key function passed to ``list.sort(key=key)`` \n now not only updates the ``hashlib.sha3_512`` proof object with \n each element in the cache, but with it's own current output. This change \n is incompatible with previous versions of the functions. The key function \n is also trimmed down of unnecessary value checking. \n- The default value for the ``cpu`` argument in ``passcrypt`` & ``apasscrypt``\n is now ``40_000``. This is right at the edge of when the argument begins\n impacting the cpu work needed to comptute the password hash when the ``kb``\n argument is the default of ``1024``.\n- Switched the ``AsyncKeys.atest_hmac`` & ``Keys.test_hmac`` methods to a \n constant time algorithm.\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Various code cleanups, refactorings & speedups.\n- Added a ``concurrent.futures.ThreadPoolExecutor`` instance to the ``asynchs``\n module for easily spinning off threads. It's available under \n ``asynchs.thread_pool``.\n- Added ``sort`` & ``asort`` chainable generator method to the ``Comprende`` \n class. They support sorting by a ``key`` sorting function as well.\n- Changed the name of ``asynchs.executor_wrapper`` to ``asynchs.wrap_in_executor``.\n- Changed the name of ``randoms.non0_digit_stream``, ``randoms.anon0_digit_stream``,\n ``randoms.digit_stream`` & ``randoms.adigit_stream`` to ``randoms.non_0_digits``,\n ``randoms.anon_0_digits``, ``randoms.digits`` & ``randoms.adigits``.\n- Several fixes to inaccurate documentation.\n- ``apasscrypt`` & ``Passcrypt.anew`` now use the synchronous version of the \n algorithm internally because it's faster & it doesn't change the \n parallelization properties of the function since it's already run \n automatically in another process.\n- Added ``shuffle``, ``ashuffle``, ``unshuffle``, & ``aunshuffle`` functions\n to ``randoms.py`` that reorder sequences pseudo-randomly based on their\n ``key`` & ``salt`` keyword arguments.\n- Fixed bugs in ``AsyncKeys`` & ``debuggers.py``.\n- Added ``debugger`` & ``adebugger`` chainable generator methods to the\n ``Comprende`` class which benchmarks & inspects running generators with\n an inline syntax.\n\n\n\n\nChanges for version 0.5.1\n-------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Fixed a bug in the methods ``auuids`` & ``uuids`` of the database classes \n that assigned to a variable within a closure that was nonlocal but which \n wasn't declared non-local. This caused an error which made the methods \n unusable. \n- Added ``passcrypt`` & ``apasscrypt`` functions which are designed to be \n tunably memory & cpu hard password-based key derivation function. It was \n inspired by the scrypt protocol but internally uses the library's tools. \n It is a first attempt at the protocol, it's internal details will likely \n change in future updates. \n- Added ``bytes_keys`` & ``abytes_keys`` generators, which are just like \n the library's ``keys`` generator, except they yield the concatenated \n ``sha3_512.digest`` instead of the ``sha3_512.hexdigest``. \n- Added new chainable generator methods to the ``Comprende`` class for \n processing bytes, integers, & hex strings into one another. \n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Various code cleanups.\n- New tests added to the test suite for ``passcrypt`` & ``apasscrypt``.\n- The ``Comprende`` class' ``alist`` & ``list`` methods can now be passed\n a boolean argument to return either a ``mutable`` list directly from the \n lru_cache, or a copy of the cached list. This list is used by the \n generator itself to yield its values, so wilely magic can be done on the\n list to mutate the underlying generator's results. \n\n\n\n\nChanges for version 0.5.0\n-------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Added interfaces in ``Database`` & ``AsyncDatabase`` to handle encrypting\n & decrypting streams (``Comprende`` generators) instead of just raw json \n data. They're methods called ``encrypt_stream``, ``decrypt_stream``,\n ``aencrypt_stream``, & ``adecrypt_stream``.\n- Changed the attribute ``_METATAG`` used by ``Database`` & ``AsyncDatabase`` \n to name the metatags entry in the database. This name is smaller, cleaner \n & is used to prevent naming collisions between user entered values & the \n metadata the classes need to organize themselves internally. This change \n will break databases from older versions keeping them from accessing their \n metatag child databases.\n- Added the methods ``auuids`` & ``uuids`` to ``AsyncDatabase`` & ``Database``\n which return coroutines that accept potentially sensitive identifiers &\n turns them into salted ``size`` length hashes distinguished by a ``salt``\n & a ``category``.\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Various code & logic cleanups / speedups.\n- Refactorings of the ``Database`` & ``AsyncDatabase`` classes.\n- Various inaccurate docstrings fixed.\n\n\n\n\nChanges for version 0.4.0\n-------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Fixed bug in ``aiootp.abytes_encrypt`` function which inaccurately called\n a synchronous ``Comprende`` end-point method on the underlying async\n generator, causing an exception and failure to function.\n- Changed the procedures in ``akeys`` & ``keys`` that generate their internal\n key derivation functions. They're now slightly faster to initialize &\n more theoretically secure since each internal state is fed by a seed\n which isn't returned to the user. This encryption algorithm change is \n incompatible with the encryption algorithms of past versions.\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Various code cleanups.\n- Various inaccurate docstrings fixed.\n- Keyword arguments in ``Keys().test_hmac`` & ``AsyncKeys().atest_hmac``\n had their order switched to be slightly more friendly to use.\n- Added documentation to ``README.rst`` on the inner workings of the\n one-time-pad algorithm's implementation.\n- Made ``Compende.arandom_sleep`` & ``Compende.random_sleep`` chainable\n generator methods.\n- Changed the ``Compende.adelimit_resize`` & ``Compende.delimit_resize``\n algorithms to not yield inbetween two joined delimiters in a sequence\n being resized.\n\n\n\n\nChanges for version 0.3.1\n-------------------------\n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Fixed bug where a static method in ``AsyncDatabase`` & ``Database`` was \n wrongly labelled a class method causing a failure to initialize.\n\n\n\n\nChanges for version 0.3.0\n-------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- The ``AsyncDatabase`` & ``Database`` now use the more secure ``afilename`` \n & ``filename`` methods to derive the hashmap name and encryption streams\n from a user-defined tag internal to their ``aencrypt`` / ``adecrypt`` / \n ``encrypt`` / ``decrypt`` methods, as well as, prior to them getting called. \n This will break past versions of databases' ability to open their files.\n- The package now has built-in functions for using the one-time-pad \n algorithm to encrypt & decrypt binary data instead of just strings\n or integers. They are available in ``aiootp.abytes_encrypt``, \n ``aiootp.abytes_decrypt``, ``aiootp.bytes_encrypt`` & ``aiootp.bytes_decrypt``.\n- The ``Comprende`` class now has generators that do encryption & decryption \n of binary data as well. They are available from any ``Comprende`` generator\n by the ``abytes_encrypt``, ``abytes_decrypt``, ``bytes_encrypt`` & ``bytes_decrypt`` \n chainable method calls.\n \n \nMinor Changes\n^^^^^^^^^^^^^\n\n- Fixed typos and inaccuracies in various docstrings.\n- Added a ``__ui_coordination.py`` module to handle inserting functionality \n from higher-level to lower-level modules and classes.\n- Various code clean ups and redundancy eliminations.\n- ``AsyncKeys`` & ``Keys`` classes now only update their ``self.salt`` key\n by default when their ``areset`` & ``reset`` methods are called. This\n aligns more closely with their intended use.\n- Added ``arandom_sleep`` & ``random_sleep`` chainable methods to the\n ``Comprende`` class which yields outputs of generators after a random \n sleep for each iteration.\n- Added several other chainable methods to the ``Comprende`` class for\n string & bytes data processing. They're viewable in ``Comprende.lazy_generators``.\n- Added new, initial tests to the test suite.\n\n\n\n\nChanges for version 0.2.0\n-------------------------\n\n\nMajor Changes\n^^^^^^^^^^^^^\n\n- Added ephemeral salts to the ``AsyncDatabase`` & ``Database`` file \n encryption procedures. This is a major security fix, as re-encryption \n of files with the same tag in a database with the same open key would \n use the same streams of key material each time, breaking encryption if \n two different versions of a tag file's ciphertext stored to disk were \n available to an adversary. The database methods ``encrypt``, ``decrypt``, \n ``aencrypt`` & ``adecrypt`` will now produce and decipher true one-time \n pad ciphertext with these ephemeral salts. \n- The ``aiootp.subkeys`` & ``aiootp.asubkeys`` generators were revamped \n to use the ``keys`` & ``akeys`` generators internally instead of using \n their own, slower algorithm. \n- ``AsyncDatabase`` file deletion is now asynchronous by running the \n ``builtins.os.remove`` function in an async thread executor. The \n decorator which does the magic is available at ``aiootp.asynchs.executor_wrapper``. \n\n\nMinor Changes\n^^^^^^^^^^^^^\n\n- Fix typos in ``__root_salt`` & ``__aroot_salt`` docstrings. Also replaced \n the ``hash(self)`` argument for their ``lru_cache`` & ``alru_cache`` \n with a secure hmac instead. \n- add ``gi_frame``, ``gi_running``, ``gi_code``, ``gi_yieldfrom``, \n ``ag_frame``, ``ag_running``, ``ag_code`` & ``ag_await`` properties to \n ``Comprende`` class to mirror async/sync generators more closely. \n- Remove ``ajson_encrypt``, ``ajson_decrypt``, ``json_encrypt``, \n ``json_decrypt`` functions' internal creation of dicts to contain the \n plaintext. It was unnecessary & therefore wasteful. \n- Fix docstrings in ``OneTimePad`` methods mentioning ``parent`` kwarg which \n is a reference to deleted, refactored code. \n- Fix incorrect docstrings in databases ``namestream`` & ``anamestream`` \n methods. \n- Added ``ASYNC_GEN_THROWN`` constant to ``Comprende`` class to try to stop \n an infrequent & difficult to debug ``RuntimeError`` when async generators \n do not stop after receiving an ``athrow``. \n- Database tags are now fully loaded when they're copied using the methods \n ``into_namespace`` & ``ainto_namespace``. \n- Updated inaccurate docstrings in ``map_encrypt``, ``amap_encrypt``, \n ``map_decrypt`` & ``amap_decrypt`` ``OneTimePad`` methods. \n- Added ``acustomize_parameters`` async function to ``aiootp.generics`` \n module. \n- Various code clean ups. \n\n\n\n\nChanges for version 0.1.0 \n------------------------- \n\n\nMinor Changes \n^^^^^^^^^^^^^ \n\n- Initial version. \n\n\nMajor Changes \n^^^^^^^^^^^^^ \n\n- Initial version. \n\n\n\n\n_`Known Issues` ................................... `Table Of Contents`_\n========================================================================\n\n- The test suite for this software is under construction, & what tests\n have been published are currently inadequate to the needs of\n cryptography software.\n- This package is currently in beta testing & active development,\n meaning major changes are still possible when there are really good\n reasons to do so. Contributions are welcome. Send us a message if \n you spot a bug or security vulnerability:\n \n - gonzo.development@protonmail.ch\n - rmlibre@riseup.net\n - ed25519-key: 70d1740f2a439da98243c43a4d7ef1cf993b87a75f3bb0851ae79de675af5b3b\n - x25519-key: 4457276dbcae91cc5b69f1aed4384b9eb6f933343bb44d9ed8a80e2ce438a450\n\n\n\n",
"bugtrack_url": null,
"license": "AGPLv3",
"summary": "aiootp - an asynchronous pseudo one-time pad based crypto and anonymity library.",
"version": "0.22.0",
"split_keywords": [
"xor",
"key",
"salt",
"pepper",
"nonce",
"aad",
"iv",
"siv",
"resuse",
"misuse",
"aead",
"auth",
"authenticated",
"authentication",
"shmac",
"hmac",
"nmac",
"mac",
"digest",
"integrity",
"infosec",
"opsec",
"appsec",
"stream",
"cipher",
"chunky2048",
"chunky",
"encrypt",
"plaintext",
"decrypt",
"ciphertext",
"passcrypt",
"passphrase",
"password",
"based",
"derivation",
"function",
"3dh",
"2dh",
"25519",
"x25519",
"ed25519",
"curve25519",
"diffie",
"hellman",
"sign",
"signature",
"verify",
"verification",
"db",
"database",
"store",
"user",
"uuid",
"unique",
"guid",
"global",
"transparent",
"encryption",
"decryption",
"chunky2048",
"indistinguishable",
"pseudo",
"one",
"time",
"pad",
"onetimepad",
"domain-specific",
"kdf",
"separation",
"bits",
"64",
"128",
"256",
"512",
"1024",
"2048",
"4096",
"hash",
"sha",
"sha3",
"sha-3",
"keccak",
"ephemeral",
"byte",
"entropy",
"prf",
"prg",
"prp",
"rng",
"prng",
"csprng",
"cryptographically",
"secure",
"random",
"number",
"generator",
"bitwise",
"operations",
"information",
"cyber",
"security",
"chosen",
"attack",
"resistance",
"resistant",
"tweak",
"tweakable",
"anonymous",
"anonymity",
"pseudonymous",
"symmetric",
"asymmetric",
"communications",
"utilities",
"simple",
"clean",
"code",
"crypto",
"cryptology",
"cryptography",
"beta",
"testing",
"data",
"science",
"processing",
"await",
"async",
"asyncio",
"parallel",
"concurrency",
"coroutine",
"coroutines",
"comprehension"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "3389a6c01271dc78909a97ab27ba28c1d799cb03806fc51a3d5b3f9f0ac1e320",
"md5": "e54773ccf6751f8b0504fe4d2c5263ff",
"sha256": "b43f9b7072daead910cc0e339a052ccb6649c4a90ab1f85a4130e2fae32ee201"
},
"downloads": -1,
"filename": "aiootp-0.22.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "e54773ccf6751f8b0504fe4d2c5263ff",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 212495,
"upload_time": "2023-02-07T16:38:54",
"upload_time_iso_8601": "2023-02-07T16:38:54.934282Z",
"url": "https://files.pythonhosted.org/packages/33/89/a6c01271dc78909a97ab27ba28c1d799cb03806fc51a3d5b3f9f0ac1e320/aiootp-0.22.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "12f17efec630717d776118e302c51bb842b4ba3c06e639f680663d4a8456703d",
"md5": "ba09c7d79599e22cd8be9ba10514bd08",
"sha256": "863b91ccae55e62b4d46013b918fcfe8e840efbe7f308d81c16e6956c4c103d0"
},
"downloads": -1,
"filename": "aiootp-0.22.0.tar.gz",
"has_sig": false,
"md5_digest": "ba09c7d79599e22cd8be9ba10514bd08",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 415428,
"upload_time": "2023-02-07T16:39:03",
"upload_time_iso_8601": "2023-02-07T16:39:03.637676Z",
"url": "https://files.pythonhosted.org/packages/12/f1/7efec630717d776118e302c51bb842b4ba3c06e639f680663d4a8456703d/aiootp-0.22.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2023-02-07 16:39:03",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "aiootp"
}
