====================================
aws-msk-iam-sasl-signer-python
====================================
|Version| |Python| |Build| |License| |SecurityScan|
.. |Build| image:: https://github.com/aws/aws-msk-iam-sasl-signer-python/actions/workflows/run-tests.yml/badge.svg?branch=main
:target: https://github.com/aws/aws-msk-iam-sasl-signer-python/actions/workflows/run-tests.yml
:alt: Build status
.. |Python| image:: https://img.shields.io/pypi/pyversions/aws-msk-iam-sasl-signer-python.svg?style=flat
:target: https://pypi.python.org/pypi/aws-msk-iam-sasl-signer-python/
:alt: Python Versions
.. |Version| image:: http://img.shields.io/pypi/v/aws-msk-iam-sasl-signer-python.svg?style=flat
:target: https://pypi.python.org/pypi/aws-msk-iam-sasl-signer-python/
:alt: Package Version
.. |License| image:: http://img.shields.io/pypi/l/aws-msk-iam-sasl-signer-python.svg?style=flat
:target: https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/LICENSE
:alt: License
.. |SecurityScan| image:: https://github.com/aws/aws-msk-iam-sasl-signer-python/actions/workflows/securityscan.yml/badge.svg?branch=main
:target: https://github.com/aws/aws-msk-iam-sasl-signer-python/actions/workflows/securityscan.yml
:alt: Security Scan
This is an Amazon MSK Library in Python. This library provides a function to generates a base 64 encoded signed url
to enable authentication/authorization with an MSK Cluster.
The signed url is generated by using your IAM credentials.
* Free software: Apache Software License 2.0
Features
--------
* Provides a function to generate auth token using IAM credentials from the AWS default credentials chain.
* Provides a function to generate auth token using IAM credentials from the AWS named profile.
* Provides a function to generate auth token using assumed IAM role's credentials.
* Provides a function to generate auth token using a CredentialProvider. The CredentialProvider should be inherited from botocore.credentials.CredentialProvider class.
Get Started
-----------
* For installation, refer to `installation guide`_
* In order to use the signer library with a Kafka client library with SASL/OAUTHBEARER mechanism, add the callback function in your code.
* For example, here is the sample code to use with `dpkp/kafka-python`_ library:
.. code-block:: python
from kafka import KafkaProducer
from kafka.errors import KafkaError
import socket
import time
from aws_msk_iam_sasl_signer import MSKAuthTokenProvider
class MSKTokenProvider():
def token(self):
token, _ = MSKAuthTokenProvider.generate_auth_token('<my aws region>')
return token
tp = MSKTokenProvider()
producer = KafkaProducer(
bootstrap_servers='<my bootstrap string>',
security_protocol='SASL_SSL',
sasl_mechanism='OAUTHBEARER',
sasl_oauth_token_provider=tp,
client_id=socket.gethostname(),
)
topic = "<my-topic>"
while True:
try:
inp=input(">")
producer.send(topic, inp.encode())
producer.flush()
print("Produced!")
except Exception:
print("Failed to send message:", e)
producer.close()
* Here is a sample consumer with `confluent-kafka-python`_ library :
.. code-block:: python
from confluent_kafka import Consumer
import socket
import time
from aws_msk_iam_sasl_signer import MSKAuthTokenProvider
def oauth_cb(oauth_config):
auth_token, expiry_ms = MSKAuthTokenProvider.generate_auth_token("<my aws region>")
# Note that this library expects oauth_cb to return expiry time in seconds since epoch, while the token generator returns expiry in ms
return auth_token, expiry_ms/1000
c = Consumer({
"debug": "all",
'bootstrap.servers': "<my bootstrap string>",
'client.id': socket.gethostname(),
'security.protocol': 'SASL_SSL',
'sasl.mechanisms': 'OAUTHBEARER',
'oauth_cb': oauth_cb,
'group.id': 'mygroup',
'auto.offset.reset': 'earliest'
})
c.subscribe(['<my-topic>'])
print("Starting consumer!")
while True:
msg = c.poll(5)
if msg is None:
continue
if msg.error():
print("Consumer error: {}".format(msg.error()))
continue
print('Received message: {}'.format(msg.value().decode('utf-8')))
c.close()
* In order to use a named profile to generate token, replace the token() function with code below :
.. code-block:: python
class MSKTokenProvider():
def token(self):
oauth2_token, _ = MSKAuthTokenProvider.generate_auth_token_from_profile('<your aws region>', '<named_profile>')
return oauth2_token
* In order to use a role arn to generate token, replace the token() function with code below :
.. code-block:: python
class MSKTokenProvider():
def token(self):
oauth2_token, _ = MSKAuthTokenProvider.generate_auth_token_from_role_arn('<your aws region>', '<role_arn>')
return oauth2_token
* In order to use a custom credentials provider, replace the token() function with code below :
.. code-block:: python
class MSKTokenProvider():
def token(self):
oauth2_token, _ = MSKAuthTokenProvider.generate_auth_token_from_credentials_provider('<your aws region>', '<your_credentials_provider')
return oauth2_token
Running Tests
~~~~~~~~~~~~~
You can run tests in all supported Python versions using ``pytest``. By default,
it will run all of the unit tests.
.. code-block:: sh
$ pytest
You can also run tests with setup.py:
.. code-block:: sh
$ python setup.py test
To fix lint issues, run the pre-commit command:
.. code-block:: sh
$ pre-commit run --all-files
To run tests with coverage information, run:
.. code-block:: sh
$ coverage run --source=aws_msk_iam_sasl_signer.MSKAuthTokenProvider -m pytest tests/test_auth_token_provider.py
$ coverage report -m
Troubleshooting
---------------
Finding out which identity is being used
----------------------------------------
You may receive an Access denied error and there may be some doubt as to which credential is being exactly used. The credential may be sourced from a role ARN, EC2 instance profile, credential profile etc.
When calling generate_auth_token(), you can set aws_debug_creds argument to True along with client side logging set to DEBUG then the signer library will print a debug log of the form:
.. code-block:: python
MSKAuthTokenProvider.generate_auth_token('<my aws region>', aws_debug_creds = True)
.. code-block:: sh
Credentials Identity: {UserId: ABCD:test124, Account: 1234567890, Arn: arn:aws:sts::1234567890:assumed-role/abc/test124}
The log line provides the IAM Account, IAM user id and the ARN of the IAM Principal corresponding to the credential being used.
Getting Help
------------
Please use these community resources for getting help. We use the GitHub issues
for tracking bugs and feature requests.
* Ask a `question <https://github.com/aws/aws-msk-iam-sasl-signer-python/discussions/new?category=q-a>`__ or open a `discussion <https://github.com/aws/aws-msk-iam-sasl-signer-python/discussions/new?category=general>`__.
* If you think you may have found a bug, please open an `issue <https://github.com/aws/aws-msk-iam-sasl-signer-python/issues/new/choose>`__.
* Open a support case with `AWS Support <http://docs.aws.amazon.com/awssupport/latest/user/getting-started.html>`__.
This repository provides a pluggable library with any Python Kafka client for SASL/OAUTHBEARER mechanism. For more information about SASL/OAUTHBEARER mechanism please go to `KIP 255 <https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75968876>`__.
Opening Issues
--------------
If you encounter a bug with the AWS MSK IAM SASL Signer for Python, we would like to hear about it.
Search the `Issues <https://github.com/aws/aws-msk-iam-sasl-signer-python/issues>`__ and see
if others are also experiencing the same issue before opening a new issue. Please
include the version of AWS MSK IAM SASL Signer for Python, Python, and OS you’re using. Please
also include reproduction case when appropriate.
The GitHub issues are intended for bug reports and feature requests. For help
and questions with using AWS MSK IAM SASL Signer for Python, please make use of the resources listed
in the Getting Help section.
Keeping the list of open issues lean will help us respond in a timely manner.
Contributing
------------
We value feedback and contributions from our community. Whether it's a bug report, new feature, correction, or additional documentation, we welcome your issues and pull requests. Please read through this `CONTRIBUTING <https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/CONTRIBUTING.rst>`__ document before submitting any issues or pull requests to ensure we have all the necessary information to effectively respond to your contribution.
More Resources
--------------
* `NOTICE <https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/NOTICE>`__
* `Changelog <https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/CHANGELOG.rst>`__
* `License <https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/LICENSE>`__
* `MSK Documentation <https://docs.aws.amazon.com/msk/latest/developerguide/getting-started.html>`__
* `Issues <https://github.com/aws/aws-msk-iam-sasl-signer-python/issues>`__
Credits
-------
This package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.
.. _Cookiecutter: https://github.com/audreyr/cookiecutter
.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage
.. _`dpkp/kafka-python`: https://github.com/dpkp/kafka-python
.. _`installation guide`: https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/docs/installation.rst
.. _`confluent-kafka-python`: https://github.com/confluentinc/confluent-kafka-python
=================
Changelog
=================
1.0.1 (2024-01-17)
------------------
* Expanding version dependency constraints
1.0.0 (2023-11-09)
------------------
* First release of AWS MSK IAM SASL Signer Python library.
Raw data
{
"_id": null,
"home_page": "https://github.com/aws/aws-msk-iam-sasl-signer-python",
"name": "aws-msk-iam-sasl-signer-python",
"maintainer": "",
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": "",
"keywords": "aws-msk-iam-sasl-signer-python",
"author": "Amazon Managed Streaming for Apache Kafka",
"author_email": "",
"download_url": "https://files.pythonhosted.org/packages/0a/69/e8db04e2b7ed7efb7eee5a6eeea9aec55a9a6a05d042a8d3d8cd7995eadd/aws-msk-iam-sasl-signer-python-1.0.1.tar.gz",
"platform": null,
"description": "====================================\naws-msk-iam-sasl-signer-python\n====================================\n|Version| |Python| |Build| |License| |SecurityScan|\n\n.. |Build| image:: https://github.com/aws/aws-msk-iam-sasl-signer-python/actions/workflows/run-tests.yml/badge.svg?branch=main\n :target: https://github.com/aws/aws-msk-iam-sasl-signer-python/actions/workflows/run-tests.yml\n :alt: Build status\n.. |Python| image:: https://img.shields.io/pypi/pyversions/aws-msk-iam-sasl-signer-python.svg?style=flat\n :target: https://pypi.python.org/pypi/aws-msk-iam-sasl-signer-python/\n :alt: Python Versions\n.. |Version| image:: http://img.shields.io/pypi/v/aws-msk-iam-sasl-signer-python.svg?style=flat\n :target: https://pypi.python.org/pypi/aws-msk-iam-sasl-signer-python/\n :alt: Package Version\n.. |License| image:: http://img.shields.io/pypi/l/aws-msk-iam-sasl-signer-python.svg?style=flat\n :target: https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/LICENSE\n :alt: License\n.. |SecurityScan| image:: https://github.com/aws/aws-msk-iam-sasl-signer-python/actions/workflows/securityscan.yml/badge.svg?branch=main\n :target: https://github.com/aws/aws-msk-iam-sasl-signer-python/actions/workflows/securityscan.yml\n :alt: Security Scan\n\n\nThis is an Amazon MSK Library in Python. This library provides a function to generates a base 64 encoded signed url\nto enable authentication/authorization with an MSK Cluster.\nThe signed url is generated by using your IAM credentials.\n\n\n* Free software: Apache Software License 2.0\n\nFeatures\n--------\n\n* Provides a function to generate auth token using IAM credentials from the AWS default credentials chain.\n* Provides a function to generate auth token using IAM credentials from the AWS named profile.\n* Provides a function to generate auth token using assumed IAM role's credentials.\n* Provides a function to generate auth token using a CredentialProvider. The CredentialProvider should be inherited from botocore.credentials.CredentialProvider class.\n\n\nGet Started\n-----------\n\n* For installation, refer to `installation guide`_\n\n* In order to use the signer library with a Kafka client library with SASL/OAUTHBEARER mechanism, add the callback function in your code.\n\n* For example, here is the sample code to use with `dpkp/kafka-python`_ library:\n\n.. code-block:: python\n\n from kafka import KafkaProducer\n from kafka.errors import KafkaError\n import socket\n import time\n from aws_msk_iam_sasl_signer import MSKAuthTokenProvider\n\n class MSKTokenProvider():\n def token(self):\n token, _ = MSKAuthTokenProvider.generate_auth_token('<my aws region>')\n return token\n\n tp = MSKTokenProvider()\n\n producer = KafkaProducer(\n bootstrap_servers='<my bootstrap string>',\n security_protocol='SASL_SSL',\n sasl_mechanism='OAUTHBEARER',\n sasl_oauth_token_provider=tp,\n client_id=socket.gethostname(),\n )\n\n topic = \"<my-topic>\"\n while True:\n try:\n inp=input(\">\")\n producer.send(topic, inp.encode())\n producer.flush()\n print(\"Produced!\")\n except Exception:\n print(\"Failed to send message:\", e)\n\n producer.close()\n\n* Here is a sample consumer with `confluent-kafka-python`_ library :\n\n.. code-block:: python\n\n from confluent_kafka import Consumer\n import socket\n import time\n from aws_msk_iam_sasl_signer import MSKAuthTokenProvider\n\n def oauth_cb(oauth_config):\n auth_token, expiry_ms = MSKAuthTokenProvider.generate_auth_token(\"<my aws region>\")\n # Note that this library expects oauth_cb to return expiry time in seconds since epoch, while the token generator returns expiry in ms\n return auth_token, expiry_ms/1000\n\n c = Consumer({\n \"debug\": \"all\",\n 'bootstrap.servers': \"<my bootstrap string>\",\n 'client.id': socket.gethostname(),\n 'security.protocol': 'SASL_SSL',\n 'sasl.mechanisms': 'OAUTHBEARER',\n 'oauth_cb': oauth_cb,\n 'group.id': 'mygroup',\n 'auto.offset.reset': 'earliest'\n })\n\n c.subscribe(['<my-topic>'])\n\n print(\"Starting consumer!\")\n\n while True:\n msg = c.poll(5)\n\n if msg is None:\n continue\n if msg.error():\n print(\"Consumer error: {}\".format(msg.error()))\n continue\n print('Received message: {}'.format(msg.value().decode('utf-8')))\n\n c.close()\n\n* In order to use a named profile to generate token, replace the token() function with code below :\n\n.. code-block:: python\n\n class MSKTokenProvider():\n def token(self):\n oauth2_token, _ = MSKAuthTokenProvider.generate_auth_token_from_profile('<your aws region>', '<named_profile>')\n return oauth2_token\n\n* In order to use a role arn to generate token, replace the token() function with code below :\n\n.. code-block:: python\n\n class MSKTokenProvider():\n def token(self):\n oauth2_token, _ = MSKAuthTokenProvider.generate_auth_token_from_role_arn('<your aws region>', '<role_arn>')\n return oauth2_token\n\n\n* In order to use a custom credentials provider, replace the token() function with code below :\n\n.. code-block:: python\n\n class MSKTokenProvider():\n def token(self):\n oauth2_token, _ = MSKAuthTokenProvider.generate_auth_token_from_credentials_provider('<your aws region>', '<your_credentials_provider')\n return oauth2_token\n\n\nRunning Tests\n~~~~~~~~~~~~~\nYou can run tests in all supported Python versions using ``pytest``. By default,\nit will run all of the unit tests.\n\n.. code-block:: sh\n\n $ pytest\n\nYou can also run tests with setup.py:\n\n.. code-block:: sh\n\n $ python setup.py test\n\nTo fix lint issues, run the pre-commit command:\n\n.. code-block:: sh\n\n $ pre-commit run --all-files\n\nTo run tests with coverage information, run:\n\n.. code-block:: sh\n\n $ coverage run --source=aws_msk_iam_sasl_signer.MSKAuthTokenProvider -m pytest tests/test_auth_token_provider.py\n $ coverage report -m\n\n\nTroubleshooting\n---------------\nFinding out which identity is being used\n----------------------------------------\nYou may receive an Access denied error and there may be some doubt as to which credential is being exactly used. The credential may be sourced from a role ARN, EC2 instance profile, credential profile etc.\nWhen calling generate_auth_token(), you can set aws_debug_creds argument to True along with client side logging set to DEBUG then the signer library will print a debug log of the form:\n\n.. code-block:: python\n\n MSKAuthTokenProvider.generate_auth_token('<my aws region>', aws_debug_creds = True)\n\n\n.. code-block:: sh\n\n Credentials Identity: {UserId: ABCD:test124, Account: 1234567890, Arn: arn:aws:sts::1234567890:assumed-role/abc/test124}\n\n\nThe log line provides the IAM Account, IAM user id and the ARN of the IAM Principal corresponding to the credential being used.\n\nGetting Help\n------------\n\nPlease use these community resources for getting help. We use the GitHub issues\nfor tracking bugs and feature requests.\n\n* Ask a `question <https://github.com/aws/aws-msk-iam-sasl-signer-python/discussions/new?category=q-a>`__ or open a `discussion <https://github.com/aws/aws-msk-iam-sasl-signer-python/discussions/new?category=general>`__.\n* If you think you may have found a bug, please open an `issue <https://github.com/aws/aws-msk-iam-sasl-signer-python/issues/new/choose>`__.\n* Open a support case with `AWS Support <http://docs.aws.amazon.com/awssupport/latest/user/getting-started.html>`__.\n\nThis repository provides a pluggable library with any Python Kafka client for SASL/OAUTHBEARER mechanism. For more information about SASL/OAUTHBEARER mechanism please go to `KIP 255 <https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=75968876>`__.\n\nOpening Issues\n--------------\n\nIf you encounter a bug with the AWS MSK IAM SASL Signer for Python, we would like to hear about it.\nSearch the `Issues <https://github.com/aws/aws-msk-iam-sasl-signer-python/issues>`__ and see\nif others are also experiencing the same issue before opening a new issue. Please\ninclude the version of AWS MSK IAM SASL Signer for Python, Python, and OS you\u2019re using. Please\nalso include reproduction case when appropriate.\n\nThe GitHub issues are intended for bug reports and feature requests. For help\nand questions with using AWS MSK IAM SASL Signer for Python, please make use of the resources listed\nin the Getting Help section.\nKeeping the list of open issues lean will help us respond in a timely manner.\n\nContributing\n------------\n\nWe value feedback and contributions from our community. Whether it's a bug report, new feature, correction, or additional documentation, we welcome your issues and pull requests. Please read through this `CONTRIBUTING <https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/CONTRIBUTING.rst>`__ document before submitting any issues or pull requests to ensure we have all the necessary information to effectively respond to your contribution.\n\nMore Resources\n--------------\n\n* `NOTICE <https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/NOTICE>`__\n* `Changelog <https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/CHANGELOG.rst>`__\n* `License <https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/LICENSE>`__\n* `MSK Documentation <https://docs.aws.amazon.com/msk/latest/developerguide/getting-started.html>`__\n* `Issues <https://github.com/aws/aws-msk-iam-sasl-signer-python/issues>`__\n\nCredits\n-------\n\nThis package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.\n\n.. _Cookiecutter: https://github.com/audreyr/cookiecutter\n.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage\n.. _`dpkp/kafka-python`: https://github.com/dpkp/kafka-python\n.. _`installation guide`: https://github.com/aws/aws-msk-iam-sasl-signer-python/blob/main/docs/installation.rst\n.. _`confluent-kafka-python`: https://github.com/confluentinc/confluent-kafka-python\n\n\n=================\nChangelog\n=================\n\n1.0.1 (2024-01-17)\n------------------\n\n* Expanding version dependency constraints\n\n\n1.0.0 (2023-11-09)\n------------------\n\n* First release of AWS MSK IAM SASL Signer Python library.\n",
"bugtrack_url": null,
"license": "Apache Software License 2.0",
"summary": "Amazon MSK Library in Python for SASL/OAUTHBEARER Auth",
"version": "1.0.1",
"project_urls": {
"Homepage": "https://github.com/aws/aws-msk-iam-sasl-signer-python"
},
"split_keywords": [
"aws-msk-iam-sasl-signer-python"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "e160ea3f735f6d557f354b76ae272ee86cd1e0e97f9dd4e513ee8fc4d0b13ecd",
"md5": "86e7dfa6356a3ffc631f3a463210db4e",
"sha256": "9e707025abaf250b79811457069c278f4714f120cccad882249b3b2f010967e8"
},
"downloads": -1,
"filename": "aws_msk_iam_sasl_signer_python-1.0.1-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "86e7dfa6356a3ffc631f3a463210db4e",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": ">=3.8",
"size": 13131,
"upload_time": "2024-01-24T18:20:04",
"upload_time_iso_8601": "2024-01-24T18:20:04.655506Z",
"url": "https://files.pythonhosted.org/packages/e1/60/ea3f735f6d557f354b76ae272ee86cd1e0e97f9dd4e513ee8fc4d0b13ecd/aws_msk_iam_sasl_signer_python-1.0.1-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "0a69e8db04e2b7ed7efb7eee5a6eeea9aec55a9a6a05d042a8d3d8cd7995eadd",
"md5": "92219033ebe39578aad08de20dbeebdd",
"sha256": "853f69487517c9d38db638d69571fd8b9cc8d55913761accfc00950697aea975"
},
"downloads": -1,
"filename": "aws-msk-iam-sasl-signer-python-1.0.1.tar.gz",
"has_sig": false,
"md5_digest": "92219033ebe39578aad08de20dbeebdd",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 23610,
"upload_time": "2024-01-24T18:20:09",
"upload_time_iso_8601": "2024-01-24T18:20:09.824572Z",
"url": "https://files.pythonhosted.org/packages/0a/69/e8db04e2b7ed7efb7eee5a6eeea9aec55a9a6a05d042a8d3d8cd7995eadd/aws-msk-iam-sasl-signer-python-1.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-24 18:20:09",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "aws",
"github_project": "aws-msk-iam-sasl-signer-python",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"tox": true,
"lcname": "aws-msk-iam-sasl-signer-python"
}
JSON
