How To Use
----------
This a Python3 library is for Dow Jones customers consuming data from a Dow Jones Factiva Stream.
To use this we recommend 'pip installing' this by making the following addition to your requirements.txt:
.. code-block::
# To fetch latest version from PyPi
dnaStreaming
# to fetch latest version from GitHub
git+https://github.com/dowjones/dj-dna-streams-python#egg=dnaStreaming
Auth
-----------
There is currently one way to authenticate, which is by using **your user key**.
Configuring
___________
To run this code, you need to provide credentials from one of the authentication methods and your subscriptions. There are 3 ways to do this: you can set environment variables, use a configuration file or pass an argument to a constructor.
1. Set environment variables.
###################################################################
To set your service account credentials, set the `USER_KEY` environment variable:
.. code-block::
export USER_KEY="1234567890-098765432"
To set your subscription ID, simply set an environment variable named 'SUBSCRIPTION_ID' like so
.. code-block::
export SUBSCRIPTION_ID="ABC1234567889"
The code above is the command line expression for setting this environment variable on Mac OSX. Other operating systems might have a slightly different techniques for setting environment variables on the command line.
2. Using the configuration file.
###################################################################
In this codebase you will find a file named 'customer_config.json'. You are not required to use this file. If you prefer to use this option, fill the JSON object within by adding your user key and your subscription ID. Remember that this is a JSON file so follow basic JSON formatting and syntax conventions.
> The listener will search for the `customer_config.json` file inside your `$HOME` directory by default.
If you prefer using an explicit path to your configuration file, pass the absolute path to the Listener constructor like so:
.. code-block:: python
from dnaStreaming.listener import Listener
# Config. file authentication
listener = Listener(config_file=<ABSOLUTE PATH TO YOUR CONFIG. FILE>)
3. Pass in variables as function arguments.
###################################################################
You may pass your user key to the Listener constructor and your subscription ID to the listen method like so:
.. code-block:: python
from dnaStreaming.listener import Listener
# Use the user_key argument to provide your credentials
listener = Listener(user_key=<YOUR USER KEY>)
# Use the subscription_id argument to provide your subscription id to the listener
listener.listen(callback, subscription_id=<YOUR SUBSCRIPTION ID>)
# same parameter for the async variation
listener.listen_async(callback, subscription_id=<YOUR SUBSCRIPTION ID>)
Or you may use the environment variables.
Remember that passing credentials and subscription ID(s) in this way will override the environment variable and the config file settings.
.. code-block:: python
from dnaStreaming.listener import Listener
listener = Listener()
4. Listening to messages
###################################################################
You may want to listen messages synchronously like so:
.. code-block:: python
def callback(message, subscription_id):
print('Subscription ID: {}: Message: {}'.format(subscription_id, message.data.__str__()))
return True # If desired return False to stop the message flow. This will unblock the process as well.
listener.listen(callback, maximum_messages=10) # Omitting maximum_messages means you will continue to get messages as they appear. Can be a firehose. Use with caution.
# You may pass subscription ID as a parameter to the listen function
You may want to listen messages asynchronously like so:
.. code-block:: python
def callback(message, subscription_id):
print('Subscription ID: {}: Message: {}'.format(subscription_id, message.data.__str__()))
future = listener.listen_async(callback)
# After calling `listed_async` you need to keep the main thread alive.
for count in range(0, 5):
sleep(1)
# Stop receiving messages after 5 seconds
if future.running():
future.cancel()
Log Files
_________
Very minimal logging is written to the module's path 'logs/dj-dna-streaming-python.log'. To keep maintenance simple this log is overwritten every time the app starts.
Testing
_______
.. code-block::
cd dnaStreaming/test
pip install -r requirements.txt
py.test . -s
or, alternatively, to test against python2.7 and python3.5:
.. code-block::
tox
Flake8
______
If you are maintaining this library, ensure you run flake8 before you commit. At project root command line:
.. code-block::
flake8 ./dnaStreaming ./tests
Running the Demonstration Code/Development
__________________________________________
If you are enhancing this codebase (and not just using it as a library), follow these example MacOS steps:
1. Checkout the Project from Git.
###################################################################
2. Go to the Project Root.
###################################################################
3. Create a Virtual Environment.
###################################################################
.. code-block::
virtualenv venv
4. Then activate the virutal environment by executing this command:
###################################################################
.. code-block::
source ./venv/bin/activate
5. Install the Dependencies
###################################################################
.. code-block::
pip install -r requirements.txt
6. Install the App:
###################################################################
.. code-block::
python setup.py install
7. Set the Configuration Variables
###################################################################
See the config section.
8. Run the Demo Code
###################################################################
Running Non-Docker Demo:
Execute the following at the project root:
.. code-block::
python ./dnaStreaming/demo/show_stream.py -s
Or
.. code-block::
python ./dnaStreaming/demo/show_stream_async.py -s
Running Docker Demo
Execute the following at the project root:
.. code-block::
docker run -it \
-e USER_KEY=<your user KEY> \
-e SUBSCRIPTION_ID=<your subscription ID> \
dj-dna-streaming-python
Raw data
{
"_id": null,
"home_page": "https://github.com/dowjones/dj-dna-streams-python",
"name": "dnaStreaming",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "DOWJONES, FACTIVA, STREAMS",
"author": "Zachary Kagan",
"author_email": "zachary.kagan@dowjones.com",
"download_url": "https://files.pythonhosted.org/packages/1f/3e/8aac5d89614198a4d105f18628983414d70d9326a3c3f02deb730007eb45/dnastreaming-2.2.1.tar.gz",
"platform": null,
"description": "How To Use\n----------\n\nThis a Python3 library is for Dow Jones customers consuming data from a Dow Jones Factiva Stream.\n\nTo use this we recommend 'pip installing' this by making the following addition to your requirements.txt:\n\n.. code-block::\n\n # To fetch latest version from PyPi\n dnaStreaming\n\n # to fetch latest version from GitHub\n git+https://github.com/dowjones/dj-dna-streams-python#egg=dnaStreaming\n\n\nAuth\n-----------\n\nThere is currently one way to authenticate, which is by using **your user key**.\n\nConfiguring\n___________\n\nTo run this code, you need to provide credentials from one of the authentication methods and your subscriptions. There are 3 ways to do this: you can set environment variables, use a configuration file or pass an argument to a constructor.\n\n1. Set environment variables.\n###################################################################\n\nTo set your service account credentials, set the `USER_KEY` environment variable:\n\n.. code-block::\n\n export USER_KEY=\"1234567890-098765432\"\n\n\nTo set your subscription ID, simply set an environment variable named 'SUBSCRIPTION_ID' like so\n\n.. code-block::\n\n export SUBSCRIPTION_ID=\"ABC1234567889\"\n\n\nThe code above is the command line expression for setting this environment variable on Mac OSX. Other operating systems might have a slightly different techniques for setting environment variables on the command line.\n\n2. Using the configuration file.\n###################################################################\n\nIn this codebase you will find a file named 'customer_config.json'. You are not required to use this file. If you prefer to use this option, fill the JSON object within by adding your user key and your subscription ID. Remember that this is a JSON file so follow basic JSON formatting and syntax conventions.\n\n> The listener will search for the `customer_config.json` file inside your `$HOME` directory by default.\n\nIf you prefer using an explicit path to your configuration file, pass the absolute path to the Listener constructor like so:\n\n.. code-block:: python\n\n from dnaStreaming.listener import Listener\n # Config. file authentication\n listener = Listener(config_file=<ABSOLUTE PATH TO YOUR CONFIG. FILE>)\n\n\n3. Pass in variables as function arguments.\n###################################################################\n\nYou may pass your user key to the Listener constructor and your subscription ID to the listen method like so:\n\n.. code-block:: python\n\n from dnaStreaming.listener import Listener\n # Use the user_key argument to provide your credentials\n listener = Listener(user_key=<YOUR USER KEY>)\n # Use the subscription_id argument to provide your subscription id to the listener\n listener.listen(callback, subscription_id=<YOUR SUBSCRIPTION ID>)\n # same parameter for the async variation\n listener.listen_async(callback, subscription_id=<YOUR SUBSCRIPTION ID>)\n\n\nOr you may use the environment variables.\nRemember that passing credentials and subscription ID(s) in this way will override the environment variable and the config file settings.\n\n.. code-block:: python\n\n from dnaStreaming.listener import Listener\n\n listener = Listener()\n\n\n4. Listening to messages\n###################################################################\n\nYou may want to listen messages synchronously like so:\n\n.. code-block:: python\n\n def callback(message, subscription_id):\n print('Subscription ID: {}: Message: {}'.format(subscription_id, message.data.__str__()))\n return True # If desired return False to stop the message flow. This will unblock the process as well.\n\n listener.listen(callback, maximum_messages=10) # Omitting maximum_messages means you will continue to get messages as they appear. Can be a firehose. Use with caution.\n # You may pass subscription ID as a parameter to the listen function\n\n\nYou may want to listen messages asynchronously like so:\n\n.. code-block:: python\n\n def callback(message, subscription_id):\n print('Subscription ID: {}: Message: {}'.format(subscription_id, message.data.__str__()))\n\n future = listener.listen_async(callback)\n # After calling `listed_async` you need to keep the main thread alive.\n\n for count in range(0, 5):\n sleep(1)\n\n # Stop receiving messages after 5 seconds\n if future.running():\n future.cancel()\n\n\nLog Files\n_________\n\nVery minimal logging is written to the module's path 'logs/dj-dna-streaming-python.log'. To keep maintenance simple this log is overwritten every time the app starts.\n\n\nTesting\n_______\n\n.. code-block::\n\n cd dnaStreaming/test\n pip install -r requirements.txt\n py.test . -s\n\n\nor, alternatively, to test against python2.7 and python3.5:\n\n.. code-block::\n\n tox\n\n\nFlake8\n______\n\nIf you are maintaining this library, ensure you run flake8 before you commit. At project root command line:\n\n.. code-block::\n\n flake8 ./dnaStreaming ./tests\n\n\nRunning the Demonstration Code/Development\n__________________________________________\n\nIf you are enhancing this codebase (and not just using it as a library), follow these example MacOS steps:\n\n1. Checkout the Project from Git.\n###################################################################\n\n2. Go to the Project Root.\n###################################################################\n\n3. Create a Virtual Environment.\n###################################################################\n\n.. code-block::\n\n virtualenv venv\n\n\n4. Then activate the virutal environment by executing this command:\n###################################################################\n\n.. code-block::\n\n source ./venv/bin/activate\n\n\n5. Install the Dependencies\n###################################################################\n\n.. code-block::\n\n pip install -r requirements.txt\n\n\n6. Install the App:\n###################################################################\n\n.. code-block::\n\n python setup.py install\n\n\n7. Set the Configuration Variables\n###################################################################\n\nSee the config section.\n\n8. Run the Demo Code\n###################################################################\n\nRunning Non-Docker Demo:\n\nExecute the following at the project root:\n\n.. code-block::\n\n python ./dnaStreaming/demo/show_stream.py -s\n\n\nOr\n\n.. code-block::\n\n python ./dnaStreaming/demo/show_stream_async.py -s\n\n\nRunning Docker Demo\n\nExecute the following at the project root:\n\n.. code-block::\n\n docker run -it \\ \n -e USER_KEY=<your user KEY> \\\n -e SUBSCRIPTION_ID=<your subscription ID> \\\n dj-dna-streaming-python\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Dow Jones DNA Streaming Project",
"version": "2.2.1",
"project_urls": {
"Download": "https://github.com/dowjones/dj-dna-streams-python/archive/release-2.2.1.tar.gz",
"Homepage": "https://github.com/dowjones/dj-dna-streams-python"
},
"split_keywords": [
"dowjones",
" factiva",
" streams"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "6b414595f5835b9842550b58e42f9a9324a60aa93ee1f1580cc04063c0d5bea9",
"md5": "38cde264d9f84604b418e53973ee56ab",
"sha256": "42d0ec678cb49b46ea21ae0065150efcbd4c3e7d19a8b5ec80592f86f87ae41f"
},
"downloads": -1,
"filename": "dnaStreaming-2.2.1-py3-none-any.whl",
"has_sig": false,
"md5_digest": "38cde264d9f84604b418e53973ee56ab",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 15638,
"upload_time": "2024-07-10T17:02:50",
"upload_time_iso_8601": "2024-07-10T17:02:50.443279Z",
"url": "https://files.pythonhosted.org/packages/6b/41/4595f5835b9842550b58e42f9a9324a60aa93ee1f1580cc04063c0d5bea9/dnaStreaming-2.2.1-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "1f3e8aac5d89614198a4d105f18628983414d70d9326a3c3f02deb730007eb45",
"md5": "f3c83905b7f2ee1ce83727ee3544e424",
"sha256": "7264ba73d6ad5961658da84cab71ea42ed4539681d6d204da852ff44dab117f6"
},
"downloads": -1,
"filename": "dnastreaming-2.2.1.tar.gz",
"has_sig": false,
"md5_digest": "f3c83905b7f2ee1ce83727ee3544e424",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 13635,
"upload_time": "2024-07-10T17:02:53",
"upload_time_iso_8601": "2024-07-10T17:02:53.106459Z",
"url": "https://files.pythonhosted.org/packages/1f/3e/8aac5d89614198a4d105f18628983414d70d9326a3c3f02deb730007eb45/dnastreaming-2.2.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-07-10 17:02:53",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "dowjones",
"github_project": "dj-dna-streams-python",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"requirements": [
{
"name": "google-auth",
"specs": [
[
">=",
"2.11.0"
]
]
},
{
"name": "google-cloud-pubsub",
"specs": [
[
"==",
"2.13.6"
]
]
},
{
"name": "google-cloud-core",
"specs": [
[
"==",
"2.3.2"
]
]
},
{
"name": "google-api-core",
"specs": [
[
"==",
"2.10.1"
]
]
},
{
"name": "mock",
"specs": [
[
">=",
"3.0.5"
]
]
},
{
"name": "oauth2client",
"specs": [
[
">=",
"4.1.3"
]
]
},
{
"name": "requests",
"specs": [
[
">=",
"2.28.0"
]
]
},
{
"name": "setuptools",
"specs": []
},
{
"name": "tox",
"specs": []
}
],
"tox": true,
"lcname": "dnastreaming"
}
JSON
