How To Use
----------
This a Python3 library is for Dow Jones customers consuming data from a Dow Jones Factiva Stream.
To use this we recommend 'pip installing' this by making the following addition to your requirements.txt:
.. code-block::
# To fetch latest version from PyPi
dnaStreaming
# to fetch latest version from GitHub
git+https://github.com/dowjones/dj-dna-streams-python#egg=dnaStreaming
Auth
-----------
There is currently one way to authenticate, which is by using **your user key**.
Configuring
___________
To run this code, you need to provide credentials from one of the authentication methods and your subscriptions. There are 3 ways to do this. You can either set environment variables or you can use a configuration file.
1. Set environment variables.
###################################################################
To set your service account credentials, set:
- An environment variable named 'USER_KEY'.
To set your subscription ID, simply set an environment variable named 'SUBSCRIPTION_ID' like so
.. code-block::
export SUBSCRIPTION_ID="ABC1234567889"
The code above is the command line expression for setting this environment variable on Mac OSX. Other operating systems might have a slightly different techniques for setting environment variables on the command line.
2. Using the configuration file.
###################################################################
In this codebase you will find a file named 'customer_config.json'. You are not required to use this file. If you prefer to use this configuration file, follow these directions: Open this file and add your service account credentials. Then add your subscription IDs. Remember that this is a JSON file so follow basic JSON formatting and syntax conventions.
3. Pass in variables as function arguments.
###################################################################
You may pass your service account credentials to the Listener constructor like so:
.. code-block:: python
from dnaStreaming.listener import Listener
# User key authentication
listener = Listener(user_key=<YOUR USER KEY>)
Or you may use the environment variables.
Remember that passing credentials and subscription ID(s) in this way will override the environment variable and the config file settings.
.. code-block:: python
from dnaStreaming.listener import Listener
listener = Listener()
4. Listening to messages
###################################################################
You may want to listen messages synchronously like so:
.. code-block:: python
def callback(message, subscription_id):
print('Subscription ID: {}: Message: {}'.format(subscription_id, message.data.__str__()))
return True # If desired return False to stop the message flow. This will unblock the process as well.
listener.listen(callback, maximum_messages=10) # Omitting maximum_messages means you will continue to get messages as they appear. Can be a firehose. Use with caution.
# You may pass subscription ID as a parameter to the listen function
You may want to listen messages asynchronously like so:
.. code-block:: python
def callback(message, subscription_id):
print('Subscription ID: {}: Message: {}'.format(subscription_id, message.data.__str__()))
future = listener.listen_async(callback)
# After calling `listed_async` you need to keep the main thread alive.
for count in range(0, 5):
sleep(1)
# Stop receiving messages after 5 seconds
if future.running():
future.cancel()
Log Files
_________
Very minimal logging is written to the module's path 'logs/dj-dna-streaming-python.log'. To keep maintenance simple this log is overwritten every time the app starts.
Testing
_______
.. code-block::
cd dnaStreaming/test
pip install -r requirements.txt
py.test . -s
or, alternatively, to test against python2.7 and python3.5:
.. code-block::
tox
Flake8
______
If you are maintaining this library, ensure you run flake8 before you commit. At project root command line:
.. code-block::
flake8 ./dnaStreaming ./tests
Running the Demonstration Code/Development
__________________________________________
If you are enhancing this codebase (and not just using it as a library), follow these example MacOS steps:
1. Checkout the Project from Git.
###################################################################
2. Go to the Project Root.
###################################################################
3. Create a Virtual Environment.
###################################################################
.. code-block::
virtualenv venv
4. Then activate the virutal environment by executing this command:
###################################################################
.. code-block::
source ./venv/bin/activate
5. Install the Dependencies
###################################################################
.. code-block::
pip install -r requirements.txt
6. Install the App:
###################################################################
.. code-block::
python setup.py install
7. Set the Configuration Variables
###################################################################
See the config section.
8. Run the Demo Code
###################################################################
Running Non-Docker Demo:
Execute the following at the project root:
.. code-block::
python ./dnaStreaming/demo/show_stream.py -s
Or
.. code-block::
python ./dnaStreaming/demo/show_stream_async.py -s
If you are having `ImportError: No module named ...` run this in your terminal before running the demo:
.. code-block::
export PYTHONPATH='.'
Running Docker Demo
Execute the following at the project root:
.. code-block::
docker run -it \
-e USER_KEY=<your user KEY> \
-e SUBSCRIPTION_ID=<your subscription ID> \
dj-dna-streaming-python
Raw data
{
"_id": null,
"home_page": "https://github.com/dowjones/dj-dna-streams-python",
"name": "dnaStreaming",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "DOWJONES,FACTIVA,STREAMS",
"author": "Zachary Kagan",
"author_email": "zachary.kagan@dowjones.com",
"download_url": "https://files.pythonhosted.org/packages/60/73/ade3fb7bb9b9e911d2e0cc589dd98a0e45c65096db9c71d7402bd657b603/dnaStreaming-2.0.5.tar.gz",
"platform": null,
"description": "How To Use\n----------\n\nThis a Python3 library is for Dow Jones customers consuming data from a Dow Jones Factiva Stream.\n\nTo use this we recommend 'pip installing' this by making the following addition to your requirements.txt:\n\n.. code-block::\n\n # To fetch latest version from PyPi\n dnaStreaming\n\n # to fetch latest version from GitHub\n git+https://github.com/dowjones/dj-dna-streams-python#egg=dnaStreaming\n\n\nAuth\n-----------\n\nThere is currently one way to authenticate, which is by using **your user key**.\n\nConfiguring\n___________\n\nTo run this code, you need to provide credentials from one of the authentication methods and your subscriptions. There are 3 ways to do this. You can either set environment variables or you can use a configuration file.\n\n1. Set environment variables.\n###################################################################\n\nTo set your service account credentials, set:\n\n- An environment variable named 'USER_KEY'.\n\nTo set your subscription ID, simply set an environment variable named 'SUBSCRIPTION_ID' like so\n\n.. code-block::\n\n export SUBSCRIPTION_ID=\"ABC1234567889\"\n\n\nThe code above is the command line expression for setting this environment variable on Mac OSX. Other operating systems might have a slightly different techniques for setting environment variables on the command line.\n\n2. Using the configuration file.\n###################################################################\n\nIn this codebase you will find a file named 'customer_config.json'. You are not required to use this file. If you prefer to use this configuration file, follow these directions: Open this file and add your service account credentials. Then add your subscription IDs. Remember that this is a JSON file so follow basic JSON formatting and syntax conventions.\n\n3. Pass in variables as function arguments.\n###################################################################\n\nYou may pass your service account credentials to the Listener constructor like so:\n\n.. code-block:: python\n\n from dnaStreaming.listener import Listener\n # User key authentication\n listener = Listener(user_key=<YOUR USER KEY>)\n\n\nOr you may use the environment variables.\nRemember that passing credentials and subscription ID(s) in this way will override the environment variable and the config file settings.\n\n.. code-block:: python\n\n from dnaStreaming.listener import Listener\n\n listener = Listener()\n\n\n4. Listening to messages\n###################################################################\n\nYou may want to listen messages synchronously like so:\n\n.. code-block:: python\n\n def callback(message, subscription_id):\n print('Subscription ID: {}: Message: {}'.format(subscription_id, message.data.__str__()))\n return True # If desired return False to stop the message flow. This will unblock the process as well.\n\n listener.listen(callback, maximum_messages=10) # Omitting maximum_messages means you will continue to get messages as they appear. Can be a firehose. Use with caution.\n # You may pass subscription ID as a parameter to the listen function\n\n\nYou may want to listen messages asynchronously like so:\n\n.. code-block:: python\n\n def callback(message, subscription_id):\n print('Subscription ID: {}: Message: {}'.format(subscription_id, message.data.__str__()))\n\n future = listener.listen_async(callback)\n # After calling `listed_async` you need to keep the main thread alive.\n\n for count in range(0, 5):\n sleep(1)\n\n # Stop receiving messages after 5 seconds\n if future.running():\n future.cancel()\n\n\nLog Files\n_________\n\nVery minimal logging is written to the module's path 'logs/dj-dna-streaming-python.log'. To keep maintenance simple this log is overwritten every time the app starts.\n\n\nTesting\n_______\n\n.. code-block::\n\n cd dnaStreaming/test\n pip install -r requirements.txt\n py.test . -s\n\n\nor, alternatively, to test against python2.7 and python3.5:\n\n.. code-block::\n\n tox\n\n\nFlake8\n______\n\nIf you are maintaining this library, ensure you run flake8 before you commit. At project root command line:\n\n.. code-block::\n\n flake8 ./dnaStreaming ./tests\n\n\nRunning the Demonstration Code/Development\n__________________________________________\n\nIf you are enhancing this codebase (and not just using it as a library), follow these example MacOS steps:\n\n1. Checkout the Project from Git.\n###################################################################\n\n2. Go to the Project Root.\n###################################################################\n\n3. Create a Virtual Environment.\n###################################################################\n\n.. code-block::\n\n virtualenv venv\n\n\n4. Then activate the virutal environment by executing this command:\n###################################################################\n\n.. code-block::\n\n source ./venv/bin/activate\n\n\n5. Install the Dependencies\n###################################################################\n\n.. code-block::\n\n pip install -r requirements.txt\n\n\n6. Install the App:\n###################################################################\n\n.. code-block::\n\n python setup.py install\n\n\n7. Set the Configuration Variables\n###################################################################\n\nSee the config section.\n\n8. Run the Demo Code\n###################################################################\n\nRunning Non-Docker Demo:\n\nExecute the following at the project root:\n\n.. code-block::\n\n python ./dnaStreaming/demo/show_stream.py -s\n\n\nOr\n\n.. code-block::\n\n python ./dnaStreaming/demo/show_stream_async.py -s\n\n\nIf you are having `ImportError: No module named ...` run this in your terminal before running the demo:\n\n.. code-block::\n\n export PYTHONPATH='.'\n\n\nRunning Docker Demo\n\nExecute the following at the project root:\n\n.. code-block::\n\n docker run -it \\ \n -e USER_KEY=<your user KEY> \\\n -e SUBSCRIPTION_ID=<your subscription ID> \\\n dj-dna-streaming-python\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "Dow Jones DNA Streaming Project",
"version": "2.0.5",
"split_keywords": [
"dowjones",
"factiva",
"streams"
],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "75ea429378b25ecb27ff34924342dd71",
"sha256": "de9fc5b2f78d465e21c61295b09c39ecedade5d89c914827335116ccafec83b6"
},
"downloads": -1,
"filename": "dnaStreaming-2.0.5-py3.9.egg",
"has_sig": false,
"md5_digest": "75ea429378b25ecb27ff34924342dd71",
"packagetype": "bdist_egg",
"python_version": "2.0.5",
"requires_python": null,
"size": 27316,
"upload_time": "2022-12-01T22:27:43",
"upload_time_iso_8601": "2022-12-01T22:27:43.933333Z",
"url": "https://files.pythonhosted.org/packages/77/fc/30068ba5316a3640fdd5815ee979e2b08b4dce02b1c547b1dd4d10f16985/dnaStreaming-2.0.5-py3.9.egg",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"md5": "2b810d4ddfa96abd9f488c47250af3fb",
"sha256": "6f91540d7030adc3fd81ef24c84ad1296e63e5e6a3afd0e014ec3bb74b314420"
},
"downloads": -1,
"filename": "dnaStreaming-2.0.5.tar.gz",
"has_sig": false,
"md5_digest": "2b810d4ddfa96abd9f488c47250af3fb",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 12886,
"upload_time": "2022-12-01T22:27:45",
"upload_time_iso_8601": "2022-12-01T22:27:45.735599Z",
"url": "https://files.pythonhosted.org/packages/60/73/ade3fb7bb9b9e911d2e0cc589dd98a0e45c65096db9c71d7402bd657b603/dnaStreaming-2.0.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-12-01 22:27:45",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "dowjones",
"github_project": "dj-dna-streams-python",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [
{
"name": "google-auth",
"specs": [
[
">=",
"2.11.0"
]
]
},
{
"name": "google-cloud-pubsub",
"specs": [
[
"==",
"2.13.6"
]
]
},
{
"name": "google-cloud-core",
"specs": [
[
"==",
"2.3.2"
]
]
},
{
"name": "google-api-core",
"specs": [
[
"==",
"2.10.1"
]
]
},
{
"name": "mock",
"specs": [
[
">=",
"3.0.5"
]
]
},
{
"name": "oauth2client",
"specs": [
[
">=",
"4.1.3"
]
]
},
{
"name": "requests",
"specs": [
[
">=",
"2.28.0"
]
]
},
{
"name": "setuptools",
"specs": []
},
{
"name": "tox",
"specs": []
}
],
"tox": true,
"lcname": "dnastreaming"
}
JSON
