=============
Tuxedo-Python
=============
.. image:: https://github.com/aivarsk/tuxedo-python/workflows/CI/badge.svg
:target: https://github.com/aivarsk/tuxedo-python
Python3 bindings for writing Oracle Tuxedo clients and servers. With Python2 support. With a book Modernizing Oracle Tuxedo Applications with Python:
.. image:: https://ws-na.amazon-adsystem.com/widgets/q?_encoding=UTF8&MarketPlace=US&ASIN=180107058X&ServiceVersion=20070822&ID=AsinImage&WS=1&Format=_SL160_&tag=aivarsk-20
:target: https://amzn.to/3ljktiH
Why?
----
I'm a fan of the way `tuxmodule <https://github.com/henschkowski/tuxmodule/blob/master/README.md>`_ enables you to interact with Oracle Tuxedo. Unfortunately, it's out-dated and somehow limited. So I cloned tuxmodule and started to clean up compiler warnings and work on some features I had in mind:
- A multi-threaded server
- Support nested FML32 buffers and more types
- Support newest Oracle Tuxedo features like ``tpadvertisex()`` and ``tpappthrinit()``
- Receive response even when the service returns TPFAIL (instead of exception)
But I realized that's too much of C for me, so I decided to write my own Python module for Oracle Tuxedo in C++ and `pybind11 <https://github.com/pybind/pybind11>`_ focusing on the parts I find most important first.
Supported platforms
-------------------
The code is tested on the latest versions of Ubuntu and Oracle Linux using GCC 8 compiler. It should work on other Linux distributions and other UNIXes with no or minor adjustments.
Windows runtime requirements (experimental)
-------------------------------------------
On Windows, the Visual C++ redistributable packages are a runtime requirement for this project. It can be found `here <https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads>`_.
I have successfully built the module with Python 3.7.7.
Python 3.8.3 fails to import the module `similar to <https://github.com/psycopg/psycopg2/issues/1006>`_ but I have no solution yet.
.. code:: Python
ImportError: DLL load failed while importing tuxedo: The specified module could not be found.
Alternatives to Oracle Tuxedo
-----------------------------
Tuxedo-Python can also be used with `Open Source alternative to Oracle Tuxedo called Fuxedo <https://github.com/fuxedo/fuxedo>`_. Just export ``TUXDIR`` pointing to the folder where `Fuxedo <http://fuxedo.io>`_ is installed everything should work.
All demo code provided with the module works with both Oracle Tuxedo and Fuxedo and you can avoid vendor lock-in by using Python and Tuxedo-Python module.
General
-------
``tuxedo`` module supports only ``STRING``, ``CARRAY`` and ``FML32`` buffer types at the moment.
``CARRAY`` is mapped to/from Python ``bytes`` type.
``STRING`` is mapped to/from Python ``str`` type.
``FML32`` is mapped to/from Python ``dict`` type with field names (``str``) as keys and lists (``list``) of different types (``int``, ``str``, ``float`` or ``dict``) as values. ``dict`` to ``FML32`` conversion also treats types ``int``, ``str``, ``float`` or ``dict`` as lists with a single element:
.. code:: python
{'TA_CLASS': 'Single value'}
converted to ``FML32`` and then back to ``dict`` becomes
.. code:: python
{'TA_CLASS': ['Single value']}
All XATMI functions that take buffer and length arguments in C take only buffer argument in Python.
Calling a service
-----------------
``tuxedo.tpcall()`` and ``tuxedo.tpgetrply()`` functions return a tuple with 3 elements or throw an exception when no data is received. This is the part I believe ``tuxmodule`` got wrong: a service may return a response
both when it succeeds (``TPSUCCESS``) and fails (``TPFAIL``) and often the failure response contains some important information.
- 0 or ``TPESVCFAIL``
- ``tpurcode`` (the second argument to ``tpreturn``)
- data buffer
.. code:: python
rval, rcode, data = t.tpcall('.TMIB', {'TA_CLASS': 'T_SVCGRP', 'TA_OPERATION': 'GET'})
if rval == 0:
# Service returned TPSUCCESS
else:
# rval == tuxedo.TPESVCFAIL
# Service returned TPFAIL
Writing servers
---------------
Tuxedo servers are written as Python classes. ``tpsvrinit`` method of object will be called when Tuxedo calls ``tpsvrinit(3c)`` function and it must return 0 on success or -1 on error. A common task for ``tpsvrinit`` is to advertise services the server provides by calling ``tuxedo.tpadvertise()`` with a service name. A method with the same name must exist. ``tpsvrdone``, ``tpsvrthrinit`` and ``tpsvrthrdone`` will be called when Tuxedo calls corresponding functions. All of these 4 methods are optional and ``tuxedo`` module always calls ``tpopen()`` and ``tpclose()`` functions before calling user-supplied methods.
Each service method receives a single argument with incoming buffer and service must end with either call to ``tuxedo.tpreturn()`` or ``tuxedo.tpforward()``. Unlike in C ``tuxedo.tpreturn()`` and ``tuxedo.tpforward()`` do not perform ``longjmp`` but set up arguments for those calls once service method will return. You can have a code that will execute after Python's ``tpreturn`` and it plays nicely with context managers. Following two code fragments are equivalent but I believe the first one is less error-prone.
.. code:: python
def ECHO(self, args):
return t.tpreturn(t.TPSUCCESS, 0, args)
.. code:: python
def ECHO(self, args):
t.tpreturn(t.TPSUCCESS, 0, args)
After that ``tuxedo.run()`` must be called with an instance of the class and command-line arguments to start Tuxedo server's main loop.
.. code:: python
#!/usr/bin/env python3
import sys
import tuxedo as t
class Server:
def tpsvrinit(self, args):
t.tpadvertise('ECHO')
return 0
def tpsvrthrinit(self, args):
return 0
def tpsvrthrdone(self):
pass
def tpsvrdone(self):
pass
def ECHO(self, args):
return t.tpreturn(t.TPSUCCESS, 0, args)
if __name__ == '__main__':
t.run(Server(), sys.argv)
UBBCONFIG
---------
To use Python code as Tuxedo server the file itself must be executable (``chmod +x *.py``) and it must contain shebang line with Python:
.. code:: python
#!/usr/bin/env python3
After that you can use the ``*.py`` file as server executable in ``UBBCONFIG``:
.. code::
"api.py" SRVGRP=GROUP1 SRVID=20 RQADDR="api" MIN=1 SECONDARYRQ=Y REPLYQ=Y
Writing clients
---------------
Nothing special is needed to implement Tuxedo clients, just import the module and start calling XATMI functions.
.. code:: python
#!/usr/bin/env python3
import sys
import tuxedo as t
rval, rcode, data = t.tpcall('.TMIB', {'TA_CLASS': 'T_SVCGRP', 'TA_OPERATION': 'GET'})
Using Oracle Database
---------------------
You can access Oracle database with ``cx_Oracle`` library and local transactions by just following the documentation of ``cx_Oracle``.
If you want a server written in Python to participate in the global transaction first specify a resource manager name to use (similar to ``buidserver``). ``tuxedo`` module currently supports:
- NONE default "null" resource manager
- Oracle_XA for Oracle Database
.. code:: python
t.run(Server(), sys.argv, 'Oracle_XA')
After that you should create a database connection in ``tpsvrinit`` by using ``tuxedo.xaoSvcCtx()`` function:
.. code:: python
def tpsvrinit(self, args):
self.db = cx_Oracle.connect(handle=t.xaoSvcCtx())
That is the only difference from standard ``cx_Oracle`` use case. Here is a complete example for a single-threaded server:
.. code:: python
#!/usr/bin/env python3
import sys
import tuxedo as t
import cx_Oracle
class Server:
def tpsvrinit(self, args):
t.userlog('Server startup')
self.db = cx_Oracle.connect(handle=t.xaoSvcCtx())
t.tpadvertise('DB')
return 0
def DB(self, args):
dbc = self.db.cursor()
dbc.execute('insert into pymsg(msg) values (:1)', ['Hello from python'])
return t.tpreturn(t.TPSUCCESS, 0, args)
if __name__ == '__main__':
t.run(Server(), sys.argv, 'Oracle_XA')
For a multi-threaded server new connections for each thread must be created in ``tpsvrthrinit()`` (instead of ``tpsvrinit()``) and stored in thread-local storage of ``threading.local()``.
Server must belong to a group with ``Oracle_XA`` as resource manager, something like this in ``UBBCONFIG``
.. code::
*GROUPS
GROUP2 LMID=tuxapp GRPNO=2 TMSNAME=ORACLETMS OPENINFO="Oracle_XA:Oracle_XA+Objects=true+Acc=P/scott/tiger+SqlNet=ORCL+SesTm=60+LogDir=/tmp+Threads=true"
*SERVERS
"db.py" SRVGRP=GROUP2 SRVID=2 CLOPT="-A"
tpadmcall
---------
``tpadmcall`` is made available for application administration even while application is down. It also has no service call overhead compared to calling ``.TMIB`` service. The Python function looks and behaves similary to ``tpcall`` except ``rcode`` (2nd element in result tuple) is always a constant 0.
.. code:: python
#!/usr/bin/env python3
import tuxedo as t
rval, _, data = t.tpadmcall({'TA_CLASS': 'T_DOMAIN', 'TA_OPERATION': 'GET'})
Global transactions
-------------------
Transactions can be started and committed or aborted by using ``tuxedo.tpbegin()``, ``tuxedo.tpcommit()``, ``tuxedo.tpabort()``. These functions take the same arguments as their corresponding C functions.
Buffer export and import
------------------------
FML32 identifiers
-----------------
``Fname32`` and ``Fldid32`` are available to find map from field identifier to name or the other way.
Functions to determine field number and type from identifier:
.. code:: python
assert t.Fldtype32(t.Fmkfldid32(t.FLD_STRING, 10)) == t.FLD_STRING
assert t.Fldno32(t.Fmkfldid32(t.FLD_STRING, 10)) == 10
Exceptions
----------
On errors either ``XatmiException`` or ``Fml32Exception`` are raised by the module. Exceptions contain additional attirbute ``code`` that contains the Tuxedo error code and you can compare it with defined errors like ``TPENOENT`` or ``TPESYSTEM``.
.. code:: python
try:
t.tpcall("whatever", {})
except t.XatmiException as e:
if e.code == t.TPENOENT:
print("Service does not exist")
Demo
----
``demo/`` folder has some proof-of-concept code:
- ``client.py`` Oracle Tuxedo client
- ``api.py`` HTTP+JSON server running inside Oracle Tuxedo server
- ``ecb.py`` HTTP+XML client running inside Oracle Tuxedo server
- ``mem.py`` multi-threaded in-memory cache
- ``db.py`` Access Oracle Database using cx_Oracle module within global transaction
- ``buf.py`` Demo of tpimport/tpexport and FML32 identifiers
TODO
----
- Implementing few more useful APIs
Raw data
{
"_id": null,
"home_page": "https://github.com/aivarsk/tuxedo-python",
"name": "tuxedo",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Aivars Kalvans",
"author_email": "aivars.kalvans@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/e6/84/8af193a980e0d4937ebf1a132befefc1671e8cc2cc48f01911e736d33574/tuxedo-1.0.9.tar.gz",
"platform": null,
"description": "=============\nTuxedo-Python\n=============\n\n.. image:: https://github.com/aivarsk/tuxedo-python/workflows/CI/badge.svg\n :target: https://github.com/aivarsk/tuxedo-python\n\nPython3 bindings for writing Oracle Tuxedo clients and servers. With Python2 support. With a book Modernizing Oracle Tuxedo Applications with Python:\n\n.. image:: https://ws-na.amazon-adsystem.com/widgets/q?_encoding=UTF8&MarketPlace=US&ASIN=180107058X&ServiceVersion=20070822&ID=AsinImage&WS=1&Format=_SL160_&tag=aivarsk-20\n :target: https://amzn.to/3ljktiH\n\nWhy?\n----\n\nI'm a fan of the way `tuxmodule <https://github.com/henschkowski/tuxmodule/blob/master/README.md>`_ enables you to interact with Oracle Tuxedo. Unfortunately, it's out-dated and somehow limited. So I cloned tuxmodule and started to clean up compiler warnings and work on some features I had in mind:\n\n- A multi-threaded server\n- Support nested FML32 buffers and more types\n- Support newest Oracle Tuxedo features like ``tpadvertisex()`` and ``tpappthrinit()``\n- Receive response even when the service returns TPFAIL (instead of exception)\n\nBut I realized that's too much of C for me, so I decided to write my own Python module for Oracle Tuxedo in C++ and `pybind11 <https://github.com/pybind/pybind11>`_ focusing on the parts I find most important first.\n\nSupported platforms\n-------------------\n\nThe code is tested on the latest versions of Ubuntu and Oracle Linux using GCC 8 compiler. It should work on other Linux distributions and other UNIXes with no or minor adjustments.\n\nWindows runtime requirements (experimental)\n-------------------------------------------\n\nOn Windows, the Visual C++ redistributable packages are a runtime requirement for this project. It can be found `here <https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads>`_.\n\nI have successfully built the module with Python 3.7.7.\n\nPython 3.8.3 fails to import the module `similar to <https://github.com/psycopg/psycopg2/issues/1006>`_ but I have no solution yet.\n\n.. code:: Python\n\n ImportError: DLL load failed while importing tuxedo: The specified module could not be found.\n\nAlternatives to Oracle Tuxedo\n-----------------------------\n\nTuxedo-Python can also be used with `Open Source alternative to Oracle Tuxedo called Fuxedo <https://github.com/fuxedo/fuxedo>`_. Just export ``TUXDIR`` pointing to the folder where `Fuxedo <http://fuxedo.io>`_ is installed everything should work.\n\nAll demo code provided with the module works with both Oracle Tuxedo and Fuxedo and you can avoid vendor lock-in by using Python and Tuxedo-Python module.\n\nGeneral\n-------\n\n``tuxedo`` module supports only ``STRING``, ``CARRAY`` and ``FML32`` buffer types at the moment.\n\n``CARRAY`` is mapped to/from Python ``bytes`` type.\n\n``STRING`` is mapped to/from Python ``str`` type.\n\n``FML32`` is mapped to/from Python ``dict`` type with field names (``str``) as keys and lists (``list``) of different types (``int``, ``str``, ``float`` or ``dict``) as values. ``dict`` to ``FML32`` conversion also treats types ``int``, ``str``, ``float`` or ``dict`` as lists with a single element:\n\n.. code:: python\n\n {'TA_CLASS': 'Single value'}\n\nconverted to ``FML32`` and then back to ``dict`` becomes\n\n.. code:: python\n\n {'TA_CLASS': ['Single value']}\n\n\nAll XATMI functions that take buffer and length arguments in C take only buffer argument in Python.\n\nCalling a service\n-----------------\n\n``tuxedo.tpcall()`` and ``tuxedo.tpgetrply()`` functions return a tuple with 3 elements or throw an exception when no data is received. This is the part I believe ``tuxmodule`` got wrong: a service may return a response\nboth when it succeeds (``TPSUCCESS``) and fails (``TPFAIL``) and often the failure response contains some important information.\n\n- 0 or ``TPESVCFAIL``\n- ``tpurcode`` (the second argument to ``tpreturn``)\n- data buffer\n\n.. code:: python\n\n rval, rcode, data = t.tpcall('.TMIB', {'TA_CLASS': 'T_SVCGRP', 'TA_OPERATION': 'GET'})\n if rval == 0:\n # Service returned TPSUCCESS\n else:\n # rval == tuxedo.TPESVCFAIL\n # Service returned TPFAIL \n\nWriting servers\n---------------\n\nTuxedo servers are written as Python classes. ``tpsvrinit`` method of object will be called when Tuxedo calls ``tpsvrinit(3c)`` function and it must return 0 on success or -1 on error. A common task for ``tpsvrinit`` is to advertise services the server provides by calling ``tuxedo.tpadvertise()`` with a service name. A method with the same name must exist. ``tpsvrdone``, ``tpsvrthrinit`` and ``tpsvrthrdone`` will be called when Tuxedo calls corresponding functions. All of these 4 methods are optional and ``tuxedo`` module always calls ``tpopen()`` and ``tpclose()`` functions before calling user-supplied methods.\n\nEach service method receives a single argument with incoming buffer and service must end with either call to ``tuxedo.tpreturn()`` or ``tuxedo.tpforward()``. Unlike in C ``tuxedo.tpreturn()`` and ``tuxedo.tpforward()`` do not perform ``longjmp`` but set up arguments for those calls once service method will return. You can have a code that will execute after Python's ``tpreturn`` and it plays nicely with context managers. Following two code fragments are equivalent but I believe the first one is less error-prone.\n\n.. code:: python\n\n def ECHO(self, args):\n return t.tpreturn(t.TPSUCCESS, 0, args)\n\n.. code:: python\n\n def ECHO(self, args):\n t.tpreturn(t.TPSUCCESS, 0, args)\n\n\nAfter that ``tuxedo.run()`` must be called with an instance of the class and command-line arguments to start Tuxedo server's main loop.\n\n.. code:: python\n\n #!/usr/bin/env python3\n import sys\n import tuxedo as t\n\n class Server:\n def tpsvrinit(self, args):\n t.tpadvertise('ECHO')\n return 0\n\n def tpsvrthrinit(self, args):\n return 0\n\n def tpsvrthrdone(self):\n pass\n\n def tpsvrdone(self):\n pass\n\n def ECHO(self, args):\n return t.tpreturn(t.TPSUCCESS, 0, args)\n\n if __name__ == '__main__':\n t.run(Server(), sys.argv)\n\nUBBCONFIG\n---------\n\nTo use Python code as Tuxedo server the file itself must be executable (``chmod +x *.py``) and it must contain shebang line with Python:\n\n.. code:: python\n\n #!/usr/bin/env python3\n\nAfter that you can use the ``*.py`` file as server executable in ``UBBCONFIG``:\n\n.. code::\n\n \"api.py\" SRVGRP=GROUP1 SRVID=20 RQADDR=\"api\" MIN=1 SECONDARYRQ=Y REPLYQ=Y\n\nWriting clients\n---------------\n\nNothing special is needed to implement Tuxedo clients, just import the module and start calling XATMI functions.\n\n.. code:: python\n\n #!/usr/bin/env python3\n import sys\n import tuxedo as t\n\n rval, rcode, data = t.tpcall('.TMIB', {'TA_CLASS': 'T_SVCGRP', 'TA_OPERATION': 'GET'})\n\nUsing Oracle Database\n---------------------\n\nYou can access Oracle database with ``cx_Oracle`` library and local transactions by just following the documentation of ``cx_Oracle``.\n\nIf you want a server written in Python to participate in the global transaction first specify a resource manager name to use (similar to ``buidserver``). ``tuxedo`` module currently supports:\n\n- NONE default \"null\" resource manager\n- Oracle_XA for Oracle Database\n\n.. code:: python\n\n t.run(Server(), sys.argv, 'Oracle_XA')\n\n\nAfter that you should create a database connection in ``tpsvrinit`` by using ``tuxedo.xaoSvcCtx()`` function:\n\n.. code:: python\n\n def tpsvrinit(self, args):\n self.db = cx_Oracle.connect(handle=t.xaoSvcCtx())\n\nThat is the only difference from standard ``cx_Oracle`` use case. Here is a complete example for a single-threaded server:\n\n.. code:: python\n\n #!/usr/bin/env python3\n\n import sys\n import tuxedo as t\n import cx_Oracle\n\n class Server:\n def tpsvrinit(self, args):\n t.userlog('Server startup')\n self.db = cx_Oracle.connect(handle=t.xaoSvcCtx())\n t.tpadvertise('DB')\n return 0\n\n def DB(self, args):\n dbc = self.db.cursor()\n dbc.execute('insert into pymsg(msg) values (:1)', ['Hello from python'])\n return t.tpreturn(t.TPSUCCESS, 0, args)\n\n if __name__ == '__main__':\n t.run(Server(), sys.argv, 'Oracle_XA')\n\nFor a multi-threaded server new connections for each thread must be created in ``tpsvrthrinit()`` (instead of ``tpsvrinit()``) and stored in thread-local storage of ``threading.local()``.\n\nServer must belong to a group with ``Oracle_XA`` as resource manager, something like this in ``UBBCONFIG``\n\n.. code::\n\n *GROUPS\n GROUP2 LMID=tuxapp GRPNO=2 TMSNAME=ORACLETMS OPENINFO=\"Oracle_XA:Oracle_XA+Objects=true+Acc=P/scott/tiger+SqlNet=ORCL+SesTm=60+LogDir=/tmp+Threads=true\"\n *SERVERS\n \"db.py\" SRVGRP=GROUP2 SRVID=2 CLOPT=\"-A\"\n\n\ntpadmcall\n---------\n\n``tpadmcall`` is made available for application administration even while application is down. It also has no service call overhead compared to calling ``.TMIB`` service. The Python function looks and behaves similary to ``tpcall`` except ``rcode`` (2nd element in result tuple) is always a constant 0.\n\n.. code:: python\n\n #!/usr/bin/env python3\n import tuxedo as t\n\n rval, _, data = t.tpadmcall({'TA_CLASS': 'T_DOMAIN', 'TA_OPERATION': 'GET'})\n\n\nGlobal transactions\n-------------------\n\nTransactions can be started and committed or aborted by using ``tuxedo.tpbegin()``, ``tuxedo.tpcommit()``, ``tuxedo.tpabort()``. These functions take the same arguments as their corresponding C functions.\n\nBuffer export and import\n------------------------\n\n\n\nFML32 identifiers\n-----------------\n\n``Fname32`` and ``Fldid32`` are available to find map from field identifier to name or the other way.\n\nFunctions to determine field number and type from identifier:\n\n.. code:: python\n\n assert t.Fldtype32(t.Fmkfldid32(t.FLD_STRING, 10)) == t.FLD_STRING\n assert t.Fldno32(t.Fmkfldid32(t.FLD_STRING, 10)) == 10\n\nExceptions\n----------\n\nOn errors either ``XatmiException`` or ``Fml32Exception`` are raised by the module. Exceptions contain additional attirbute ``code`` that contains the Tuxedo error code and you can compare it with defined errors like ``TPENOENT`` or ``TPESYSTEM``.\n\n.. code:: python\n\n try:\n t.tpcall(\"whatever\", {})\n except t.XatmiException as e:\n if e.code == t.TPENOENT:\n print(\"Service does not exist\")\n\n\nDemo\n----\n\n``demo/`` folder has some proof-of-concept code:\n\n- ``client.py`` Oracle Tuxedo client\n- ``api.py`` HTTP+JSON server running inside Oracle Tuxedo server\n- ``ecb.py`` HTTP+XML client running inside Oracle Tuxedo server\n- ``mem.py`` multi-threaded in-memory cache\n- ``db.py`` Access Oracle Database using cx_Oracle module within global transaction\n- ``buf.py`` Demo of tpimport/tpexport and FML32 identifiers\n\nTODO\n----\n\n- Implementing few more useful APIs",
"bugtrack_url": null,
"license": "MIT",
"summary": "Python3 bindings for writing Oracle Tuxedo clients and servers",
"version": "1.0.9",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"md5": "a688c2e87c1b3106e0f3c8253196f3a4",
"sha256": "79b5071a8077a5b861162af9c6d1e5a244b54085a951250f9f4456399d43f57e"
},
"downloads": -1,
"filename": "tuxedo-1.0.9.tar.gz",
"has_sig": false,
"md5_digest": "a688c2e87c1b3106e0f3c8253196f3a4",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 22625,
"upload_time": "2022-12-20T20:40:40",
"upload_time_iso_8601": "2022-12-20T20:40:40.830589Z",
"url": "https://files.pythonhosted.org/packages/e6/84/8af193a980e0d4937ebf1a132befefc1671e8cc2cc48f01911e736d33574/tuxedo-1.0.9.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2022-12-20 20:40:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"github_user": "aivarsk",
"github_project": "tuxedo-python",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "tuxedo"
}
JSON
