========
json-cfg
========
.. image:: https://img.shields.io/travis/pasztorpisti/json-cfg.svg?style=flat
:target: https://travis-ci.org/pasztorpisti/json-cfg
:alt: build
.. image:: https://img.shields.io/codacy/25854a088e89472f9fbf2bd5c1633834.svg?style=flat
:target: https://www.codacy.com/app/pasztorpisti/json-cfg
:alt: code quality
.. image:: https://landscape.io/github/pasztorpisti/json-cfg/master/landscape.svg?style=flat
:target: https://landscape.io/github/pasztorpisti/json-cfg/master
:alt: code health
.. image:: https://img.shields.io/coveralls/pasztorpisti/json-cfg/master.svg?style=flat
:target: https://coveralls.io/r/pasztorpisti/json-cfg?branch=master
:alt: coverage
.. image:: https://img.shields.io/pypi/v/json-cfg.svg?style=flat
:target: https://pypi.python.org/pypi/json-cfg
:alt: pypi
.. image:: https://img.shields.io/github/tag/pasztorpisti/json-cfg.svg?style=flat
:target: https://github.com/pasztorpisti/json-cfg
:alt: github
.. image:: https://img.shields.io/github/license/pasztorpisti/json-cfg.svg?style=flat
:target: https://github.com/pasztorpisti/json-cfg/blob/master/LICENSE.txt
:alt: license: MIT
.. contents::
------------
Introduction
------------
The goal of this library is providing a json config file loader that has
the following extras compared to the standard ``json.load()``:
- A larger subset of javascript (and not some weird/exotic extension to json that
would turn it into something that has nothing to do with json/javascript):
- backward compatible with json so you can still load standard json files too
- single and multi-line comments - this is more useful then you would think:
it is good not only for documentation but also for temporarily disabling
a block in your config without actually deleting entries
- object (dictionary) keys without quotes: less quotation marks, less noise
- trailing commas (allowing a comma after the last item of objects and arrays)
- Providing line number information for each element of the loaded config file
and using this to display useful error messages that help locating errors not
only while parsing the file but also when processing/interpreting it.
- A simple config query syntax that handles default values, required elements and
automatically raises an exception in case of error (with useful info including
the location of the error in the config file).
Config file examples
--------------------
A traditional json config file:
.. code-block:: javascript
{
"servers": [
{
"ip_address": "127.0.0.1",
"port": 8080
},
{
"ip_address": "127.0.0.1",
"port": 8081
}
],
"superuser_name": "tron"
}
Something similar but better with json-cfg:
.. code-block:: javascript
{
// Note that we can get rid of most quotation marks.
servers: [
{
ip_address: "127.0.0.1",
port: 8080
},
// We have commented out the block of the second server below.
// Trailing commas are allowed so the comma after the
// first block (above) doesn't cause any problems.
/*
{
ip_address: "127.0.0.1",
port: 8081
}, // <-- optional trailing comma
/**/
],
superuser_name: "tron", // <-- optional trailing comma
}
Note that json-cfg can load both config files because standard json is a subset of the extended
syntax allowed by json-cfg.
.. tip::
Use javascript syntax highlight in your text editor for json config files
whenever possible - this makes reading config files much easier especially
when you have a lot of comments or large commented config blocks.
-----
Usage
-----
Installation
------------
.. code-block:: sh
pip install json-cfg
Alternatively you can download the zipped library from https://pypi.python.org/pypi/json-cfg
Quick-starter
-------------
The json-cfg library provides two modes when it comes to loading config files: One that is very
similar to the standard ``json.loads()`` and another one that returns the json wrapped into special
config nodes that make handling the config file much easier:
- ``jsoncfg.load()`` and ``jsoncfg.loads()`` are very similar to the standard ``json.loads()``.
These functions allow you to load config files into bare python representation of the json
data (dictionaries, lists, numbers, etc...).
- ``jsoncfg.load_config()`` and ``jsoncfg.loads_config()`` load the json data into special wrapper
objects that help you to query the config with much nicer syntax. At the same time if you
are looking for a value that doesn't exist in the config then these problems are handled with
exceptions that contain line/column number info about the location of the error.
One of the biggest problems with loading the config into bare python objects with a simple json
library is that the loaded json data doesn't contain the line/column numbers for the loaded json
nodes/elements. This means that by using a simple json library you can report the location of errors
with config file line/column numbers only in case of json syntax errors (in best case).
By loading the json nodes/elements into our wrapper objects we can retain the line/column numbers
for the json nodes/elements and we can use them in our error messages in case of semantic errors.
I assume that you have already installed json-cfg and you have the previously shown server config
example in a ``server.cfg`` file in the current directory.
This is how to load and process the above server configuration with a simple json library:
.. code-block:: python
import json
with open('server.cfg') as f:
config = json.load(f)
for server in config['servers']:
listen_on_interface(server['ip_address'], server.get('port', 8000))
superuser_name = config['superuser_name']
The same with json-cfg:
.. code-block:: python
import jsoncfg
config = jsoncfg.load_config('server.cfg')
for server in config.servers:
listen_on_interface(server.ip_address(), server.port(8000))
superuser_name = config.superuser_name()
Seemingly the difference isn't that big. With json-cfg you can use extended syntax in the config
file and the code that loads/processes the config is also somewhat nicer but real difference is
what happens when you encounter an error. With json-cfg you get an exception with a message that
points to the problematic part of the json config file while the pure-json example can't tell you
line/column numbers in the config file. In case of larger configs this can cause headaches.
Open your ``server.cfg`` file and remove the required ``ip_address`` attribute from one of the server
config blocks. This will cause an error when we try to load the config file with the above code
examples. The above code snippets report the following error messages in this scenario:
json:
.. code-block::
KeyError: 'ip_address'
json-cfg:
.. code-block::
jsoncfg.config_classes.JSONConfigValueNotFoundError: Required config node not found. Missing query path: .ip_address (relative to error location) [line=3;col=9]
Detailed explanation of the library interface
---------------------------------------------
When you load your json with ``jsoncfg.load_config()`` or ``jsoncfg.loads_config()`` the returned json
data - the hierarchy - is a tree of wrapper objects provided by this library. These wrapper objects
make it possible to store the column/line numbers for each json node/element (for error reporting)
and these wrappers allow you to query the config with the nice syntax you've seen above.
This library differentiates 3 types of json nodes/elements and each of these have their own wrapper
classes:
- json object (dictionary like stuff)
- json array (list like stuff)
- json scalar (I use "scalar" to refer any json value that isn't a container)
I use *json value* to refer to any json node/element whose type is unknown or unimportant.
The public API of the wrapper classes is very simple: they have no public methods. All they provide
is a few magic methods that you can use to read/query the loaded json data. (These magic methods
are ``__contains__``, ``__getattr__``, ``__getitem__``, ``__len__``, ``__iter__`` and ``__call__`` but don't
worry if you don't know about these magic methods as I will demonstrate the usage with simple code
examples that don't assume that you know them.)
The reason for having no public methods is simple: We allow querying json object keys with
``__getattr__`` (with the dot or member access operator like ``config.myvalue``) and we don't want any
public methods to conflict with the key values in your config file.
After loading the config you have a tree of wrapper object nodes and you have to perform these two
operations to get values from the config:
1. querying/reading/traversing the json hierarchy: the result of querying is a wrapper object
2. fetching the python value from the selected wrapper object: this can be done by calling the
queried wrapper object.
The following sections explain these two operations in detail.
Querying the json config hierarchy
""""""""""""""""""""""""""""""""""
To read and query the json hierarchy and the wrapper object nodes that build up the tree you have
to exploit the ``__contains__``, ``__getattr__``, ``__getitem__``, ``__len__``, ``__iter__`` magic methods
of the wrapper objects. We will use the previously shown server config for the following examples.
.. code-block:: python
import jsoncfg
config = jsoncfg.load_config('server.cfg')
# Using __getattr__ to get the servers key from the config json object.
# The result of this expression is a wrapper object that wraps the servers array/list.
server_array = config.servers
# The equivalent of the previous expression using __getitem__:
server_array = config['servers']
# Note that querying a non-existing key from an object doesn't raise an error. Instead
# it returns a special ValueNotFoundNode instance that you can continue using as a
# wrapper object. The error happens only if you try to fetch the value of this key
# without specifying a default value - but more on this later in the section where we
# discuss value fetching from wrapper objects.
special_value_not_found_node = config.non_existing_key
# Checking whether a key exists in a json object:
servers_exists = 'servers' in config
# Using __getitem__ to index into json array wrapper objects:
# Over-indexing the array would raise an exception with useful error message
# containing the location of the servers_array in the config file.
first_item_wrapper_object = servers_array[0]
# Getting the length of json object and json array wrappers:
num_config_key_value_pairs = len(config)
servers_array_len = len(servers_array)
# Iterating the items of a json object or array:
for key_string, value_wrapper_object in config:
pass
for value_wrapper_object in config.servers:
pass
Not all node types (object, array, scalar) support all operations. For example a scalar json value
doesn't support ``len()`` and you can not iterate it. What happens if someone puts a scalar value
into the config in place of the servers array? In that case the config loader code sooner or
later performs an array-specific operation on that scalar value (for example iteration) and this
raises an exception with a useful error message pointing the the loader code with the stack trace
and pointing to the scalar value in the config file with line/column numbers. You can find more info
about json-node-type related checks and error handling mechanisms in the following sections (value
fetching and error handling).
Fetching python values from the queried wrapper objects
"""""""""""""""""""""""""""""""""""""""""""""""""""""""
After selecting any of the wrapper object nodes from the json config hierarchy you can fetch its
wrapped value by using its ``__call__`` magic method. This works on all json node types: objects,
arrays and scalars. If you fetch a container (object or array) then this fetch is recursive: it
fetches the whole subtree whose root node is the fetched wrapper object. In most cases it is a
good practice to fetch only leaf nodes of the config. Leaving the containers (objects, arrays) in
wrappers helps getting better error messages if something goes wrong while you are processing the
config data.
.. code-block:: python
import jsoncfg
config = jsoncfg.load_config('server.cfg')
# Fetching the value of the whole json object hierarchy.
# python_hierarchy now looks like something you normally
# get as a result of a standard ``json.load()``.
python_hierarchy = config()
# Converting only the servers array into python-object format:
python_server_list = config.servers()
# Getting the ip_address of the first server.
server_0_ip_address_str = config.servers[0].ip_address()
Fetching optional config values (by specifying a default value)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The value fetcher call has some optional parameters. You can call it with an optional default value
followed by zero or more ``jsoncfg.JSONValueMapper`` instances. The default value comes in handy when
you are querying an **optional** item from a json object:
.. code-block:: python
# If "optional_value" isn't in the config then return the default value (50).
v0 = config.optional_value(50)
# This raises an exception if "required_value" isn't in the config.
v1 = config.required_value()
Using value mappers to validate and/or transform fetched values
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Whether you are using a default value or not you can specify zero or more ``jsoncfg.JSONValueMapper``
instances too in the parameter list of the fetcher function call. These instances have to be
callable, they have to have a ``__call__`` method that receives one parameter - the fetched value -
and they have to return the transformed (or untouched) value. If you specify more than one value
mapper instances then these value mappers are applied to the fetched value in left-to-right order
as you specify them in the argument list. You can use these value mapper instances not only to
transform the fetched value, but also to perform (type) checks on them. The ``jsoncfg.value_mappers``
module contains a few predefined type-checkers but you can create your own value mappers.
.. warning::
If you specify both a default value and one or more value mapper instances in your value fetcher
call then the value mappers are never applied to the default value. The value mappers are used
only when you fetch a value that exists in the config. json-cfg uses either the default value
or the list of value mapper instances but not both.
.. code-block:: python
from jsoncfg.value_mappers import RequireType
from jsoncfg.value_mappers import require_list, require_string, require_integer, require_number
# require_list is a jsoncfg.JSONValueMapper instance that checks if the fetched value is a list.
# If the "servers" key is missing form the config or its type isn't list then an exception is
# raised because we haven't specified a default value.
python_server_list = config.servers(require_list)
# If the "servers" key is missing from the config then the return value is None. If "servers"
# is in the config and it isn't a list instance then an exception is raised otherwise the
# return value is the servers list.
python_server_list = config.servers(None, require_list)
# Querying the required ip_address parameter with required string type.
ip_address = config.servers[0].ip_address(require_string)
# Querying the optional port parameter with a default value of 8000.
# If the optional port parameter is specified in the config then it has to be an integer.
ip_address = config.servers[0].port(8000, require_integer)
# An optional timeout parameter with a default value of 5. If the timeout parameter is in
# the config then it has to be a number (int, long, or float).
timeout = config.timeout(5, require_number)
# Getting a required guest_name parameter from the config. The parameter has to be either
# None (null in the json file) or a string.
guest_name = config.guest_name(RequireType(type(None), str))
Writing a custom value mapper (or validator)
````````````````````````````````````````````
- Derive your own value mapper class from ``jsoncfg.JSONValueMapper``.
- Implement the ``__call__`` method that receives one value and returns one value:
- Your ``__call__`` method can return the received value intact but it is allowed to
return a completely different transformed value.
- Your ``__call__`` implementation can perform validation. If the validation fails then
you have to raise an exception. This exception can be anything but if you don't have
a better idea then simply use the standard ``ValueError`` or ``TypeError``. This exception
is caught by the value fetcher call and re-raised as another json-cfg specific
exception that contains useful error message with the location of the error and that
exception also contains the exception you raised while validating.
Custom value mapper example code:
.. code-block:: python
import datetime
import jsoncfg
from jsoncfg import JSONValueMapper
from jsoncfg.value_mappers import require_integer
class OneOf(JSONValueMapper):
def __init__(self, *enum_members):
self.enum_members = set(enum_members)
def __call__(self, v):
if v not in self.enum_members:
raise ValueError('%r is not one of these: %r' % (v, self.enum_members))
return v
class RangeCheck(JSONValueMapper):
def __init__(self, min_, max_):
self.min = min_
self.max = max_
def __call__(self, v):
if self.min <= v < self.max:
return v
raise ValueError('%r is not in range [%r, %r)' % (v, self.min, self.max))
class ToDateTime(JSONValueMapper):
def __call__(self, v):
if not isinstance(v, str):
raise TypeError('Expected a naive iso8601 datetime string but found %r' % (v,))
return datetime.datetime.strptime(v, '%Y-%m-%dT%H:%M:%S')
config = jsoncfg.load_config('server.cfg')
# Creating a value mapper instance for reuse.
require_cool_superuser_name = OneOf('tron', 'neo')
superuser_name = config.superuser_name(None, require_cool_superuser_name)
check_http_port_range = RangeCheck(8000, 9000)
port = config.servers[0].port(8000, check_http_port_range)
# Chaining value mappers. First require_integer receives the value of the port
# attribute, checks/transforms it and the output of require_integer goes
# to the check_http_port_range value mapper. What you receive as a result of
# value fetching is the output of check_http_port_range.
port = config.servers[0].port(require_integer, check_http_port_range)
# to_datetime converts a naive iso8601 datetime string into a datetime instance.
to_datetime = ToDateTime()
superuser_birthday = config.superuser_birthday(None, to_datetime)
Error handling: exceptions
--------------------------
The base of all library exceptions is ``jsoncfg.JSONConfigException``. If the parsed json contains a
syntax error then you receive a ``jsoncfg.JSONConfigParserException`` - this exception has no
subclasses. In case of config query errors you receive a ``jsoncfg.JSONConfigQueryError`` - this
exception has several subclasses.
.. code-block::
+---------------------+
| JSONConfigException |
+---------------------+
^ ^
| |
+-------------------+-------+ |
| JSONConfigParserException | |
+---------------------------+ |
+-----+----------------+
+---------------------->| JSONConfigQueryError |<------------------------+
| +----------------------+ |
| ^ ^ |
| | | |
| +----------------------+-----+ +-----+------------------------+ |
| | JSONConfigValueMapperError | | JSONConfigValueNotFoundError | |
| +----------------------------+ +------------------------------+ |
| |
+-----+-------------------+ +----------------+-----+
| JSONConfigNodeTypeError | | JSONConfigIndexError |
+-------------------------+ +----------------------+
jsoncfg.\ **JSONConfigException**
This is the mother of all exceptions raised by the library (aside from some some ``ValueError``s
and ``TypeErrors`` that are raised in case of trivial programming mistakes). Note that this
exception is never raised directly - the library raises only exceptions that are derived from
this.
jsoncfg.\ **JSONConfigParserException**
You receive this exception if there is a syntax error in the parsed json.
- ``error_message``: The error message without the line/column number
info. The standard ``Exception.message`` field contains this very same message but with the
line/column info formatted into it as a postfix.
- ``line``, ``column``: line and column information to locate the error easily in the parsed json.
jsoncfg.\ **JSONConfigQueryError**
You receive this exception in case of errors you make while processing the parsed json. This
exception class is never instantiated directly, only its subclasses are used.
- ``config_node``: The json node/element that was processed when the error happened.
- ``line``, ``column``: line and column information to locate the error easily in the parsed json.
jsoncfg.\ **JSONConfigValueMapperError**
Raised when you query and fetch a value by specifying a value mapper but the value mapper
instance raises an exception during while fetching the value.
- ``mapper_exception``: The exception instance raised by the value mapper.
jsoncfg.\ **JSONConfigValueNotFoundError**
This is raised when you try to fetch a required (non-optional) value that doesn't exist in the
config file.
jsoncfg.\ **JSONConfigNodeTypeError**
You get this exception if you try to perform an operation on a node that is not allowed for
that node type (object, array or scalar), for example indexing into an array with a string.
jsoncfg.\ **JSONConfigIndexError**
Over-indexing a json array results in this exception.
- ``index``: The index used to over-index the array.
Utility functions
-----------------
The config wrapper objects have no public methods but in some cases you may want to extract some info from them
(for example line/column number, type of node). You can do that with utility functions that can be imported from
the ``jsoncfg`` module.
jsoncfg.\ **node_location**\ *(config_node)*
Returns the location of the specified config node in the file it was parsed from. The returned location is a
named tuple ``NodeLocation(line, column)`` containing the 1-based line and column numbers.
jsoncfg.\ **node_exists**\ *(config_node)*
The library doesn't raise an error if you query a non-existing key. It raises error only when you try to fetch
a value from it. Querying a non-existing key returns a special ``ValueNotFoundNode`` instance and this function
actually checks whether the node is something else than a ``ValueNotFoundNode`` instance. A node can be
any part of the json: an object/dict, a list, or any other json value. Before trying to fetch a value from the
queried node you can test the result of a query with ``node_exists()`` whether it is an existing or non-existing
node in order to handle missing/optional config blocks gracefully without exceptions.
.. code-block:: python
from jsoncfg import load_config, node_exists
config = load_config('my_config.cfg')
if node_exists(config.whatever1.whatever2.whatever3):
...
# OR an equivalent piece of code:
node = config.whatever1.whatever2.whatever3
if node_exists(node):
...
# This node_exists() call returns True:
exists_1 = node_exists(config.existing_key1.existing_key2.existing_key3)
# This node_exists() call returns False:
exists_2 = node_exists(config.non_existing_key1.non_existing_key2)
jsoncfg.\ **node_is_object**\ *(config_node)*
Returns ``True`` if the specified ``config_node`` is a json object/dict.
jsoncfg.\ **node_is_array**\ *(config_node)*
Returns ``True`` if the specified ``config_node`` is a json array/list.
jsoncfg.\ **node_is_scalar**\ *(config_node)*
Returns ``True`` if the specified ``config_node`` is a json value other than an object or array - if it isn't a
container.
jsoncfg.\ **ensure_exists**\ *(config_node)*
Returns the specified ``config_node`` if it is an existing node, otherwise it raises a config error (with
config file location info when possible).
jsoncfg.\ **expect_object**\ *(config_node)*
Returns the specified ``config_node`` if it is a json object/dict, otherwise it raises a config error (with
config file location info when possible).
In many cases you can just query and fetch objects using jsoncfg without doing explicit error handling and
jsoncfg provides useful error messages when an error occurs (like trying the fetch the value from a non-existing
node, trying to map a non-integer value to an integer, etc...). There is however at least one exception when
jsoncfg can't really auto-detect problems in a smart way: When you iterate over a json object or array. A json
object returns `(key, value)` pairs during iteration while an array returns simple items. If you just assume
(without actually checking) that a config node is a json object/dict and you iterate over it with auto-unpacking
the returned `(key, value)` pairs into two variables then you might get into trouble if your assumption is
incorrect and the actual config node is a json array. If it is an array then it will return simple items and
python fails to unpack it into two variables. The result is an ugly python runtime error and not a nice jsoncfg
error that says that the config node is an array and not an object/dict that your code expected. To overcome this
problem you can use this ``jsoncfg.expect_object()`` function to ensure that the node you iterate is a json
object. The same is recommended in case of json arrays: it is recommended to check them with
``jsoncfg.expect_array()`` before iteration:
.. code-block:: python
from jsoncfg import load_config, expect_object, expect_array
config = load_config('server.cfg')
for server in expect_array(config.servers):
print('------------')
for key, value in expect_object(server):
print('%s: %r' % (key, value))
jsoncfg.\ **expect_array**\ *(config_node)*
Returns the specified ``config_node`` if it is a json array/list, otherwise it raises a config error (with
config file location info when possible).
jsoncfg.\ **expect_scalar**\ *(config_node)*
Returns the specified ``config_node`` if it isn't a json object or array, otherwise it raises a config error (with
config file location info when possible).
Raw data
{
"_id": null,
"home_page": "https://github.com/pasztorpisti/json-cfg",
"name": "json-cfg",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "json config file parser configuration comment",
"author": "Istv\u00e1n P\u00e1sztor",
"author_email": "pasztorpisti@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/70/d8/34e37fb051be7c3b143bdb3cc5827cb52e60ee1014f4f18a190bb0237759/json-cfg-0.4.2.tar.gz",
"platform": "",
"description": "========\njson-cfg\n========\n\n.. image:: https://img.shields.io/travis/pasztorpisti/json-cfg.svg?style=flat\n :target: https://travis-ci.org/pasztorpisti/json-cfg\n :alt: build\n\n.. image:: https://img.shields.io/codacy/25854a088e89472f9fbf2bd5c1633834.svg?style=flat\n :target: https://www.codacy.com/app/pasztorpisti/json-cfg\n :alt: code quality\n\n.. image:: https://landscape.io/github/pasztorpisti/json-cfg/master/landscape.svg?style=flat\n :target: https://landscape.io/github/pasztorpisti/json-cfg/master\n :alt: code health\n\n.. image:: https://img.shields.io/coveralls/pasztorpisti/json-cfg/master.svg?style=flat\n :target: https://coveralls.io/r/pasztorpisti/json-cfg?branch=master\n :alt: coverage\n\n.. image:: https://img.shields.io/pypi/v/json-cfg.svg?style=flat\n :target: https://pypi.python.org/pypi/json-cfg\n :alt: pypi\n\n.. image:: https://img.shields.io/github/tag/pasztorpisti/json-cfg.svg?style=flat\n :target: https://github.com/pasztorpisti/json-cfg\n :alt: github\n\n.. image:: https://img.shields.io/github/license/pasztorpisti/json-cfg.svg?style=flat\n :target: https://github.com/pasztorpisti/json-cfg/blob/master/LICENSE.txt\n :alt: license: MIT\n\n.. contents::\n\n------------\nIntroduction\n------------\n\nThe goal of this library is providing a json config file loader that has\nthe following extras compared to the standard ``json.load()``:\n\n- A larger subset of javascript (and not some weird/exotic extension to json that\n would turn it into something that has nothing to do with json/javascript):\n\n - backward compatible with json so you can still load standard json files too\n - single and multi-line comments - this is more useful then you would think:\n it is good not only for documentation but also for temporarily disabling\n a block in your config without actually deleting entries\n - object (dictionary) keys without quotes: less quotation marks, less noise\n - trailing commas (allowing a comma after the last item of objects and arrays)\n\n- Providing line number information for each element of the loaded config file\n and using this to display useful error messages that help locating errors not\n only while parsing the file but also when processing/interpreting it.\n- A simple config query syntax that handles default values, required elements and\n automatically raises an exception in case of error (with useful info including\n the location of the error in the config file).\n\n\nConfig file examples\n--------------------\n\nA traditional json config file:\n\n.. code-block:: javascript\n\n {\n \"servers\": [\n {\n \"ip_address\": \"127.0.0.1\",\n \"port\": 8080\n },\n {\n \"ip_address\": \"127.0.0.1\",\n \"port\": 8081\n }\n ],\n \"superuser_name\": \"tron\"\n }\n\nSomething similar but better with json-cfg:\n\n.. code-block:: javascript\n\n {\n // Note that we can get rid of most quotation marks.\n servers: [\n {\n ip_address: \"127.0.0.1\",\n port: 8080\n },\n // We have commented out the block of the second server below.\n // Trailing commas are allowed so the comma after the\n // first block (above) doesn't cause any problems.\n /*\n {\n ip_address: \"127.0.0.1\",\n port: 8081\n }, // <-- optional trailing comma\n /**/\n ],\n superuser_name: \"tron\", // <-- optional trailing comma\n }\n\nNote that json-cfg can load both config files because standard json is a subset of the extended\nsyntax allowed by json-cfg.\n\n.. tip::\n\n Use javascript syntax highlight in your text editor for json config files\n whenever possible - this makes reading config files much easier especially\n when you have a lot of comments or large commented config blocks.\n\n-----\nUsage\n-----\n\nInstallation\n------------\n\n.. code-block:: sh\n\n pip install json-cfg\n\nAlternatively you can download the zipped library from https://pypi.python.org/pypi/json-cfg\n\nQuick-starter\n-------------\n\nThe json-cfg library provides two modes when it comes to loading config files: One that is very\nsimilar to the standard ``json.loads()`` and another one that returns the json wrapped into special\nconfig nodes that make handling the config file much easier:\n\n- ``jsoncfg.load()`` and ``jsoncfg.loads()`` are very similar to the standard ``json.loads()``.\n These functions allow you to load config files into bare python representation of the json\n data (dictionaries, lists, numbers, etc...).\n- ``jsoncfg.load_config()`` and ``jsoncfg.loads_config()`` load the json data into special wrapper\n objects that help you to query the config with much nicer syntax. At the same time if you\n are looking for a value that doesn't exist in the config then these problems are handled with\n exceptions that contain line/column number info about the location of the error.\n\nOne of the biggest problems with loading the config into bare python objects with a simple json\nlibrary is that the loaded json data doesn't contain the line/column numbers for the loaded json\nnodes/elements. This means that by using a simple json library you can report the location of errors\nwith config file line/column numbers only in case of json syntax errors (in best case).\nBy loading the json nodes/elements into our wrapper objects we can retain the line/column numbers\nfor the json nodes/elements and we can use them in our error messages in case of semantic errors.\n\nI assume that you have already installed json-cfg and you have the previously shown server config\nexample in a ``server.cfg`` file in the current directory.\n\nThis is how to load and process the above server configuration with a simple json library:\n\n.. code-block:: python\n\n import json\n\n with open('server.cfg') as f:\n config = json.load(f)\n for server in config['servers']:\n listen_on_interface(server['ip_address'], server.get('port', 8000))\n superuser_name = config['superuser_name']\n\nThe same with json-cfg:\n\n.. code-block:: python\n\n import jsoncfg\n\n config = jsoncfg.load_config('server.cfg')\n for server in config.servers:\n listen_on_interface(server.ip_address(), server.port(8000))\n superuser_name = config.superuser_name()\n\nSeemingly the difference isn't that big. With json-cfg you can use extended syntax in the config\nfile and the code that loads/processes the config is also somewhat nicer but real difference is\nwhat happens when you encounter an error. With json-cfg you get an exception with a message that\npoints to the problematic part of the json config file while the pure-json example can't tell you\nline/column numbers in the config file. In case of larger configs this can cause headaches.\n\nOpen your ``server.cfg`` file and remove the required ``ip_address`` attribute from one of the server\nconfig blocks. This will cause an error when we try to load the config file with the above code\nexamples. The above code snippets report the following error messages in this scenario:\n\njson:\n\n.. code-block::\n\n KeyError: 'ip_address'\n\njson-cfg:\n\n.. code-block::\n\n jsoncfg.config_classes.JSONConfigValueNotFoundError: Required config node not found. Missing query path: .ip_address (relative to error location) [line=3;col=9]\n\nDetailed explanation of the library interface\n---------------------------------------------\n\nWhen you load your json with ``jsoncfg.load_config()`` or ``jsoncfg.loads_config()`` the returned json\ndata - the hierarchy - is a tree of wrapper objects provided by this library. These wrapper objects\nmake it possible to store the column/line numbers for each json node/element (for error reporting)\nand these wrappers allow you to query the config with the nice syntax you've seen above.\n\nThis library differentiates 3 types of json nodes/elements and each of these have their own wrapper\nclasses:\n\n- json object (dictionary like stuff)\n- json array (list like stuff)\n- json scalar (I use \"scalar\" to refer any json value that isn't a container)\n\nI use *json value* to refer to any json node/element whose type is unknown or unimportant.\nThe public API of the wrapper classes is very simple: they have no public methods. All they provide\nis a few magic methods that you can use to read/query the loaded json data. (These magic methods\nare ``__contains__``, ``__getattr__``, ``__getitem__``, ``__len__``, ``__iter__`` and ``__call__`` but don't\nworry if you don't know about these magic methods as I will demonstrate the usage with simple code\nexamples that don't assume that you know them.)\nThe reason for having no public methods is simple: We allow querying json object keys with\n``__getattr__`` (with the dot or member access operator like ``config.myvalue``) and we don't want any\npublic methods to conflict with the key values in your config file.\n\nAfter loading the config you have a tree of wrapper object nodes and you have to perform these two\noperations to get values from the config:\n\n1. querying/reading/traversing the json hierarchy: the result of querying is a wrapper object\n2. fetching the python value from the selected wrapper object: this can be done by calling the\n queried wrapper object.\n\nThe following sections explain these two operations in detail.\n\nQuerying the json config hierarchy\n\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\n\nTo read and query the json hierarchy and the wrapper object nodes that build up the tree you have\nto exploit the ``__contains__``, ``__getattr__``, ``__getitem__``, ``__len__``, ``__iter__`` magic methods\nof the wrapper objects. We will use the previously shown server config for the following examples.\n\n.. code-block:: python\n\n import jsoncfg\n\n config = jsoncfg.load_config('server.cfg')\n\n # Using __getattr__ to get the servers key from the config json object.\n # The result of this expression is a wrapper object that wraps the servers array/list.\n server_array = config.servers\n\n # The equivalent of the previous expression using __getitem__:\n server_array = config['servers']\n\n # Note that querying a non-existing key from an object doesn't raise an error. Instead\n # it returns a special ValueNotFoundNode instance that you can continue using as a\n # wrapper object. The error happens only if you try to fetch the value of this key\n # without specifying a default value - but more on this later in the section where we\n # discuss value fetching from wrapper objects.\n special_value_not_found_node = config.non_existing_key\n\n # Checking whether a key exists in a json object:\n servers_exists = 'servers' in config\n\n # Using __getitem__ to index into json array wrapper objects:\n # Over-indexing the array would raise an exception with useful error message\n # containing the location of the servers_array in the config file.\n first_item_wrapper_object = servers_array[0]\n\n # Getting the length of json object and json array wrappers:\n num_config_key_value_pairs = len(config)\n servers_array_len = len(servers_array)\n\n # Iterating the items of a json object or array:\n for key_string, value_wrapper_object in config:\n pass\n for value_wrapper_object in config.servers:\n pass\n\nNot all node types (object, array, scalar) support all operations. For example a scalar json value\ndoesn't support ``len()`` and you can not iterate it. What happens if someone puts a scalar value\ninto the config in place of the servers array? In that case the config loader code sooner or\nlater performs an array-specific operation on that scalar value (for example iteration) and this\nraises an exception with a useful error message pointing the the loader code with the stack trace\nand pointing to the scalar value in the config file with line/column numbers. You can find more info\nabout json-node-type related checks and error handling mechanisms in the following sections (value\nfetching and error handling).\n\nFetching python values from the queried wrapper objects\n\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\"\n\nAfter selecting any of the wrapper object nodes from the json config hierarchy you can fetch its\nwrapped value by using its ``__call__`` magic method. This works on all json node types: objects,\narrays and scalars. If you fetch a container (object or array) then this fetch is recursive: it\nfetches the whole subtree whose root node is the fetched wrapper object. In most cases it is a\ngood practice to fetch only leaf nodes of the config. Leaving the containers (objects, arrays) in\nwrappers helps getting better error messages if something goes wrong while you are processing the\nconfig data.\n\n.. code-block:: python\n\n import jsoncfg\n\n config = jsoncfg.load_config('server.cfg')\n\n # Fetching the value of the whole json object hierarchy.\n # python_hierarchy now looks like something you normally\n # get as a result of a standard ``json.load()``.\n python_hierarchy = config()\n\n # Converting only the servers array into python-object format:\n python_server_list = config.servers()\n\n # Getting the ip_address of the first server.\n server_0_ip_address_str = config.servers[0].ip_address()\n\n\nFetching optional config values (by specifying a default value)\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nThe value fetcher call has some optional parameters. You can call it with an optional default value\nfollowed by zero or more ``jsoncfg.JSONValueMapper`` instances. The default value comes in handy when\nyou are querying an **optional** item from a json object:\n\n.. code-block:: python\n\n # If \"optional_value\" isn't in the config then return the default value (50).\n v0 = config.optional_value(50)\n\n # This raises an exception if \"required_value\" isn't in the config.\n v1 = config.required_value()\n\n\nUsing value mappers to validate and/or transform fetched values\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nWhether you are using a default value or not you can specify zero or more ``jsoncfg.JSONValueMapper``\ninstances too in the parameter list of the fetcher function call. These instances have to be\ncallable, they have to have a ``__call__`` method that receives one parameter - the fetched value -\nand they have to return the transformed (or untouched) value. If you specify more than one value\nmapper instances then these value mappers are applied to the fetched value in left-to-right order\nas you specify them in the argument list. You can use these value mapper instances not only to\ntransform the fetched value, but also to perform (type) checks on them. The ``jsoncfg.value_mappers``\nmodule contains a few predefined type-checkers but you can create your own value mappers.\n\n.. warning::\n\n If you specify both a default value and one or more value mapper instances in your value fetcher\n call then the value mappers are never applied to the default value. The value mappers are used\n only when you fetch a value that exists in the config. json-cfg uses either the default value\n or the list of value mapper instances but not both.\n\n.. code-block:: python\n\n from jsoncfg.value_mappers import RequireType\n from jsoncfg.value_mappers import require_list, require_string, require_integer, require_number\n\n # require_list is a jsoncfg.JSONValueMapper instance that checks if the fetched value is a list.\n # If the \"servers\" key is missing form the config or its type isn't list then an exception is\n # raised because we haven't specified a default value.\n python_server_list = config.servers(require_list)\n\n # If the \"servers\" key is missing from the config then the return value is None. If \"servers\"\n # is in the config and it isn't a list instance then an exception is raised otherwise the\n # return value is the servers list.\n python_server_list = config.servers(None, require_list)\n\n # Querying the required ip_address parameter with required string type.\n ip_address = config.servers[0].ip_address(require_string)\n\n # Querying the optional port parameter with a default value of 8000.\n # If the optional port parameter is specified in the config then it has to be an integer.\n ip_address = config.servers[0].port(8000, require_integer)\n\n # An optional timeout parameter with a default value of 5. If the timeout parameter is in\n # the config then it has to be a number (int, long, or float).\n timeout = config.timeout(5, require_number)\n\n # Getting a required guest_name parameter from the config. The parameter has to be either\n # None (null in the json file) or a string.\n guest_name = config.guest_name(RequireType(type(None), str))\n\n\nWriting a custom value mapper (or validator)\n````````````````````````````````````````````\n\n- Derive your own value mapper class from ``jsoncfg.JSONValueMapper``.\n- Implement the ``__call__`` method that receives one value and returns one value:\n\n - Your ``__call__`` method can return the received value intact but it is allowed to\n return a completely different transformed value.\n - Your ``__call__`` implementation can perform validation. If the validation fails then\n you have to raise an exception. This exception can be anything but if you don't have\n a better idea then simply use the standard ``ValueError`` or ``TypeError``. This exception\n is caught by the value fetcher call and re-raised as another json-cfg specific\n exception that contains useful error message with the location of the error and that\n exception also contains the exception you raised while validating.\n\nCustom value mapper example code:\n\n.. code-block:: python\n\n import datetime\n import jsoncfg\n from jsoncfg import JSONValueMapper\n from jsoncfg.value_mappers import require_integer\n\n class OneOf(JSONValueMapper):\n def __init__(self, *enum_members):\n self.enum_members = set(enum_members)\n\n def __call__(self, v):\n if v not in self.enum_members:\n raise ValueError('%r is not one of these: %r' % (v, self.enum_members))\n return v\n\n class RangeCheck(JSONValueMapper):\n def __init__(self, min_, max_):\n self.min = min_\n self.max = max_\n\n def __call__(self, v):\n if self.min <= v < self.max:\n return v\n raise ValueError('%r is not in range [%r, %r)' % (v, self.min, self.max))\n\n class ToDateTime(JSONValueMapper):\n def __call__(self, v):\n if not isinstance(v, str):\n raise TypeError('Expected a naive iso8601 datetime string but found %r' % (v,))\n return datetime.datetime.strptime(v, '%Y-%m-%dT%H:%M:%S')\n\n config = jsoncfg.load_config('server.cfg')\n\n # Creating a value mapper instance for reuse.\n require_cool_superuser_name = OneOf('tron', 'neo')\n superuser_name = config.superuser_name(None, require_cool_superuser_name)\n\n check_http_port_range = RangeCheck(8000, 9000)\n port = config.servers[0].port(8000, check_http_port_range)\n\n # Chaining value mappers. First require_integer receives the value of the port\n # attribute, checks/transforms it and the output of require_integer goes\n # to the check_http_port_range value mapper. What you receive as a result of\n # value fetching is the output of check_http_port_range.\n port = config.servers[0].port(require_integer, check_http_port_range)\n\n # to_datetime converts a naive iso8601 datetime string into a datetime instance.\n to_datetime = ToDateTime()\n superuser_birthday = config.superuser_birthday(None, to_datetime)\n\n\nError handling: exceptions\n--------------------------\n\nThe base of all library exceptions is ``jsoncfg.JSONConfigException``. If the parsed json contains a\nsyntax error then you receive a ``jsoncfg.JSONConfigParserException`` - this exception has no\nsubclasses. In case of config query errors you receive a ``jsoncfg.JSONConfigQueryError`` - this\nexception has several subclasses.\n\n.. code-block::\n\n +---------------------+\n | JSONConfigException |\n +---------------------+\n ^ ^\n | |\n +-------------------+-------+ |\n | JSONConfigParserException | |\n +---------------------------+ |\n +-----+----------------+\n +---------------------->| JSONConfigQueryError |<------------------------+\n | +----------------------+ |\n | ^ ^ |\n | | | |\n | +----------------------+-----+ +-----+------------------------+ |\n | | JSONConfigValueMapperError | | JSONConfigValueNotFoundError | |\n | +----------------------------+ +------------------------------+ |\n | |\n +-----+-------------------+ +----------------+-----+\n | JSONConfigNodeTypeError | | JSONConfigIndexError |\n +-------------------------+ +----------------------+\n\njsoncfg.\\ **JSONConfigException**\n\n This is the mother of all exceptions raised by the library (aside from some some ``ValueError``s\n and ``TypeErrors`` that are raised in case of trivial programming mistakes). Note that this\n exception is never raised directly - the library raises only exceptions that are derived from\n this.\n\njsoncfg.\\ **JSONConfigParserException**\n\n You receive this exception if there is a syntax error in the parsed json.\n\n - ``error_message``: The error message without the line/column number\n info. The standard ``Exception.message`` field contains this very same message but with the\n line/column info formatted into it as a postfix.\n - ``line``, ``column``: line and column information to locate the error easily in the parsed json.\n\njsoncfg.\\ **JSONConfigQueryError**\n\n You receive this exception in case of errors you make while processing the parsed json. This\n exception class is never instantiated directly, only its subclasses are used.\n\n - ``config_node``: The json node/element that was processed when the error happened.\n - ``line``, ``column``: line and column information to locate the error easily in the parsed json.\n\njsoncfg.\\ **JSONConfigValueMapperError**\n\n Raised when you query and fetch a value by specifying a value mapper but the value mapper\n instance raises an exception during while fetching the value.\n\n - ``mapper_exception``: The exception instance raised by the value mapper.\n\njsoncfg.\\ **JSONConfigValueNotFoundError**\n\n This is raised when you try to fetch a required (non-optional) value that doesn't exist in the\n config file.\n\njsoncfg.\\ **JSONConfigNodeTypeError**\n\n You get this exception if you try to perform an operation on a node that is not allowed for\n that node type (object, array or scalar), for example indexing into an array with a string.\n\njsoncfg.\\ **JSONConfigIndexError**\n\n Over-indexing a json array results in this exception.\n\n - ``index``: The index used to over-index the array.\n\nUtility functions\n-----------------\n\nThe config wrapper objects have no public methods but in some cases you may want to extract some info from them\n(for example line/column number, type of node). You can do that with utility functions that can be imported from\nthe ``jsoncfg`` module.\n\n\njsoncfg.\\ **node_location**\\ *(config_node)*\n\n Returns the location of the specified config node in the file it was parsed from. The returned location is a\n named tuple ``NodeLocation(line, column)`` containing the 1-based line and column numbers.\n\njsoncfg.\\ **node_exists**\\ *(config_node)*\n\n The library doesn't raise an error if you query a non-existing key. It raises error only when you try to fetch\n a value from it. Querying a non-existing key returns a special ``ValueNotFoundNode`` instance and this function\n actually checks whether the node is something else than a ``ValueNotFoundNode`` instance. A node can be\n any part of the json: an object/dict, a list, or any other json value. Before trying to fetch a value from the\n queried node you can test the result of a query with ``node_exists()`` whether it is an existing or non-existing\n node in order to handle missing/optional config blocks gracefully without exceptions.\n\n .. code-block:: python\n\n from jsoncfg import load_config, node_exists\n\n config = load_config('my_config.cfg')\n if node_exists(config.whatever1.whatever2.whatever3):\n ...\n\n # OR an equivalent piece of code:\n\n node = config.whatever1.whatever2.whatever3\n if node_exists(node):\n ...\n\n # This node_exists() call returns True:\n exists_1 = node_exists(config.existing_key1.existing_key2.existing_key3)\n\n # This node_exists() call returns False:\n exists_2 = node_exists(config.non_existing_key1.non_existing_key2)\n\n\njsoncfg.\\ **node_is_object**\\ *(config_node)*\n\n Returns ``True`` if the specified ``config_node`` is a json object/dict.\n\n\njsoncfg.\\ **node_is_array**\\ *(config_node)*\n\n Returns ``True`` if the specified ``config_node`` is a json array/list.\n\n\njsoncfg.\\ **node_is_scalar**\\ *(config_node)*\n\n Returns ``True`` if the specified ``config_node`` is a json value other than an object or array - if it isn't a\n container.\n\n\njsoncfg.\\ **ensure_exists**\\ *(config_node)*\n\n Returns the specified ``config_node`` if it is an existing node, otherwise it raises a config error (with\n config file location info when possible).\n\n\njsoncfg.\\ **expect_object**\\ *(config_node)*\n\n Returns the specified ``config_node`` if it is a json object/dict, otherwise it raises a config error (with\n config file location info when possible).\n\n In many cases you can just query and fetch objects using jsoncfg without doing explicit error handling and\n jsoncfg provides useful error messages when an error occurs (like trying the fetch the value from a non-existing\n node, trying to map a non-integer value to an integer, etc...). There is however at least one exception when\n jsoncfg can't really auto-detect problems in a smart way: When you iterate over a json object or array. A json\n object returns `(key, value)` pairs during iteration while an array returns simple items. If you just assume\n (without actually checking) that a config node is a json object/dict and you iterate over it with auto-unpacking\n the returned `(key, value)` pairs into two variables then you might get into trouble if your assumption is\n incorrect and the actual config node is a json array. If it is an array then it will return simple items and\n python fails to unpack it into two variables. The result is an ugly python runtime error and not a nice jsoncfg\n error that says that the config node is an array and not an object/dict that your code expected. To overcome this\n problem you can use this ``jsoncfg.expect_object()`` function to ensure that the node you iterate is a json\n object. The same is recommended in case of json arrays: it is recommended to check them with\n ``jsoncfg.expect_array()`` before iteration:\n\n .. code-block:: python\n\n from jsoncfg import load_config, expect_object, expect_array\n\n config = load_config('server.cfg')\n for server in expect_array(config.servers):\n print('------------')\n for key, value in expect_object(server):\n print('%s: %r' % (key, value))\n\n\njsoncfg.\\ **expect_array**\\ *(config_node)*\n\n Returns the specified ``config_node`` if it is a json array/list, otherwise it raises a config error (with\n config file location info when possible).\n\n\njsoncfg.\\ **expect_scalar**\\ *(config_node)*\n\n Returns the specified ``config_node`` if it isn't a json object or array, otherwise it raises a config error (with\n config file location info when possible).\n\n\n",
"bugtrack_url": null,
"license": "MIT",
"summary": "JSON config file parser with extended syntax (e.g.: comments), line/column numbers in error messages, etc...",
"version": "0.4.2",
"project_urls": {
"Homepage": "https://github.com/pasztorpisti/json-cfg"
},
"split_keywords": [
"json",
"config",
"file",
"parser",
"configuration",
"comment"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "b7f5ecdfc00830bcbaf7743f0237cf4f3ced5511d57257408db01aa320e09458",
"md5": "28f420ea51b97ebdffb1040a0befbfb6",
"sha256": "70c8d0a37a7133bdfa650760cc901172bffdb90cd9ac158fde70668d48b716c8"
},
"downloads": -1,
"filename": "json_cfg-0.4.2-py2.py3-none-any.whl",
"has_sig": false,
"md5_digest": "28f420ea51b97ebdffb1040a0befbfb6",
"packagetype": "bdist_wheel",
"python_version": "py2.py3",
"requires_python": null,
"size": 36430,
"upload_time": "2017-02-15T20:44:17",
"upload_time_iso_8601": "2017-02-15T20:44:17.011652Z",
"url": "https://files.pythonhosted.org/packages/b7/f5/ecdfc00830bcbaf7743f0237cf4f3ced5511d57257408db01aa320e09458/json_cfg-0.4.2-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "70d834e37fb051be7c3b143bdb3cc5827cb52e60ee1014f4f18a190bb0237759",
"md5": "1fc2673a7110253af5975b0f4557d019",
"sha256": "d3dd1ab30b16a3bb249b6eb35fcc42198f9656f33127e36a3fadb5e37f50d45b"
},
"downloads": -1,
"filename": "json-cfg-0.4.2.tar.gz",
"has_sig": false,
"md5_digest": "1fc2673a7110253af5975b0f4557d019",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 44930,
"upload_time": "2017-02-15T20:44:19",
"upload_time_iso_8601": "2017-02-15T20:44:19.420855Z",
"url": "https://files.pythonhosted.org/packages/70/d8/34e37fb051be7c3b143bdb3cc5827cb52e60ee1014f4f18a190bb0237759/json-cfg-0.4.2.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2017-02-15 20:44:19",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "pasztorpisti",
"github_project": "json-cfg",
"travis_ci": true,
"coveralls": false,
"github_actions": false,
"lcname": "json-cfg"
}
JSON
