testresources: extensions to python unittest to allow declarative use
of resources by test cases.
Copyright (C) 2005-2013 Robert Collins <robertc@robertcollins.net>
Licensed under either the Apache License, Version 2.0 or the BSD 3-clause
license at the users choice. A copy of both licenses are available in the
project source as Apache-2.0 and BSD. You may not use this file except in
compliance with one of these two licences.
Unless required by applicable law or agreed to in writing, software
distributed under these licenses is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
license you chose for the specific language governing permissions and
limitations under that license.
See the COPYING file for full details on the licensing of Testresources.
Testresources
+++++++++++++
testresources extends unittest with a clean and simple api to provide test
optimisation where expensive common resources are needed for test cases - for
example sample working trees for VCS systems, reference databases for
enterprise applications, or web servers ... let imagination run wild.
Dependencies to build/selftest
==============================
* Python 2.6+ (or 3.3+)
* docutils
* testtools (http://pypi.python.org/pypi/testtools/)
* fixtures (http://pypi.python.org/pypi/fixtures)
Dependencies to use testresources
=================================
* Python 2.6+ (or 3.3+)
For older versions of Python, testresources <= 1.0.0 supported 2.4, 2.5 and
3.2.
How testresources Works
=======================
The basic idea of testresources is:
* Tests declare the resources they need in a ``resources`` attribute.
* When the test is run, the required resource objects are allocated (either
newly constructed, or reused), and assigned to attributes of the TestCase.
testresources distinguishes a 'resource manager' (a subclass of
``TestResourceManager``) which acts as a kind of factory, and a 'resource'
which can be any kind of object returned from the manager class's
``getResource`` method.
Resources are either clean or dirty. Being clean means they have same state in
all important ways as a newly constructed instance and they can therefore be
safely reused.
At this time, testresources is incompatible with setUpClass and setUpModule -
when an OptimisingTestSuite is wrapped around a test suite using those
features, the result will be flattened for optimisation and those setup's will
not run at all.
Main Classes
============
testresources.ResourcedTestCase
-------------------------------
By extending or mixing-in this class, tests can have necessary resources
automatically allocated and disposed or recycled.
ResourceTestCase can be used as a base class for tests, and when that is done
tests will have their ``resources`` attribute automatically checked for
resources by both OptimisingTestSuite and their own setUp() and tearDown()
methods. (This allows tests to remain functional without needing this specific
TestSuite as a container). Alternatively, you can call setUpResources(self,
resources, test_result) and tearDownResources(self, resources, test_result)
from your own classes setUp and tearDown and the same behaviour will be
activated.
To declare the use of a resource, set the ``resources`` attribute to a list of
tuples of ``(attribute_name, resource_manager)``.
During setUp, for each declared requirement, the test gains an attribute
pointing to an allocated resource, which is the result of calling
``resource_manager.getResource()``. ``finishedWith`` will be called on each
resource during tearDown().
For example::
class TestLog(testresources.ResourcedTestCase):
resources = [('branch', BzrPopulatedBranch())]
def test_log(self):
show_log(self.branch, ...)
testresources.TestResourceManager
---------------------------------
A TestResourceManager is an object that tests can use to create resources. It
can be overridden to manage different types of resources. Normally test code
doesn't need to call any methods on it, as this will be arranged by the
testresources machinery.
When implementing a new ``TestResourceManager`` subclass you should consider
overriding these methods:
``make``
Must be overridden in every concrete subclass.
Returns a new instance of the resource object
(the actual resource, not the TestResourceManager). Doesn't need to worry about
reuse, which is taken care of separately. This method is only called when a
new resource is definitely needed.
``make`` is called by ``getResource``; you should not normally need to override
the latter.
``clean``
Cleans up an existing resource instance, eg by deleting a directory or
closing a network connection. By default this does nothing, which may be
appropriate for resources that are automatically garbage collected.
``_reset``
Reset a no-longer-used dirty resource to a clean state. By default this
just discards it and creates a new one, but for some resources there may be a
faster way to reset them.
``isDirty``
Check whether an existing resource is dirty. By default this just reports
whether ``TestResourceManager.dirtied`` has been called or any of the
dependency resources are dirty.
For instance::
class TemporaryDirectoryResource(TestResourceManager):
def clean(self, resource):
shutil.rmtree(resource)
def make(self):
return tempfile.mkdtemp()
def isDirty(self, resource):
# Can't detect when the directory is written to, so assume it
# can never be reused. We could list the directory, but that might
# not catch it being open as a cwd etc.
return True
The ``resources`` list on the TestResourceManager object is used to declare
dependencies. For instance, a DataBaseResource that needs a TemporaryDirectory
might be declared with a resources list::
class DataBaseResource(TestResourceManager):
resources = [("scratchdir", TemporaryDirectoryResource())]
Most importantly, two getResources to the same TestResourceManager with no
finishedWith call in the middle, will return the same object as long as it is
not dirty.
When a Test has a dependency and that dependency successfully completes but
returns None, the framework does *not* consider this an error: be sure to always
return a valid resource, or raise an error. Error handling hasn't been heavily
exercised, but any bugs in this area will be promptly dealt with.
A sample TestResourceManager can be found in the doc/ folder.
See pydoc testresources.TestResourceManager for details.
testresources.GenericResource
-----------------------------
Glue to adapt testresources to an existing resource-like class.
testresources.FixtureResource
-----------------------------
Glue to adapt testresources to the simpler fixtures.Fixture API. Long
term testresources is likely to consolidate on that simpler API as the
recommended method of writing resources.
testresources.OptimisingTestSuite
---------------------------------
This TestSuite will introspect all the test cases it holds directly and if
they declare needed resources, will run the tests in an order that attempts to
minimise the number of setup and tear downs required. It attempts to achieve
this by callling getResource() and finishedWith() around the sequence of tests
that use a specific resource.
Tests are added to an OptimisingTestSuite as normal. Any standard library
TestSuite objects will be flattened, while any custom TestSuite subclasses
will be distributed across their member tests. This means that any custom
logic in test suites should be preserved, at the price of some level of
optimisation.
Because the test suite does the optimisation, you can control the amount of
optimising that takes place by adding more or fewer tests to a single
OptimisingTestSuite. You could add everything to a single OptimisingTestSuite,
getting global optimisation or you could use several smaller
OptimisingTestSuites.
testresources.TestLoader
------------------------
This is a trivial TestLoader that creates OptimisingTestSuites by default.
unittest.TestResult
-------------------
testresources will log activity about resource creation and destruction to the
result object tests are run with. 6 extension methods are looked for:
``startCleanResource``, ``stopCleanResource``, ``startMakeResource``,
``stopMakeResource``, ``startResetResource`` and finally ``stopResetResource``.
``testresources.tests.ResultWithResourceExtensions`` is
an example of a ``TestResult`` with these methods present.
Controlling Resource Reuse
==========================
When or how do I mark the resource dirtied?
The simplest approach is to have ``TestResourceManager.make`` call ``self.dirtied``:
the resource is always immediately dirty and will never be reused without first
being reset. This is appropriate when the underlying resource is cheap to
reset or recreate, or when it's hard to detect whether it's been dirtied or to
trap operations that change it.
Alternatively, override ``TestResourceManager.isDirty`` and inspect the resource to
see if it is safe to reuse.
Finally, you can arrange for the returned resource to always call back to
``TestResourceManager.dirtied`` on the first operation that mutates it.
FAQ
===
* Can I dynamically request resources inside a test method?
Generally, no, you shouldn't do this. The idea is that the resources are
declared statically, so that testresources can "smooth" resource usage across
several tests.
But, you may be able to find some object that is statically declared and reusable
to act as the resource, which can then provide methods to generate sub-elements
of itself during a test.
* If the resource is held inside the TestResourceManager object, and the
TestResourceManager is typically constructed inline in the test case
``resources`` attribute, how can they be shared across different test
classes?
Good question.
I guess you should arrange for a single instance to be held in an appropriate
module scope, then referenced by the test classes that want to share it.
Releasing
=========
1. Add a section to NEWS (after In Development).
2. git tag -s
3. python setup.py sdist bdist_wheel upload -s
Raw data
{
"_id": null,
"home_page": "https://launchpad.net/testresources",
"name": "testresources",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": null,
"author": "Testresources developers",
"author_email": "https://launchpad.net/~testresources-developers",
"download_url": "https://files.pythonhosted.org/packages/9d/57/8e3986cd95a80dd23195f599befa023eb85d031d2d870c47124fa5ccbf06/testresources-2.0.1.tar.gz",
"platform": "UNKNOWN",
"description": "testresources: extensions to python unittest to allow declarative use\nof resources by test cases.\n\nCopyright (C) 2005-2013 Robert Collins <robertc@robertcollins.net>\n\n Licensed under either the Apache License, Version 2.0 or the BSD 3-clause\n license at the users choice. A copy of both licenses are available in the\n project source as Apache-2.0 and BSD. You may not use this file except in\n compliance with one of these two licences.\n\n Unless required by applicable law or agreed to in writing, software\n distributed under these licenses is distributed on an \"AS IS\" BASIS, WITHOUT\n WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the\n license you chose for the specific language governing permissions and\n limitations under that license.\n\n See the COPYING file for full details on the licensing of Testresources.\n\n\nTestresources\n+++++++++++++\n\ntestresources extends unittest with a clean and simple api to provide test\noptimisation where expensive common resources are needed for test cases - for\nexample sample working trees for VCS systems, reference databases for\nenterprise applications, or web servers ... let imagination run wild.\n\nDependencies to build/selftest\n==============================\n\n* Python 2.6+ (or 3.3+)\n* docutils\n* testtools (http://pypi.python.org/pypi/testtools/)\n* fixtures (http://pypi.python.org/pypi/fixtures)\n\nDependencies to use testresources\n=================================\n\n* Python 2.6+ (or 3.3+)\n\nFor older versions of Python, testresources <= 1.0.0 supported 2.4, 2.5 and\n3.2.\n\nHow testresources Works\n=======================\n\nThe basic idea of testresources is:\n\n* Tests declare the resources they need in a ``resources`` attribute.\n* When the test is run, the required resource objects are allocated (either\n newly constructed, or reused), and assigned to attributes of the TestCase.\n\ntestresources distinguishes a 'resource manager' (a subclass of\n``TestResourceManager``) which acts as a kind of factory, and a 'resource'\nwhich can be any kind of object returned from the manager class's\n``getResource`` method.\n\nResources are either clean or dirty. Being clean means they have same state in\nall important ways as a newly constructed instance and they can therefore be\nsafely reused.\n\nAt this time, testresources is incompatible with setUpClass and setUpModule -\nwhen an OptimisingTestSuite is wrapped around a test suite using those\nfeatures, the result will be flattened for optimisation and those setup's will\nnot run at all.\n\nMain Classes\n============\n\ntestresources.ResourcedTestCase\n-------------------------------\n\nBy extending or mixing-in this class, tests can have necessary resources\nautomatically allocated and disposed or recycled.\n\nResourceTestCase can be used as a base class for tests, and when that is done\ntests will have their ``resources`` attribute automatically checked for\nresources by both OptimisingTestSuite and their own setUp() and tearDown()\nmethods. (This allows tests to remain functional without needing this specific\nTestSuite as a container). Alternatively, you can call setUpResources(self,\nresources, test_result) and tearDownResources(self, resources, test_result)\nfrom your own classes setUp and tearDown and the same behaviour will be\nactivated.\n\nTo declare the use of a resource, set the ``resources`` attribute to a list of\ntuples of ``(attribute_name, resource_manager)``.\n\nDuring setUp, for each declared requirement, the test gains an attribute\npointing to an allocated resource, which is the result of calling\n``resource_manager.getResource()``. ``finishedWith`` will be called on each\nresource during tearDown().\n\nFor example::\n\n class TestLog(testresources.ResourcedTestCase):\n\n resources = [('branch', BzrPopulatedBranch())]\n\n def test_log(self):\n show_log(self.branch, ...)\n\ntestresources.TestResourceManager\n---------------------------------\n\nA TestResourceManager is an object that tests can use to create resources. It\ncan be overridden to manage different types of resources. Normally test code\ndoesn't need to call any methods on it, as this will be arranged by the\ntestresources machinery.\n\nWhen implementing a new ``TestResourceManager`` subclass you should consider\noverriding these methods:\n\n``make``\n Must be overridden in every concrete subclass.\n\n Returns a new instance of the resource object\n (the actual resource, not the TestResourceManager). Doesn't need to worry about\n reuse, which is taken care of separately. This method is only called when a\n new resource is definitely needed.\n\n ``make`` is called by ``getResource``; you should not normally need to override\n the latter.\n\n``clean``\n Cleans up an existing resource instance, eg by deleting a directory or\n closing a network connection. By default this does nothing, which may be\n appropriate for resources that are automatically garbage collected.\n\n``_reset``\n Reset a no-longer-used dirty resource to a clean state. By default this\n just discards it and creates a new one, but for some resources there may be a\n faster way to reset them.\n\n``isDirty``\n Check whether an existing resource is dirty. By default this just reports\n whether ``TestResourceManager.dirtied`` has been called or any of the\n dependency resources are dirty.\n\nFor instance::\n\n class TemporaryDirectoryResource(TestResourceManager):\n\n def clean(self, resource):\n shutil.rmtree(resource)\n\n def make(self):\n return tempfile.mkdtemp()\n\n def isDirty(self, resource):\n # Can't detect when the directory is written to, so assume it\n # can never be reused. We could list the directory, but that might\n # not catch it being open as a cwd etc.\n return True\n\nThe ``resources`` list on the TestResourceManager object is used to declare\ndependencies. For instance, a DataBaseResource that needs a TemporaryDirectory\nmight be declared with a resources list::\n\n class DataBaseResource(TestResourceManager):\n\n resources = [(\"scratchdir\", TemporaryDirectoryResource())]\n\nMost importantly, two getResources to the same TestResourceManager with no\nfinishedWith call in the middle, will return the same object as long as it is\nnot dirty.\n\nWhen a Test has a dependency and that dependency successfully completes but\nreturns None, the framework does *not* consider this an error: be sure to always\nreturn a valid resource, or raise an error. Error handling hasn't been heavily\nexercised, but any bugs in this area will be promptly dealt with.\n\nA sample TestResourceManager can be found in the doc/ folder.\n\nSee pydoc testresources.TestResourceManager for details.\n\ntestresources.GenericResource\n-----------------------------\n\nGlue to adapt testresources to an existing resource-like class.\n\ntestresources.FixtureResource\n-----------------------------\n\nGlue to adapt testresources to the simpler fixtures.Fixture API. Long\nterm testresources is likely to consolidate on that simpler API as the\nrecommended method of writing resources.\n\ntestresources.OptimisingTestSuite\n---------------------------------\n\nThis TestSuite will introspect all the test cases it holds directly and if\nthey declare needed resources, will run the tests in an order that attempts to\nminimise the number of setup and tear downs required. It attempts to achieve\nthis by callling getResource() and finishedWith() around the sequence of tests\nthat use a specific resource.\n\nTests are added to an OptimisingTestSuite as normal. Any standard library\nTestSuite objects will be flattened, while any custom TestSuite subclasses\nwill be distributed across their member tests. This means that any custom\nlogic in test suites should be preserved, at the price of some level of\noptimisation.\n\nBecause the test suite does the optimisation, you can control the amount of\noptimising that takes place by adding more or fewer tests to a single\nOptimisingTestSuite. You could add everything to a single OptimisingTestSuite,\ngetting global optimisation or you could use several smaller\nOptimisingTestSuites.\n\n\ntestresources.TestLoader\n------------------------\n\nThis is a trivial TestLoader that creates OptimisingTestSuites by default.\n\nunittest.TestResult\n-------------------\n\ntestresources will log activity about resource creation and destruction to the\nresult object tests are run with. 6 extension methods are looked for:\n``startCleanResource``, ``stopCleanResource``, ``startMakeResource``,\n``stopMakeResource``, ``startResetResource`` and finally ``stopResetResource``.\n``testresources.tests.ResultWithResourceExtensions`` is\nan example of a ``TestResult`` with these methods present.\n\nControlling Resource Reuse\n==========================\n\nWhen or how do I mark the resource dirtied?\n\nThe simplest approach is to have ``TestResourceManager.make`` call ``self.dirtied``:\nthe resource is always immediately dirty and will never be reused without first\nbeing reset. This is appropriate when the underlying resource is cheap to\nreset or recreate, or when it's hard to detect whether it's been dirtied or to\ntrap operations that change it.\n\nAlternatively, override ``TestResourceManager.isDirty`` and inspect the resource to\nsee if it is safe to reuse.\n\nFinally, you can arrange for the returned resource to always call back to\n``TestResourceManager.dirtied`` on the first operation that mutates it.\n\nFAQ\n===\n\n* Can I dynamically request resources inside a test method?\n\n Generally, no, you shouldn't do this. The idea is that the resources are\n declared statically, so that testresources can \"smooth\" resource usage across\n several tests.\n\n But, you may be able to find some object that is statically declared and reusable\n to act as the resource, which can then provide methods to generate sub-elements\n of itself during a test.\n\n* If the resource is held inside the TestResourceManager object, and the\n TestResourceManager is typically constructed inline in the test case\n ``resources`` attribute, how can they be shared across different test\n classes?\n\n Good question.\n\n I guess you should arrange for a single instance to be held in an appropriate\n module scope, then referenced by the test classes that want to share it.\n\nReleasing\n=========\n\n1. Add a section to NEWS (after In Development).\n2. git tag -s\n3. python setup.py sdist bdist_wheel upload -s",
"bugtrack_url": null,
"license": "UNKNOWN",
"summary": "Testresources, a pyunit extension for managing expensive test resources",
"version": "2.0.1",
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "454d79a9a1f71de22fbc6c6433ac135f68d005de72fbe73e2137d2e77da9252c",
"md5": "f30436791c3c6fe92a36c6c1d135d84d",
"sha256": "67a361c3a2412231963b91ab04192209aa91a1aa052f0ab87245dbea889d1282"
},
"downloads": -1,
"filename": "testresources-2.0.1-py2.py3-none-any.whl",
"has_sig": true,
"md5_digest": "f30436791c3c6fe92a36c6c1d135d84d",
"packagetype": "bdist_wheel",
"python_version": "2.7",
"requires_python": null,
"size": 36474,
"upload_time": "2016-06-17T00:46:21",
"upload_time_iso_8601": "2016-06-17T00:46:21.023881Z",
"url": "https://files.pythonhosted.org/packages/45/4d/79a9a1f71de22fbc6c6433ac135f68d005de72fbe73e2137d2e77da9252c/testresources-2.0.1-py2.py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "9d578e3986cd95a80dd23195f599befa023eb85d031d2d870c47124fa5ccbf06",
"md5": "8873ab443db5569528964f524228a2d7",
"sha256": "ee9d1982154a1e212d4e4bac6b610800bfb558e4fb853572a827bc14a96e4417"
},
"downloads": -1,
"filename": "testresources-2.0.1.tar.gz",
"has_sig": true,
"md5_digest": "8873ab443db5569528964f524228a2d7",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 41948,
"upload_time": "2016-06-17T00:46:31",
"upload_time_iso_8601": "2016-06-17T00:46:31.889917Z",
"url": "https://files.pythonhosted.org/packages/9d/57/8e3986cd95a80dd23195f599befa023eb85d031d2d870c47124fa5ccbf06/testresources-2.0.1.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2016-06-17 00:46:31",
"github": false,
"gitlab": false,
"bitbucket": false,
"lcname": "testresources"
}
JSON