| Name | cs-seq JSON |
| Version |
20250801
JSON |
| download |
| home_page | None |
| Summary | Stuff to do with counters, sequences and iterables. |
| upload_time | 2025-08-01 00:30:28 |
| maintainer | None |
| docs_url | None |
| author | None |
| requires_python | >=3 |
| license | None |
| keywords |
python2
python3
|
| VCS |
 |
| bugtrack_url |
|
| requirements |
No requirements were recorded.
|
| Travis-CI |
No Travis.
|
| coveralls test coverage |
No coveralls.
|
Stuff to do with counters, sequences and iterables.
*Latest release 20250801*:
ClonedIterator: bugfix direct iteration, noticed by @Matiss.
Note that any function accepting an iterable
will consume some or all of the derived iterator
in the course of its function.
Short summary:
* `ClonedIterator`: A thread safe clone of some orginal iterator.
* `common_prefix_length`: Return the length of the common prefix of sequences `seqs`.
* `common_suffix_length`: Return the length of the common suffix of sequences `seqs`.
* `first`: Return the first item from an iterable; raise `IndexError` on empty iterables.
* `get0`: Return first element of an iterable, or the default.
* `greedy`: A decorator or function for greedy computation of iterables.
* `imerge`: Merge an iterable of ordered iterables in order.
* `infill`: A generator accepting an iterable of objects which yields `(obj,missing_keys)` 2-tuples indicating missing records requiring infill for each object.
* `infill_from_batches`: A batched version of `infill(objs)` accepting an iterable of batches of objects which yields `(obj,obj_key)` 2-tuples indicating missing records requiring infill for each object.
* `isordered`: Test whether an iterable is ordered. Note that the iterable is iterated, so this is a destructive test for nonsequences.
* `last`: Return the last item from an iterable; raise `IndexError` on empty iterables.
* `onetomany`: A decorator for a method of a sequence to merge the results of passing every element of the sequence to the function, expecting multiple values back.
* `onetoone`: A decorator for a method of a sequence to merge the results of passing every element of the sequence to the function, expecting a single value back.
* `Seq`: A numeric sequence implemented as a thread safe wrapper for `itertools.count()`.
* `seq`: Return a new sequential value.
* `skip_map`: A version of `map()` which will skip items where `func(item)` raises an exception in `except_types`, a tuple of exception types. If a skipped exception occurs a warning will be issued unless `quiet` is true (default `False`).
* `splitoff`: Split a sequence into (usually short) prefixes and a tail, for example to construct subdirectory trees based on a UUID.
* `StatefulIterator`: A trivial iterator which wraps another iterator to expose some tracking state.
* `tee`: A generator yielding the items from an iterable which also copies those items to a series of queues.
* `the`: Returns the first element of an iterable, but requires there to be exactly one.
* `TrackingCounter`: A wrapper for a counter which can be incremented and decremented.
* `unrepeated`: A generator yielding items from the iterable `it` with no repetitions.
Module contents:
- <a name="ClonedIterator"></a>`class ClonedIterator(collections.abc.Iterable, typing.Generic)`: A thread safe clone of some orginal iterator.
`next()` of this yields the next item from the supplied iterator.
`iter()` of this returns a generator yielding from the
historic items and then from the original iterator.
Note that this accrues all of the items from the original
iterator in memory.
*`ClonedIterator.__init__(self, it: Iterator)`*:
Initialise the clone with its original iterator.
*`ClonedIterator.__iter__(self)`*:
Iterate over the clone, returning a new iterator.
In mild violation of the iterator protocol, instead of
returning `self`` iter(self)` returns a generator yielding
the historic and then current contents of the original iterator.
*`ClonedIterator.__next__(self)`*:
Return the next item from the original iterator.
- <a name="common_prefix_length"></a>`common_prefix_length(*seqs)`: Return the length of the common prefix of sequences `seqs`.
- <a name="common_suffix_length"></a>`common_suffix_length(*seqs)`: Return the length of the common suffix of sequences `seqs`.
- <a name="first"></a>`first(iterable)`: Return the first item from an iterable; raise `IndexError` on empty iterables.
- <a name="get0"></a>`get0(iterable, default=None)`: Return first element of an iterable, or the default.
- <a name="greedy"></a>`greedy(g=None, queue_depth=0)`: A decorator or function for greedy computation of iterables.
If `g` is omitted or callable
this is a decorator for a generator function
causing it to compute greedily,
capacity limited by `queue_depth`.
If `g` is iterable
this function dispatches it in a `Thread` to compute greedily,
capacity limited by `queue_depth`.
Example with an iterable:
for packet in greedy(parse_data_stream(stream)):
... process packet ...
which does some readahead of the stream.
Example as a function decorator:
@greedy
def g(n):
for item in range(n):
yield n
This can also be used directly on an existing iterable:
for item in greedy(range(n)):
yield n
Normally a generator runs on demand.
This function dispatches a `Thread` to run the iterable
(typically a generator)
putting yielded values to a queue
and returns a new generator yielding from the queue.
The `queue_depth` parameter specifies the depth of the queue
and therefore how many values the original generator can compute
before blocking at the queue's capacity.
The default `queue_depth` is `0` which creates a `Channel`
as the queue - a zero storage buffer - which lets the generator
compute only a single value ahead of time.
A larger `queue_depth` allocates a `Queue` with that much storage
allowing the generator to compute as many as `queue_depth+1` values
ahead of time.
Here's a comparison of the behaviour:
Example without `@greedy`
where the "yield 1" step does not occur until after the "got 0":
>>> from time import sleep
>>> def g():
... for i in range(2):
... print("yield", i)
... yield i
... print("g done")
...
>>> G = g(); sleep(0.1)
>>> for i in G:
... print("got", i)
... sleep(0.1)
...
yield 0
got 0
yield 1
got 1
g done
Example with `@greedy`
where the "yield 1" step computes before the "got 0":
>>> from time import sleep
>>> @greedy
... def g():
... for i in range(2):
... print("yield", i)
... yield i
... print("g done")
...
>>> G = g(); sleep(0.1)
yield 0
>>> for i in G:
... print("got", repr(i))
... sleep(0.1)
...
yield 1
got 0
g done
got 1
Example with `@greedy(queue_depth=1)`
where the "yield 1" step computes before the "got 0":
>>> from cs.x import X
>>> from time import sleep
>>> @greedy
... def g():
... for i in range(3):
... X("Y")
... print("yield", i)
... yield i
... print("g done")
...
>>> G = g(); sleep(2)
yield 0
yield 1
>>> for i in G:
... print("got", repr(i))
... sleep(0.1)
...
yield 2
got 0
yield 3
got 1
g done
got 2
- <a name="imerge"></a>`imerge(*iters, **kw)`: Merge an iterable of ordered iterables in order.
Parameters:
* `iters`: an iterable of iterators
* `reverse`: keyword parameter: if true, yield items in reverse order.
This requires the iterables themselves to also be in
reversed order.
This function relies on the source iterables being ordered
and their elements being comparable, through slightly misordered
iterables (for example, as extracted from web server logs)
will produce only slightly misordered results, as the merging
is done on the basis of the front elements of each iterable.
- <a name="infill"></a>`infill(objs: Iterable[~_infill_T], *, obj_keys: Callable[[~_infill_T], ~_infill_K], existing_keys: Callable[[~_infill_T], ~_infill_K], all: Optional[bool] = False) -> Iterable[Tuple[~_infill_T, ~_infill_K]]`: A generator accepting an iterable of objects
which yields `(obj,missing_keys)` 2-tuples
indicating missing records requiring infill for each object.
Parameters:
* `objs`: an iterable of objects
* `obj_keys`: a callable accepting an object and returning
an iterable of the expected keys
* `existsing_keys`: a callable accepting an object and returning
an iterable of the existing keys
* `all`: optional flag, default `False`: if true then yield
`(obj,())` for objects with no missing records
Example:
for obj, missing_key in infill(objs,...):
... infill a record for missing_key ...
- <a name="infill_from_batches"></a>`infill_from_batches(objss: Iterable[Iterable[~_infill_T]], *, obj_keys: Callable[[~_infill_T], ~_infill_K], existing_keys: Callable[[~_infill_T], ~_infill_K], all: Optional[bool] = False, amend_batch: Optional[Callable[[Iterable[~_infill_T]], Iterable[~_infill_T]]] = <function <lambda> at 0x107aa2f20>)`: A batched version of `infill(objs)` accepting an iterable of
batches of objects which yields `(obj,obj_key)` 2-tuples
indicating missing records requiring infill for each object.
This is aimed at processing batches of objects where it is
more efficient to prepare each batch as a whole, such as a
Django `QuerySet` which lets the caller make single database
queries for a batch of `Model` instances.
Thus this function can be used with `cs.djutils.model_batches_qs`
for more efficient infill processing.
Parameters:
* `objss`: an iterable of iterables of objects
* `obj_keys`: a callable accepting an object and returning
an iterable of the expected keys
* `existsing_keys`: a callable accepting an object and returning
an iterable of the existing keys
* `all`: optional flag, default `False`: if true then yield
`(obj,())` for objects with no missing records
* `amend_batch`: optional callable to amend the batch of objects,
for example to amend a `QuerySet` with `.select_related()` or similar
- <a name="isordered"></a>`isordered(items, reverse=False, strict=False)`: Test whether an iterable is ordered.
Note that the iterable is iterated, so this is a destructive
test for nonsequences.
- <a name="last"></a>`last(iterable)`: Return the last item from an iterable; raise `IndexError` on empty iterables.
- <a name="onetomany"></a>`onetomany(func)`: A decorator for a method of a sequence to merge the results of
passing every element of the sequence to the function, expecting
multiple values back.
Example:
class X(list):
@onetomany
def chars(self, item):
return item
strs = X(['Abc', 'Def'])
all_chars = X.chars()
- <a name="onetoone"></a>`onetoone(func)`: A decorator for a method of a sequence to merge the results of
passing every element of the sequence to the function, expecting a
single value back.
Example:
class X(list):
@onetoone
def lower(self, item):
return item.lower()
strs = X(['Abc', 'Def'])
lower_strs = X.lower()
- <a name="Seq"></a>`class Seq`: A numeric sequence implemented as a thread safe wrapper for
`itertools.count()`.
A `Seq` is iterable and both iterating and calling it return
the next number in the sequence.
- <a name="seq"></a>`seq()`: Return a new sequential value.
- <a name="skip_map"></a>`skip_map(func, *iterables, except_types, quiet=False)`: A version of `map()` which will skip items where `func(item)`
raises an exception in `except_types`, a tuple of exception types.
If a skipped exception occurs a warning will be issued unless
`quiet` is true (default `False`).
- <a name="splitoff"></a>`splitoff(sq, *sizes)`: Split a sequence into (usually short) prefixes and a tail,
for example to construct subdirectory trees based on a UUID.
Example:
>>> from uuid import UUID
>>> uuid = 'd6d9c510-785c-468c-9aa4-b7bda343fb79'
>>> uu = UUID(uuid).hex
>>> uu
'd6d9c510785c468c9aa4b7bda343fb79'
>>> splitoff(uu, 2, 2)
['d6', 'd9', 'c510785c468c9aa4b7bda343fb79']
- <a name="StatefulIterator"></a>`class StatefulIterator`: A trivial iterator which wraps another iterator to expose some tracking state.
This has 2 attributes:
* `.it`: the internal iterator which should yield `(item,new_state)`
* `.state`: the last state value from the internal iterator
The originating use case is resuse of an iterator by independent
calls that are typically sequential, specificly the .read
method of file like objects. Naive sequential reads require
the underlying storage to locate the data on every call, even
though the previous call has just performed this task for the
previous read. Saving the iterator used from the preceeding
call allows the iterator to pick up directly if the file
offset hasn't been fiddled in the meantime.
- <a name="tee"></a>`tee(iterable, *Qs)`: A generator yielding the items from an iterable
which also copies those items to a series of queues.
Parameters:
* `iterable`: the iterable to copy
* `Qs`: the queues, objects accepting a `.put` method.
Note: the item is `.put` onto every queue
before being yielded from this generator.
- <a name="the"></a>`the(iterable, context=None)`: Returns the first element of an iterable, but requires there to be
exactly one.
- <a name="TrackingCounter"></a>`class TrackingCounter`: A wrapper for a counter which can be incremented and decremented.
A facility is provided to wait for the counter to reach a specific value.
The .inc and .dec methods also accept a `tag` argument to keep
individual counts based on the tag to aid debugging.
TODO: add `strict` option to error and abort if any counter tries
to go below zero.
*`TrackingCounter.__init__(self, value=0, name=None, lock=None)`*:
Initialise the counter to `value` (default 0) with the optional `name`.
*`TrackingCounter.check(self)`*:
Internal consistency check.
*`TrackingCounter.dec(self, tag=None)`*:
Decrement the counter.
Wake up any threads waiting for its new value.
*`TrackingCounter.inc(self, tag=None)`*:
Increment the counter.
Wake up any threads waiting for its new value.
*`TrackingCounter.wait(self, value)`*:
Wait for the counter to reach the specified `value`.
- <a name="unrepeated"></a>`unrepeated(it, seen=None, signature=None)`: A generator yielding items from the iterable `it` with no repetitions.
Parameters:
* `it`: the iterable to process
* `seen`: an optional setlike container supporting `in` and `.add()`
* `signature`: an optional signature function for items from `it`
which produces the value to compare to recognise repeated items;
its values are stored in the `seen` set
The default `signature` function is equality;
the items are stored n `seen` and compared.
This requires the items to be hashable and support equality tests.
The same applies to whatever values the `signature` function produces.
Another common signature is identity: `id`, useful for
traversing a graph which may have cycles.
Since `seen` accrues all the signature values for yielded items
generally it will grow monotonicly as iteration proceeeds.
If the items are complex or large it is well worth providing a signature
function even if the items themselves can be used in a set.
# Release Log
*Release 20250801*:
ClonedIterator: bugfix direct iteration, noticed by @Matiss.
*Release 20250724*:
New ClonedIterator class to provide a reiterable clone of an iterator.
*Release 20250306*:
New infill() and infill_from_batches() generators for identifying missing records requiring an infill.
*Release 20250103*:
New skip_map(func, *iterables, except_types, quiet=False) generator function, like map() but skipping certain exceptions.
*Release 20221118*:
Small doc improvement.
*Release 20220530*:
Seq: calling a Seq is like next(seq).
*Release 20210924*:
New greedy(iterable) or @greedy(generator_function) to let generators precompute.
*Release 20210913*:
New unrepeated() generator removing duplicates from an iterable.
*Release 20201025*:
New splitoff() function to split a sequence into (usually short) prefixes and a tail.
*Release 20200914*:
New common_prefix_length and common_suffix_length for comparing prefixes and suffixes of sequences.
*Release 20190103*:
Documentation update.
*Release 20190101*:
* New and UNTESTED class StatefulIterator to associate some externally visible state with an iterator.
* Seq: accept optional `lock` parameter.
*Release 20171231*:
* Python 2 backport for imerge().
* New tee function to duplicate an iterable to queues.
* Function isordered() is now a test instead of an assertion.
* Drop NamedTuple, NamedTupleClassFactory (unused).
*Release 20160918*:
* New function isordered() to test ordering of a sequence.
* imerge: accept new `reverse` parameter for merging reversed iterables.
*Release 20160828*:
Modify DISTINFO to say "install_requires", fixes pypi requirements.
*Release 20160827*:
TrackingCounter: accept presupplied lock object. Python 3 exec fix.
*Release 20150118*:
metadata update
*Release 20150111*:
Initial PyPI release.
Raw data
{
"_id": null,
"home_page": null,
"name": "cs-seq",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3",
"maintainer_email": null,
"keywords": "python2, python3",
"author": null,
"author_email": "Cameron Simpson <cs@cskk.id.au>",
"download_url": "https://files.pythonhosted.org/packages/f7/e0/a8acf18eae6f03d386099fdaa3a1de943c70ca7e91bc819281bf5b268078/cs_seq-20250801.tar.gz",
"platform": null,
"description": "Stuff to do with counters, sequences and iterables.\n\n*Latest release 20250801*:\nClonedIterator: bugfix direct iteration, noticed by @Matiss.\n\nNote that any function accepting an iterable\nwill consume some or all of the derived iterator\nin the course of its function.\n\nShort summary:\n* `ClonedIterator`: A thread safe clone of some orginal iterator.\n* `common_prefix_length`: Return the length of the common prefix of sequences `seqs`.\n* `common_suffix_length`: Return the length of the common suffix of sequences `seqs`.\n* `first`: Return the first item from an iterable; raise `IndexError` on empty iterables.\n* `get0`: Return first element of an iterable, or the default.\n* `greedy`: A decorator or function for greedy computation of iterables.\n* `imerge`: Merge an iterable of ordered iterables in order.\n* `infill`: A generator accepting an iterable of objects which yields `(obj,missing_keys)` 2-tuples indicating missing records requiring infill for each object.\n* `infill_from_batches`: A batched version of `infill(objs)` accepting an iterable of batches of objects which yields `(obj,obj_key)` 2-tuples indicating missing records requiring infill for each object.\n* `isordered`: Test whether an iterable is ordered. Note that the iterable is iterated, so this is a destructive test for nonsequences.\n* `last`: Return the last item from an iterable; raise `IndexError` on empty iterables.\n* `onetomany`: A decorator for a method of a sequence to merge the results of passing every element of the sequence to the function, expecting multiple values back.\n* `onetoone`: A decorator for a method of a sequence to merge the results of passing every element of the sequence to the function, expecting a single value back.\n* `Seq`: A numeric sequence implemented as a thread safe wrapper for `itertools.count()`.\n* `seq`: Return a new sequential value.\n* `skip_map`: A version of `map()` which will skip items where `func(item)` raises an exception in `except_types`, a tuple of exception types. If a skipped exception occurs a warning will be issued unless `quiet` is true (default `False`).\n* `splitoff`: Split a sequence into (usually short) prefixes and a tail, for example to construct subdirectory trees based on a UUID.\n* `StatefulIterator`: A trivial iterator which wraps another iterator to expose some tracking state.\n* `tee`: A generator yielding the items from an iterable which also copies those items to a series of queues.\n* `the`: Returns the first element of an iterable, but requires there to be exactly one.\n* `TrackingCounter`: A wrapper for a counter which can be incremented and decremented.\n* `unrepeated`: A generator yielding items from the iterable `it` with no repetitions.\n\nModule contents:\n- <a name=\"ClonedIterator\"></a>`class ClonedIterator(collections.abc.Iterable, typing.Generic)`: A thread safe clone of some orginal iterator.\n\n `next()` of this yields the next item from the supplied iterator.\n `iter()` of this returns a generator yielding from the\n historic items and then from the original iterator.\n\n Note that this accrues all of the items from the original\n iterator in memory.\n\n*`ClonedIterator.__init__(self, it: Iterator)`*:\nInitialise the clone with its original iterator.\n\n*`ClonedIterator.__iter__(self)`*:\nIterate over the clone, returning a new iterator.\n\nIn mild violation of the iterator protocol, instead of\nreturning `self`` iter(self)` returns a generator yielding\nthe historic and then current contents of the original iterator.\n\n*`ClonedIterator.__next__(self)`*:\nReturn the next item from the original iterator.\n- <a name=\"common_prefix_length\"></a>`common_prefix_length(*seqs)`: Return the length of the common prefix of sequences `seqs`.\n- <a name=\"common_suffix_length\"></a>`common_suffix_length(*seqs)`: Return the length of the common suffix of sequences `seqs`.\n- <a name=\"first\"></a>`first(iterable)`: Return the first item from an iterable; raise `IndexError` on empty iterables.\n- <a name=\"get0\"></a>`get0(iterable, default=None)`: Return first element of an iterable, or the default.\n- <a name=\"greedy\"></a>`greedy(g=None, queue_depth=0)`: A decorator or function for greedy computation of iterables.\n\n If `g` is omitted or callable\n this is a decorator for a generator function\n causing it to compute greedily,\n capacity limited by `queue_depth`.\n\n If `g` is iterable\n this function dispatches it in a `Thread` to compute greedily,\n capacity limited by `queue_depth`.\n\n Example with an iterable:\n\n for packet in greedy(parse_data_stream(stream)):\n ... process packet ...\n\n which does some readahead of the stream.\n\n Example as a function decorator:\n\n @greedy\n def g(n):\n for item in range(n):\n yield n\n\n This can also be used directly on an existing iterable:\n\n for item in greedy(range(n)):\n yield n\n\n Normally a generator runs on demand.\n This function dispatches a `Thread` to run the iterable\n (typically a generator)\n putting yielded values to a queue\n and returns a new generator yielding from the queue.\n\n The `queue_depth` parameter specifies the depth of the queue\n and therefore how many values the original generator can compute\n before blocking at the queue's capacity.\n\n The default `queue_depth` is `0` which creates a `Channel`\n as the queue - a zero storage buffer - which lets the generator\n compute only a single value ahead of time.\n\n A larger `queue_depth` allocates a `Queue` with that much storage\n allowing the generator to compute as many as `queue_depth+1` values\n ahead of time.\n\n Here's a comparison of the behaviour:\n\n Example without `@greedy`\n where the \"yield 1\" step does not occur until after the \"got 0\":\n\n >>> from time import sleep\n >>> def g():\n ... for i in range(2):\n ... print(\"yield\", i)\n ... yield i\n ... print(\"g done\")\n ...\n >>> G = g(); sleep(0.1)\n >>> for i in G:\n ... print(\"got\", i)\n ... sleep(0.1)\n ...\n yield 0\n got 0\n yield 1\n got 1\n g done\n\n Example with `@greedy`\n where the \"yield 1\" step computes before the \"got 0\":\n\n >>> from time import sleep\n >>> @greedy\n ... def g():\n ... for i in range(2):\n ... print(\"yield\", i)\n ... yield i\n ... print(\"g done\")\n ...\n >>> G = g(); sleep(0.1)\n yield 0\n >>> for i in G:\n ... print(\"got\", repr(i))\n ... sleep(0.1)\n ...\n yield 1\n got 0\n g done\n got 1\n\n Example with `@greedy(queue_depth=1)`\n where the \"yield 1\" step computes before the \"got 0\":\n\n >>> from cs.x import X\n >>> from time import sleep\n >>> @greedy\n ... def g():\n ... for i in range(3):\n ... X(\"Y\")\n ... print(\"yield\", i)\n ... yield i\n ... print(\"g done\")\n ...\n >>> G = g(); sleep(2)\n yield 0\n yield 1\n >>> for i in G:\n ... print(\"got\", repr(i))\n ... sleep(0.1)\n ...\n yield 2\n got 0\n yield 3\n got 1\n g done\n got 2\n- <a name=\"imerge\"></a>`imerge(*iters, **kw)`: Merge an iterable of ordered iterables in order.\n\n Parameters:\n * `iters`: an iterable of iterators\n * `reverse`: keyword parameter: if true, yield items in reverse order.\n This requires the iterables themselves to also be in\n reversed order.\n\n This function relies on the source iterables being ordered\n and their elements being comparable, through slightly misordered\n iterables (for example, as extracted from web server logs)\n will produce only slightly misordered results, as the merging\n is done on the basis of the front elements of each iterable.\n- <a name=\"infill\"></a>`infill(objs: Iterable[~_infill_T], *, obj_keys: Callable[[~_infill_T], ~_infill_K], existing_keys: Callable[[~_infill_T], ~_infill_K], all: Optional[bool] = False) -> Iterable[Tuple[~_infill_T, ~_infill_K]]`: A generator accepting an iterable of objects\n which yields `(obj,missing_keys)` 2-tuples\n indicating missing records requiring infill for each object.\n\n Parameters:\n * `objs`: an iterable of objects\n * `obj_keys`: a callable accepting an object and returning\n an iterable of the expected keys\n * `existsing_keys`: a callable accepting an object and returning\n an iterable of the existing keys\n * `all`: optional flag, default `False`: if true then yield\n `(obj,())` for objects with no missing records\n\n Example:\n\n for obj, missing_key in infill(objs,...):\n ... infill a record for missing_key ...\n- <a name=\"infill_from_batches\"></a>`infill_from_batches(objss: Iterable[Iterable[~_infill_T]], *, obj_keys: Callable[[~_infill_T], ~_infill_K], existing_keys: Callable[[~_infill_T], ~_infill_K], all: Optional[bool] = False, amend_batch: Optional[Callable[[Iterable[~_infill_T]], Iterable[~_infill_T]]] = <function <lambda> at 0x107aa2f20>)`: A batched version of `infill(objs)` accepting an iterable of\n batches of objects which yields `(obj,obj_key)` 2-tuples\n indicating missing records requiring infill for each object.\n\n This is aimed at processing batches of objects where it is\n more efficient to prepare each batch as a whole, such as a\n Django `QuerySet` which lets the caller make single database\n queries for a batch of `Model` instances.\n Thus this function can be used with `cs.djutils.model_batches_qs`\n for more efficient infill processing.\n\n Parameters:\n * `objss`: an iterable of iterables of objects\n * `obj_keys`: a callable accepting an object and returning\n an iterable of the expected keys\n * `existsing_keys`: a callable accepting an object and returning\n an iterable of the existing keys\n * `all`: optional flag, default `False`: if true then yield\n `(obj,())` for objects with no missing records\n * `amend_batch`: optional callable to amend the batch of objects,\n for example to amend a `QuerySet` with `.select_related()` or similar\n- <a name=\"isordered\"></a>`isordered(items, reverse=False, strict=False)`: Test whether an iterable is ordered.\n Note that the iterable is iterated, so this is a destructive\n test for nonsequences.\n- <a name=\"last\"></a>`last(iterable)`: Return the last item from an iterable; raise `IndexError` on empty iterables.\n- <a name=\"onetomany\"></a>`onetomany(func)`: A decorator for a method of a sequence to merge the results of\n passing every element of the sequence to the function, expecting\n multiple values back.\n\n Example:\n\n class X(list):\n @onetomany\n def chars(self, item):\n return item\n strs = X(['Abc', 'Def'])\n all_chars = X.chars()\n- <a name=\"onetoone\"></a>`onetoone(func)`: A decorator for a method of a sequence to merge the results of\n passing every element of the sequence to the function, expecting a\n single value back.\n\n Example:\n\n class X(list):\n @onetoone\n def lower(self, item):\n return item.lower()\n strs = X(['Abc', 'Def'])\n lower_strs = X.lower()\n- <a name=\"Seq\"></a>`class Seq`: A numeric sequence implemented as a thread safe wrapper for\n `itertools.count()`.\n\n A `Seq` is iterable and both iterating and calling it return\n the next number in the sequence.\n- <a name=\"seq\"></a>`seq()`: Return a new sequential value.\n- <a name=\"skip_map\"></a>`skip_map(func, *iterables, except_types, quiet=False)`: A version of `map()` which will skip items where `func(item)`\n raises an exception in `except_types`, a tuple of exception types.\n If a skipped exception occurs a warning will be issued unless\n `quiet` is true (default `False`).\n- <a name=\"splitoff\"></a>`splitoff(sq, *sizes)`: Split a sequence into (usually short) prefixes and a tail,\n for example to construct subdirectory trees based on a UUID.\n\n Example:\n\n >>> from uuid import UUID\n >>> uuid = 'd6d9c510-785c-468c-9aa4-b7bda343fb79'\n >>> uu = UUID(uuid).hex\n >>> uu\n 'd6d9c510785c468c9aa4b7bda343fb79'\n >>> splitoff(uu, 2, 2)\n ['d6', 'd9', 'c510785c468c9aa4b7bda343fb79']\n- <a name=\"StatefulIterator\"></a>`class StatefulIterator`: A trivial iterator which wraps another iterator to expose some tracking state.\n\n This has 2 attributes:\n * `.it`: the internal iterator which should yield `(item,new_state)`\n * `.state`: the last state value from the internal iterator\n\n The originating use case is resuse of an iterator by independent\n calls that are typically sequential, specificly the .read\n method of file like objects. Naive sequential reads require\n the underlying storage to locate the data on every call, even\n though the previous call has just performed this task for the\n previous read. Saving the iterator used from the preceeding\n call allows the iterator to pick up directly if the file\n offset hasn't been fiddled in the meantime.\n- <a name=\"tee\"></a>`tee(iterable, *Qs)`: A generator yielding the items from an iterable\n which also copies those items to a series of queues.\n\n Parameters:\n * `iterable`: the iterable to copy\n * `Qs`: the queues, objects accepting a `.put` method.\n\n Note: the item is `.put` onto every queue\n before being yielded from this generator.\n- <a name=\"the\"></a>`the(iterable, context=None)`: Returns the first element of an iterable, but requires there to be\n exactly one.\n- <a name=\"TrackingCounter\"></a>`class TrackingCounter`: A wrapper for a counter which can be incremented and decremented.\n\n A facility is provided to wait for the counter to reach a specific value.\n The .inc and .dec methods also accept a `tag` argument to keep\n individual counts based on the tag to aid debugging.\n\n TODO: add `strict` option to error and abort if any counter tries\n to go below zero.\n\n*`TrackingCounter.__init__(self, value=0, name=None, lock=None)`*:\nInitialise the counter to `value` (default 0) with the optional `name`.\n\n*`TrackingCounter.check(self)`*:\nInternal consistency check.\n\n*`TrackingCounter.dec(self, tag=None)`*:\nDecrement the counter.\nWake up any threads waiting for its new value.\n\n*`TrackingCounter.inc(self, tag=None)`*:\nIncrement the counter.\nWake up any threads waiting for its new value.\n\n*`TrackingCounter.wait(self, value)`*:\nWait for the counter to reach the specified `value`.\n- <a name=\"unrepeated\"></a>`unrepeated(it, seen=None, signature=None)`: A generator yielding items from the iterable `it` with no repetitions.\n\n Parameters:\n * `it`: the iterable to process\n * `seen`: an optional setlike container supporting `in` and `.add()`\n * `signature`: an optional signature function for items from `it`\n which produces the value to compare to recognise repeated items;\n its values are stored in the `seen` set\n\n The default `signature` function is equality;\n the items are stored n `seen` and compared.\n This requires the items to be hashable and support equality tests.\n The same applies to whatever values the `signature` function produces.\n\n Another common signature is identity: `id`, useful for\n traversing a graph which may have cycles.\n\n Since `seen` accrues all the signature values for yielded items\n generally it will grow monotonicly as iteration proceeeds.\n If the items are complex or large it is well worth providing a signature\n function even if the items themselves can be used in a set.\n\n# Release Log\n\n\n\n*Release 20250801*:\nClonedIterator: bugfix direct iteration, noticed by @Matiss.\n\n*Release 20250724*:\nNew ClonedIterator class to provide a reiterable clone of an iterator.\n\n*Release 20250306*:\nNew infill() and infill_from_batches() generators for identifying missing records requiring an infill.\n\n*Release 20250103*:\nNew skip_map(func, *iterables, except_types, quiet=False) generator function, like map() but skipping certain exceptions.\n\n*Release 20221118*:\nSmall doc improvement.\n\n*Release 20220530*:\nSeq: calling a Seq is like next(seq).\n\n*Release 20210924*:\nNew greedy(iterable) or @greedy(generator_function) to let generators precompute.\n\n*Release 20210913*:\nNew unrepeated() generator removing duplicates from an iterable.\n\n*Release 20201025*:\nNew splitoff() function to split a sequence into (usually short) prefixes and a tail.\n\n*Release 20200914*:\nNew common_prefix_length and common_suffix_length for comparing prefixes and suffixes of sequences.\n\n*Release 20190103*:\nDocumentation update.\n\n*Release 20190101*:\n* New and UNTESTED class StatefulIterator to associate some externally visible state with an iterator.\n* Seq: accept optional `lock` parameter.\n\n*Release 20171231*:\n* Python 2 backport for imerge().\n* New tee function to duplicate an iterable to queues.\n* Function isordered() is now a test instead of an assertion.\n* Drop NamedTuple, NamedTupleClassFactory (unused).\n\n*Release 20160918*:\n* New function isordered() to test ordering of a sequence.\n* imerge: accept new `reverse` parameter for merging reversed iterables.\n\n*Release 20160828*:\nModify DISTINFO to say \"install_requires\", fixes pypi requirements.\n\n*Release 20160827*:\nTrackingCounter: accept presupplied lock object. Python 3 exec fix.\n\n*Release 20150118*:\nmetadata update\n\n*Release 20150111*:\nInitial PyPI release.\n",
"bugtrack_url": null,
"license": null,
"summary": "Stuff to do with counters, sequences and iterables.",
"version": "20250801",
"project_urls": {
"MonoRepo Commits": "https://bitbucket.org/cameron_simpson/css/commits/branch/main",
"Monorepo Git Mirror": "https://github.com/cameron-simpson/css",
"Monorepo Hg/Mercurial Mirror": "https://hg.sr.ht/~cameron-simpson/css",
"Source": "https://github.com/cameron-simpson/css/blob/main/lib/python/cs/seq.py"
},
"split_keywords": [
"python2",
" python3"
],
"urls": [
{
"comment_text": null,
"digests": {
"blake2b_256": "1697a0b3670ba18288b0159b89f981875501fc4f0ff78655ad52bda7fa515ba7",
"md5": "ed6d0dd565ea6feba49a15a5ea7cffa2",
"sha256": "077ac11770f7c8a0794483c4ff17a6e3a6e83eddb361842f5c04e87c78c7b608"
},
"downloads": -1,
"filename": "cs_seq-20250801-py3-none-any.whl",
"has_sig": false,
"md5_digest": "ed6d0dd565ea6feba49a15a5ea7cffa2",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3",
"size": 13218,
"upload_time": "2025-08-01T00:30:27",
"upload_time_iso_8601": "2025-08-01T00:30:27.553675Z",
"url": "https://files.pythonhosted.org/packages/16/97/a0b3670ba18288b0159b89f981875501fc4f0ff78655ad52bda7fa515ba7/cs_seq-20250801-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": null,
"digests": {
"blake2b_256": "f7e0a8acf18eae6f03d386099fdaa3a1de943c70ca7e91bc819281bf5b268078",
"md5": "913e9925da8aac450c10e390c57d1cd0",
"sha256": "5cd18f3308d7e8ad3f86897c69363cc82d9ee90a13dd0c0001d13390af8a5e6a"
},
"downloads": -1,
"filename": "cs_seq-20250801.tar.gz",
"has_sig": false,
"md5_digest": "913e9925da8aac450c10e390c57d1cd0",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3",
"size": 12535,
"upload_time": "2025-08-01T00:30:28",
"upload_time_iso_8601": "2025-08-01T00:30:28.709382Z",
"url": "https://files.pythonhosted.org/packages/f7/e0/a8acf18eae6f03d386099fdaa3a1de943c70ca7e91bc819281bf5b268078/cs_seq-20250801.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2025-08-01 00:30:28",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cameron-simpson",
"github_project": "css",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "cs-seq"
}
