.. image:: https://github.com/cjrh/excitertools/workflows/Python%20application/badge.svg
:target: https://github.com/cjrh/excitertools/actions
.. image:: https://coveralls.io/repos/github/cjrh/excitertools/badge.svg?branch=master
:target: https://coveralls.io/github/cjrh/excitertools?branch=master
.. image:: https://img.shields.io/pypi/pyversions/excitertools.svg
:target: https://pypi.python.org/pypi/excitertools
.. image:: https://img.shields.io/pypi/implementation/excitertools.svg
:target: https://pypi.python.org/pypi/excitertools
.. image:: https://img.shields.io/github/tag/cjrh/excitertools.svg
:target: https://github.com/cjrh/excitertools
.. image:: https://img.shields.io/badge/install-pip%20install%20excitertools-ff69b4.svg
:target: https://img.shields.io/badge/install-pip%20install%20excitertools-ff69b4.svg
.. image:: https://img.shields.io/badge/dependencies-more--itertools-4488ff.svg
:target: https://more-itertools.readthedocs.io/en/stable/
.. image:: https://img.shields.io/pypi/v/excitertools.svg
:target: https://img.shields.io/pypi/v/excitertools.svg
.. image:: https://img.shields.io/badge/calver-YYYY.MM.MINOR-22bfda.svg
:target: http://calver.org/
.. image:: https://img.shields.io/badge/code%20style-black-000000.svg
:target: https://github.com/ambv/black
.. _more-itertools: https://more-itertools.readthedocs.io/en/stable/index.html
.. _excitertools:
excitertools
############
``itertools`` reimagined as a `fluent interface <https://en.wikipedia.org/wiki/Fluent_interface>`_.
In software engineering, a fluent interface is an object-oriented API whose design
relies extensively on method chaining. Its goal is to increase code legibility by
creating a domain-specific language (DSL). The term was coined in 2005 by Eric
Evans and Martin Fowler.
`*Wikipedia - Fluent Interface* <https://en.wikipedia.org/wiki/Fluent_interface>`_
Note that nearly all of the ``more-itertools`` extension library is included.
Demo
****
.. code-block:: python
>>> range(10).map(lambda x: x*7).filter(lambda x: x % 3 == 0).collect()
[0, 21, 42, 63]
>>> range(10).map(lambda x: x*7).filter(lambda x: x > 0 and x % 3 == 0).collect()
[21, 42, 63]
When the lines get long, parens can be used to split up each instruction:
.. code-block:: python
>>> (
... range(10)
... .map(lambda x: x*7)
... .filter(lambda x: x % 3 == 0)
... .collect()
... )
[0, 21, 42, 63]
What's also interesting about that is how lambda's can easily contain these
processing chains, since an entire chain is a single expression. For
example:
.. code-block:: python
>>> names = ['caleb', 'james', 'gina']
>>> Iter(names).map(
... lambda name: (
... Iter(name)
... .map(lambda c: c.upper() if c in 'aeiouy' else c)
... .collect(str)
... )
... ).collect()
['cAlEb', 'jAmEs', 'gInA']
Something I've noticed is that ``reduce`` seems easier to use and reason
about with this fluent interface, as compared to the conventional usage
as a standalone function; also, the operator module makes ``reduce`` quite
useful for simple cases:
.. code-block:: python
>>> from operator import add, mul
>>> (
... range(10)
... .map(lambda x: x*7)
... .filter(lambda x: x > 0 and x % 3 == 0)
... .reduce(add)
... )
126
>>> (
... range(10)
... .map(lambda x: x*7)
... .filter(lambda x: x > 0 and x % 3 == 0)
... .reduce(mul)
... )
55566
.. contents::
:depth: 1
.. |warning| unicode:: U+26A0
.. |cool| unicode:: U+2728
.. |flux| unicode:: U+1F6E0
.. |source| unicode:: U+1F3A4
.. |sink| unicode:: U+1F3A7
.. |inf| unicode:: U+267E
How to Understand the API Documentation
#######################################
Several symbols are used to indicate things about parts of the API:
- |source| This function is a *source*, meaning that it produces data
that will be processed in an iterator chain.
- |sink| This function is a *sink*, meaning that it consumes data that
was processed in an iterator chain.
- |inf| This function returns an infinite iterable
- |warning| Warning - pay attention
- |flux| This API is still in flux, and might be changed or
removed in the future
- |cool| Noteworthy; could be especially useful in many situations.
The API is arranged roughly with the module-level functions first, and
thereafter the Iter_ class itself. It is the Iter_ class that does
the work to allow these iterators to be chained together. However, the
module-level functions are more likely to be used directly and that's
why they're presented first.
The API includes wrappers for the stdlib *itertools* module, including
the "recipes" given in the *itertools* docs, as well as wrappers for
the iterators from the more-itertools_ 3rd-party package.
Module-level Replacements for Builtins
######################################
The following module-level functions, like range_, zip_ and so on, are
intended to be used as replacements for their homonymous builtins. The
only difference between these and the builtin versions is that these
return instances of the Iter_ class. Note that because Iter_ is itself
iterable, it means that the functions here can be used as drop-in
replacements.
Once you have an Iter_ instance, all of its methods become available
via function call chaining, so these toplevel functions are really only
a convenience to "get started" using the chaining syntax with minimal
upfront cost in your own code.
.. contents::
:local:
.. _range:
|source| ``range(*args) -> "Iter[int]"``
****************************************
Replacement for the builtin ``range`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
All the same calling variations work because this function merely wraps
the original function.
.. code-block:: python
>>> range(3).collect()
[0, 1, 2]
>>> range(1, 4).collect()
[1, 2, 3]
>>> range(1, 6, 2).collect()
[1, 3, 5]
>>> range(1, 101, 3).filter(lambda x: x % 7 == 0).collect()
[7, 28, 49, 70, 91]
This example multiples, element by element, the series ``[0:5]`` with the
series ``[1:6]``. Two things to note: Firstly, Iter.zip_ is used to emit
the tuples from each series. Secondly, Iter.starmap_ is used to receive
those tuples into separate arguments in the ``lambda``.
.. code-block:: python
>>> range(5).zip(range(1, 6)).starmap(lambda x, y: x * y).collect()
[0, 2, 6, 12, 20]
When written in a single line as above, it can get difficult to follow
the chain of logic if there are many processing steps. Parentheses in
Python allow grouping such that expressions can be spread over multiple
lines.
This is the same example as the prior one, but formatted to be spread
over several lines. This is much clearer:
.. code-block:: python
>>> # Written out differently
>>> (
... range(5)
... .zip(range(1, 6))
... .starmap(lambda x, y: x * y)
... .collect()
... )
[0, 2, 6, 12, 20]
If you wanted the sum instead, it isn't necessary to do the collection
at all:
.. code-block:: python
>>> (
... range(5)
... .zip(range(1, 6))
... .starmap(lambda x, y: x * y)
... .sum()
... )
40
.. _zip:
``zip(*iterables: Any) -> "Iter[Tuple[T, ...]]"``
*************************************************
Replacement for the builtin ``zip`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. _enumerate:
``enumerate(iterable) -> "Iter[Tuple[int, T]]"``
************************************************
Replacement for the builtin ``enumerate`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> import string
>>> enumerate(string.ascii_lowercase).take(3).collect()
[(0, 'a'), (1, 'b'), (2, 'c')]
.. _map:
``map(func: Union[Callable[..., C], str], iterable) -> "Iter[C]"``
******************************************************************
Replacement for the builtin ``map`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> result = map(lambda x: (x, ord(x)), 'caleb').dict()
>>> assert result == {'a': 97, 'b': 98, 'c': 99, 'e': 101, 'l': 108}
>>> result = map('x, ord(x)', 'caleb').dict()
>>> assert result == {'a': 97, 'b': 98, 'c': 99, 'e': 101, 'l': 108}
.. _filter:
``filter(function: "Callable[[Any], bool]", iterable: "Iterable[T]") -> "Iter[T]"``
***********************************************************************************
Replacement for the builtin ``filter`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> filter(lambda x: x % 3 == 0, range(10)).collect()
[0, 3, 6, 9]
.. _count:
|source| ``count(start=0, step: int = 1) -> "Iter[int]"``
*********************************************************
Replacement for the itertools ``count`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> count().take(5).collect()
[0, 1, 2, 3, 4]
>>> count(0).take(0).collect()
[]
>>> count(10).take(0).collect()
[]
>>> count(10).take(5).collect()
[10, 11, 12, 13, 14]
>>> count(1).filter(lambda x: x > 10).take(5).collect()
[11, 12, 13, 14, 15]
.. _cycle:
``cycle(iterable) -> "Iter[T]"``
********************************
Replacement for the itertools ``count`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> cycle(range(3)).take(6).collect()
[0, 1, 2, 0, 1, 2]
>>> cycle([]).take(6).collect()
[]
>>> cycle(range(3)).take(0).collect()
[]
.. _repeat:
|source| ``repeat(object: C, times=None) -> "Iter[C]"``
*******************************************************
Replacement for the itertools ``count`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> repeat('a').take(3).collect()
['a', 'a', 'a']
>>> repeat([1, 2]).take(3).collect()
[[1, 2], [1, 2], [1, 2]]
>>> repeat([1, 2]).take(3).collapse().collect()
[1, 2, 1, 2, 1, 2]
>>> repeat([1, 2]).collapse().take(3).collect()
[1, 2, 1]
>>> repeat('a', times=3).collect()
['a', 'a', 'a']
This next set of functions return iterators that terminate on the shortest
input sequence.
.. _accumulate:
``accumulate(iterable, func=None, *, initial=None)``
****************************************************
Replacement for the itertools ``accumulate`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> accumulate([1, 2, 3, 4, 5]).collect()
[1, 3, 6, 10, 15]
>>> if sys.version_info >= (3, 8):
... output = accumulate([1, 2, 3, 4, 5], initial=100).collect()
... assert output == [100, 101, 103, 106, 110, 115]
>>> accumulate([1, 2, 3, 4, 5], operator.mul).collect()
[1, 2, 6, 24, 120]
>>> accumulate([]).collect()
[]
>>> accumulate('abc').collect()
['a', 'ab', 'abc']
>>> accumulate(b'abc').collect()
[97, 195, 294]
>>> accumulate(bytearray(b'abc')).collect()
[97, 195, 294]
.. _chain:
``chain(*iterables: Iterable[T]) -> "Iter[T]"``
***********************************************
Replacement for the itertools ``chain`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> chain('ABC', 'DEF').collect()
['A', 'B', 'C', 'D', 'E', 'F']
>>> chain().collect()
[]
.. _chain_from_iterable:
``chain_from_iterable(iterable) -> "Iter[T]"``
**********************************************
Replacement for the itertools ``chain.from_iterable`` method.
This version returns an instance of Iter_ to allow
further iterable chaining.
.. code-block:: python
>>> chain_from_iterable(['ABC', 'DEF']).collect()
['A', 'B', 'C', 'D', 'E', 'F']
>>> chain_from_iterable([]).collect()
[]
.. _compress:
``compress(data, selectors)``
*****************************
Replacement for the itertools ``compress`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> compress('ABCDEF', [1, 0, 1, 0, 1, 1]).collect()
['A', 'C', 'E', 'F']
.. _dropwhile:
``dropwhile(pred, iterable)``
*****************************
Replacement for the itertools ``dropwhile`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> dropwhile(lambda x: x < 4, range(6)).collect()
[4, 5]
.. _filterfalse:
``filterfalse(pred, iterable)``
*******************************
Replacement for the itertools ``filterfalse`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> filterfalse(None, [2, 0, 3, None, 4, 0]).collect()
[0, None, 0]
.. _groupby:
``groupby(iterable, key=None)``
*******************************
Replacement for the itertools ``groupby`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
groupby_ returns an iterator of a key and "grouper" iterable. In the
example below, we use Iter.starmap_ to collect each grouper iterable
into a list, as this makes it neater for display here in the docstring.
.. code-block:: python
>>> (
... groupby(['john', 'jill', 'anne', 'jack'], key=lambda x: x[0])
... .starmap(lambda k, g: (k, list(g)))
... .collect()
... )
[('j', ['john', 'jill']), ('a', ['anne']), ('j', ['jack'])]
.. _islice:
``islice(iterable, *args) -> "Iter"``
*************************************
Replacement for the itertools ``islice`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> islice('ABCDEFG', 2).collect()
['A', 'B']
>>> islice('ABCDEFG', 2, 4).collect()
['C', 'D']
>>> islice('ABCDEFG', 2, None).collect()
['C', 'D', 'E', 'F', 'G']
>>> islice('ABCDEFG', 0, None, 2).collect()
['A', 'C', 'E', 'G']
.. _starmap:
``starmap(func, iterable)``
***************************
Replacement for the itertools ``starmap`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> starmap(pow, [(2, 5), (3, 2), (10, 3)]).collect()
[32, 9, 1000]
.. _takewhile:
``takewhile(pred, iterable)``
*****************************
Replacement for the itertools ``takewhile`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> takewhile(lambda x: x < 5, [1, 4, 6, 4, 1]).collect()
[1, 4]
.. _tee:
``tee(iterable, n=2)``
**********************
Replacement for the itertools ``tee`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> a, b = tee(range(5))
>>> a.collect()
[0, 1, 2, 3, 4]
>>> b.sum()
10
It is also possible to operate on the returned iterators in the chain
but it gets quite difficult to understand:
.. code-block:: python
>>> tee(range(5)).map(lambda it: it.sum()).collect()
[10, 10]
In the example above we passed in range_, but with excitertools_
it's usually more natural to push data sources further left:
.. code-block:: python
>>> range(5).tee().map(lambda it: it.sum()).collect()
[10, 10]
Pay close attention to the above. The map_ is acting on each of the
copied iterators.
.. _zip_longest:
``zip_longest(*iterables, fillvalue=None)``
*******************************************
Replacement for the itertools ``zip_longest`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> zip_longest('ABCD', 'xy', fillvalue='-').collect()
[('A', 'x'), ('B', 'y'), ('C', '-'), ('D', '-')]
>>> (
... zip_longest('ABCD', 'xy', fillvalue='-')
... .map(lambda tup: concat(tup, ''))
... .collect()
... )
['Ax', 'By', 'C-', 'D-']
>>> (
... zip_longest('ABCD', 'xy', fillvalue='-')
... .starmap(operator.add)
... .collect()
... )
['Ax', 'By', 'C-', 'D-']
.. _finditer_regex:
``finditer_regex(pat: "re.Pattern[AnyStr]", s: AnyStr, flags: Union[int, re.RegexFlag] = 0) -> "Iter[AnyStr]"``
***************************************************************************************************************
Wrapper for ``re.finditer``. Returns an instance of Iter_ to allow
chaining.
.. code-block:: python
>>> pat = r"\w+"
>>> text = "Well hello there! How ya doin!"
>>> finditer_regex(pat, text).map(str.lower).filter(lambda w: 'o' in w).collect()
['hello', 'how', 'doin']
>>> finditer_regex(r"[A-Za-z']+", "A programmer's RegEx test.").collect()
['A', "programmer's", 'RegEx', 'test']
>>> finditer_regex(r"[A-Za-z']+", "").collect()
[]
>>> finditer_regex("", "").collect()
['']
>>> finditer_regex("", "").filter(None).collect()
[]
.. _splititer_regex:
``splititer_regex(pat: "re.Pattern[AnyStr]", s: AnyStr, flags: Union[int, re.RegexFlag] = 0) -> "Iter[AnyStr]"``
****************************************************************************************************************
Lazy string splitting using regular expressions.
Most of the time you want ``str.split``. Really! That will almost
always be fastest. You might think that ``str.split`` is inefficient
because it always has to build a list, but it can do this very, very
quickly.
The lazy splitting shown here is more about supporting a particular
kind of programming model, rather than performance.
See more discussion `here <https://stackoverflow.com/questions/3862010/is-there-a-generator-version-of-string-split-in-python>`_.
.. code-block:: python
>>> splititer_regex(r"\s", "A programmer's RegEx test.").collect()
['A', "programmer's", 'RegEx', 'test.']
Note that splitting at a single whitespace character will return blanks
for each found. This is different to how ``str.split()`` works.
.. code-block:: python
>>> splititer_regex(r"\s", "aaa bbb \n ccc\nddd\teee").collect()
['aaa', '', '', '', '', 'bbb', '', '', '', '', 'ccc', 'ddd', 'eee']
To match ``str.split()``, specify a sequence of whitespace as the
regex pattern.
.. code-block:: python
>>> splititer_regex(r"\s+", "aaa bbb \n ccc\nddd\teee").collect()
['aaa', 'bbb', 'ccc', 'ddd', 'eee']
Counting the whitespace
.. code-block:: python
>>> from collections import Counter
>>> splititer_regex(r"\s", "aaa bbb \n ccc\nddd\teee").collect(Counter)
Counter({'': 8, 'aaa': 1, 'bbb': 1, 'ccc': 1, 'ddd': 1, 'eee': 1})
Lazy splitting at newlines
.. code-block:: python
>>> splititer_regex(r"\n", "aaa bbb \n ccc\nddd\teee").collect()
['aaa bbb ', ' ccc', 'ddd\teee']
A few more examples:
.. code-block:: python
>>> splititer_regex(r"", "aaa").collect()
['', 'a', 'a', 'a', '']
>>> splititer_regex(r"", "").collect()
['', '']
>>> splititer_regex(r"\s", "").collect()
['']
>>> splititer_regex(r"a", "").collect()
['']
>>> splititer_regex(r"\s", "aaa").collect()
['aaa']
.. _concat:
``concat(iterable: Iterable[AnyStr], glue: AnyStr) -> "AnyStr"``
****************************************************************
Concatenate strings, bytes and bytearrays. It is careful to avoid the
problem with single bytes becoming integers, and it looks at the value
of `glue` to know whether to handle bytes or strings.
This function can raise ``ValueError`` if called with something
other than ``bytes``, ``bytearray`` or ``str``.
.. _from_queue:
|cool| |source| ``from_queue(q: queue.Queue, timeout=None, sentinel=None) -> "Iter"``
*************************************************************************************
Wrap a queue with an iterator interface. This allows it to participate
in chaining operations. The iterator will block while waiting for
new values to appear on the queue. This is useful: it allows you
to easily and safely pass data between threads or processes, and
feed the incoming data into a pipeline.
The sentinel value, default ``None``, will terminate the iterator.
.. code-block:: python
>>> q = queue.Queue()
>>> # This line puts stuff onto a queue
>>> range(10).chain([None]).map(q.put).consume()
>>> from_queue(q).filter(lambda x: 2 < x < 9).collect()
[3, 4, 5, 6, 7, 8]
This can be used in the same way you would normally use a queue, in
that it will block while waiting for future input. This makes it
convenient to run in a thread and wait for work. Below is a rough
sketch of how one might cobble together a thread pool using this
feature. Note the use of Iter.side_effect_ to call ``task_done()``
on the queue.
.. code-block:: python
import queue
from threading import Thread
import logging
from excitertools import from_queue
logger = logging.getLogger(__name__)
def process_job(job):
result = ...
return result
def worker(inputs: Queue, results: Queue):
(
from_queue(inputs)
.side_effect(lambda job: logger.info(f"Received job {job}")
.map(process_job)
.side_effect(lambda result: logger.info(f"Got result {job}")
.into_queue(results)
# Note that we only mark the task as done after the result
# is added to the results queue.
.side_effect(lambda _: inputs.task_done()
)
def create_job_pool(n: int) -> Tuple[Queue, Queue, Callable]:
"""Returns two queues, and a pool shutdown method. The
shutdown function can be called to shut down the pool and
the ``inputs`` queue. Caller is responsible for draining
the ``results`` queue."""
# Independent control of the sizes of the input and output
# queues is interesting: it lets you decide how to bias
# backpressure depending on the details of your workload.
inputs, results = Queue(maxsize=100), Queue(maxsize=3)
kwargs = dict(target=worker, args=(inputs, results), daemon=True)
threads = repeat(Thread).map(lambda T: T(**kwargs)).take(n).collect()
def shutdown():
# Shut down each thread
repeat(None).map(inputs.put).take(n).consume()
inputs.join()
Iter(threads).map(lambda t: t.join()).consume()
return inputs, results, shutdown
Now the two queues ``inputs`` and ``results`` can be used in various
other threads to supply and consume data.
The ``Iter`` Class
##################
.. contents::
:backlinks: entry
:local:
.. _Iter:
|cool| ``class Iter(Generic[T])``
*********************************
This class is what allows chaining. Many of the methods in this class
return an instance of Iter_, which allows further chaining. There
are two exceptions to this: *sources* and *sinks*.
A "source" is usually a ``classmethod`` which can be used as an
initializer to produce data via an iterable. For example, the Iter.range_
classmethod can be used to get a sequence of numbers:
.. code-block:: python
>>> Iter.range(1_000_000).take(3).collect()
[0, 1, 2]
Even though our range was a million elements, the iterator chaining
took only 3 of those elements before collecting.
A "sink" is a method that is usually the last component of a processing
chain and often (but not always!) consumes the entire iterator. In the
example above, the call to Iter.collect_ was a sink. Note that we still
call it a sink even though it did not consume the entire iterator.
We're using the term "source" to refer to a classmethod of Iter_ that
produces data; but, the most typical source is going to be data that
you provide. Iter_ can be called with anything that is iterable, including
sequences, iterators, mappings, sets, generators and so on.
Examples:
.. code-block:: python
List
>>> Iter([1, 2, 3]).map(lambda x: x * 2).sum()
12
Generator
>>> Iter((1, 2, 3)).map(lambda x: x * 2).sum()
12
>>> def g():
... for i in [1, 2, 3]:
... yield i
>>> Iter(g()).map(lambda x: x * 2).sum()
12
Iterator
>>> Iter(iter([1, 2, 3])).map(lambda x: x * 2).sum()
12
Dict
>>> Iter(dict(a=1, b=2)).map(lambda x: x.upper()).collect()
['A', 'B']
>>> d = dict(a=1, b=2, c=3)
>>> Iter(d.items()).starmap(lambda k, v: v).map(lambda x: x * 2).sum()
12
A common error with generators is forgetting to actually evaluate, i.e.,
call a generator function. If you do this there's a friendly error
pointing out the mistake:
.. code-block:: python
>>> def mygen(): yield 123
>>> Iter(mygen).collect()
Traceback (most recent call last):
...
TypeError: It seems you passed a generator function, but you
probably intended to pass a generator. Remember to evaluate the
function to obtain a generator instance:
def mygen():
yield 123
Iter(mygen) # ERROR - a generator function object is not iterable
Iter(mygen()) # CORRECT - a generator instance is iterable.
>>> Iter(mygen()).collect()
[123]
Instance of Iter_ are resumable. Once an instance it created, it can
be partially iterated in successive calls, like the following example
shows:
.. code-block:: python
>>> it = Iter.range(1_000_000)
>>> it.take(3).collect()
[0, 1, 2]
>>> it.take(4).collect()
[3, 4, 5, 6]
>>> # Consume most of the stream, collect the last few
>>> it.consume(999_990).collect()
[999997, 999998, 999999]
This class implements the chaining. However, the module-level functions
in excitertools_, such as range_, zip_ and so on, also return
instances of Iter_, so they allow the chaining to continue. These are
equivalent:
.. code-block:: python
>>> Iter.range(10).filter(lambda x: x > 7).collect()
[8, 9]
>>> range(10).filter(lambda x: x > 7).collect()
[8, 9]
It is intended that the module-level functions can act as drop-in
replacements for the builtins they wrap:
>>> import builtins
>>> list(builtins.range(3))
[0, 1, 2]
>>> list(range(3)) # This is excitertools.range!
[0, 1, 2]
>>> list(Iter.range(3))
[0, 1, 2]
In your own code where you might like to use the excitertools_ version of
range_ and the other functions, you can just import it and use it to access all the other
cool stuff:
.. code-block:: python
# mymodule.py
from excitertools import (
range,
map,
filter,
reduce,
repeat,
count,
enumerate,
zip,
...
)
def func(inputs):
data = (
map(lambda x: x + 2, inputs)
.enumerate()
.filter(lambda x: x[1] > 10)
...
.collect()
)
Alternatively, if you don't want to hide the builtins you can do just
fine with importing this class only, or even importing the module only:
.. code-block:: python
# mymodule.py - same example as before
import excitertools
def func(inputs):
data = (
excitertools.Iter(inputs)
.map(lambda x: x + 2, inputs)
.enumerate()
.filter(lambda x: x[1] > 10)
...
.collect()
)
# Do something with data
There are several valuable additions to the standard *itertools* and
more-itertools_ functions. These usually involve sources and sinks,
which are ways of getting data into an iterator pipeline, and then
getting results out again. In the majority of documentation examples
shown here, the Iter.collect_ method is used to collect all the
remaining data on a stream into a list; but in practice this is not
useful because large lists consume memory.
In practice it is more useful to send iterator data to one of these
common sinks:
- files
- sockets
- queues
- HTTP APIs
- Cloud storage buckets
- (Ideas for more to add here?)
Iter_ has support for these use-cases, both for reading and for writing.
.. _Iter.register:
``@classmethod Iter.register(cls, *func)``
==========================================
Add a new method to Iter_. Sure, you could subclass Iter_ to get
new chaining features, but it would be neat to let all existing
Iter_ instance just immediately have the new registered function
available.
The new function must take ``iterable`` as the first parameter.
.. code-block:: python
>>> def up(iterable):
... for v in iterable:
... yield v.upper()
>>> Iter.register(up)
>>> Iter('abc').up().collect()
['A', 'B', 'C']
>>> def poly(iterable, a, b, c):
... # Polynomials a.x^2 + b.x + c
... for x in iterable:
... yield a*x**2 + b*x + c
>>> Iter.register(poly)
>>> Iter(range(-5, 5, 1)).poly(1, -5, 6).collect()
[56, 42, 30, 20, 12, 6, 2, 0, 0, 2]
Here's a math round-trip rollercoaster.
.. code-block:: python
>>> import math
>>> def log(iterable):
... for x in iterable:
... yield math.log(x)
>>> def exp(iterable):
... for x in iterable:
... yield math.exp(x)
>>> def rnd(iterable):
... for x in iterable:
... yield round(x)
>>> Iter.register(log, exp, rnd)
>>> Iter(range(5)).exp().log().rnd().collect()
[0, 1, 2, 3, 4]
These are silly examples, but hopefully you get the idea.
.. _Iter.collect:
|sink| ``Iter.collect(self, container=list) -> "List[T]"``
==========================================================
This is the most common way of "realizing" an interable chain
into a concrete data structure. It should be the case that this
is where most of the memory allocation occurs.
The default container is a list and you'll see throughout this
documentation that most examples produce lists. However,
any container, and indeed any function, can be used as the sink.
The basic example:
.. code-block:: python
>>> Iter(range(3)).collect()
[0, 1, 2]
>>> Iter(range(3)).collect(tuple)
(0, 1, 2)
You must pay attention to some things. For example, if your
iterable is a string, the characters of the string are what
get iterated over, and when you collect you'll get a collection
of those atoms. You can however use ``str`` as your "container
function" and that will give you back a string. It's like a join
with blank joiner.
.. code-block:: python
>>> Iter('abc').collect()
['a', 'b', 'c']
>>> Iter('abc').collect(str)
'abc'
With some types, things get a little more tricky. Take ``bytes``
for example:
.. code-block:: python
>>> Iter(b'abc').collect()
[97, 98, 99]
You probably didn't expect to get the integers back right? Anyhow,
you can use ``bytes`` as the "collection container", just like
we did with strings and that will work:
.. code-block:: python
>>> Iter(b'abc').collect(bytes)
b'abc'
>>> Iter(b'abc').collect(bytearray)
bytearray(b'abc')
The other standard collections also work, here's a set for
completeness.
.. code-block:: python
>>> Iter('abcaaaabbbbccc').collect(set) == {'a', 'b', 'c'}
True
.. _Iter.open:
|cool| |source| ``@classmethod Iter.open(cls, file, mode="r", buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None, ) -> "Iter"``
==============================================================================================================================================================
Wrap the ``open()`` builtin precisely, but return an ``Iter``
instance to allow function chaining on the result.
I know you're thinking that we should always use a context
manager for files. Don't worry, there is one being used
internally. When the iterator chain is terminated the underlying
file will be closed.
>>> import tempfile
>>> with tempfile.TemporaryDirectory() as td:
... # Put some random text into a temporary file
... with open(td + 'text.txt', 'w') as f:
... f.writelines(['abc\n', 'def\n', 'ghi\n'])
...
... # Open the file, filter some lines, collect the result
... Iter.open(td + 'text.txt').filter(lambda line: 'def' in line).collect()
['def\n']
Note that this is a convenience method for *reading* from a file,
not for writing. The function signature includes the ``mode``
parameter for parity with the builtin ``open()`` function, but
only reading is supported.
.. _Iter.read_lines:
|source| ``@classmethod Iter.read_lines(cls, stream: IO[str], rewind=True)``
============================================================================
Read lines from a file-like object.
First, let's put some data in a file. We'll be using that
file in the examples that follow.
.. code-block:: python
>>> import tempfile
>>> td = tempfile.TemporaryDirectory()
... # Put some random text into a temporary file
>>> with open(td.name + 'text.txt', 'w') as f:
... f.writelines(['abc\n', 'def\n', 'ghi\n'])
...
Use read_lines to process the file data
.. code-block:: python
>>> with open(td.name + 'text.txt') as f:
... Iter.read_lines(f).filter(lambda line: 'def' in line).collect()
['def\n']
The ``rewind`` parameter can be used to read sections of a file.
.. code-block:: python
>>> with open(td.name + 'text.txt') as f:
... part1 = Iter.read_lines(f).take(1).collect()
... part2 = Iter.read_lines(f, rewind=False).collect()
>>> part1
['abc\n']
>>> part2
['def\n', 'ghi\n']
>>> td.cleanup()
.. _Iter.read_bytes:
|source| ``@classmethod Iter.read_bytes(cls, stream: IO[bytes], size: Union[Callable[[], int], int] = -1, rewind=True)``
========================================================================================================================
The ``size`` parameter can be used to control how many bytes are
read for each advancement of the iterator chain. Here we set ``size=1``
which means we'll get back one byte at a time.
.. code-block:: python
>>> import tempfile
>>> td = tempfile.TemporaryDirectory()
>>> filename = td.name + 'bytes.bin'
Put some random text into a temporary file:
.. code-block:: python
>>> with open(filename, 'wb') as f:
... x = f.write(b'\x00' * 100)
...
>>> with open(filename, 'rb') as f:
... data = Iter.read_bytes(f, size=1).collect()
... len(data)
100
>>> with open(filename, 'rb') as f:
... data = Iter.read_bytes(f).collect()
... len(data)
1
A little more ambitious. Because ``size`` is a callable, we can use
a ``deque`` and a ``side_effect`` to pass information back into
the reader to control how many bytes are read in each chunk.
In this example we're reading 1 byte at a time. In a real example
you might have a sequence of headers and bodies, where headers
give size information about how many bytes are in the body
corresponding to that header. Then you can precisely read
each body in sequence.
.. code-block:: python
>>> from collections import deque
>>> read_sizes = deque([1])
>>> with open(filename, 'rb') as f:
... data = (
... Iter
... .read_bytes(f, size=lambda: read_sizes.popleft())
... .side_effect(lambda bytes: read_sizes.append(1))
... .collect()
... )
... len(data)
100
The ``rewind`` parameter can be used to read sections of a file.
.. code-block:: python
>>> with open(filename, 'rb') as f:
... part1 = Iter.read_bytes(f, size=10).take(1).collect()
... part2 = Iter.read_bytes(f, rewind=False).collect()
>>> part1
[b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00']
>>> len(part2[0])
90
>>> td.cleanup()
.. _Iter.write_text_to_stream:
|sink| ``Iter.write_text_to_stream(self, stream: IO[str], insert_newlines=True, flush=True)``
=============================================================================================
.. code-block:: python
>>> import tempfile
>>> td = tempfile.TemporaryDirectory()
>>> filename = td.name + 'text.txt'
>>> data = ['a', 'b', 'c']
>>> with open(filename, 'w') as f:
... Iter(data).map(str.upper).write_text_to_stream(f)
... with open(filename) as f2:
... Iter.read_lines(f2).concat()
'A\nB\nC'
If some prior step adds newlines, or more commonly, newlines
originate with a data source and are simply carried through the
processing chain unaltered, disable the insertion of newlines:
.. code-block:: python
>>> with open(filename, 'w') as f:
... Iter(data).map(str.upper).write_text_to_stream(f, insert_newlines=False)
... with open(filename) as f2:
... Iter.read_lines(f2).concat()
'ABC'
Multiple successive writes may be slowed down by the default
``flush=True`` parameter. In this case you can delay flushing until
everything has been written.
.. code-block:: python
>>> with open(filename, 'w') as f:
... Iter(data).map(str.upper).write_text_to_stream(f, flush=False)
... Iter(data).map(str.upper).write_text_to_stream(f, flush=False)
... Iter(data).map(str.upper).write_text_to_stream(f, flush=True)
... with open(filename) as f2:
... Iter.read_lines(f2).concat()
'A\nB\nCA\nB\nCA\nB\nC'
>>> td.cleanup()
.. _Iter.write_bytes_to_stream:
|sink| ``Iter.write_bytes_to_stream(self, stream: IO[bytes], flush=True)``
==========================================================================
.. code-block:: python
>>> import tempfile
>>> td = tempfile.TemporaryDirectory()
>>> filename = td.name + 'bytes.bin'
>>> data = [b'a', b'b', b'c']
>>> with open(filename, 'wb') as f:
... Iter(data).map(lambda x: x * 2 ).write_bytes_to_stream(f)
... with open(filename, 'rb') as f2:
... Iter.read_bytes(f2).collect()
[b'aabbcc']
>>> with open(filename, 'wb') as f:
... Iter(data).map(lambda x: x * 2 ).write_bytes_to_stream(f)
... with open(filename, 'rb') as f2:
... Iter.read_bytes(f2).concat(b'')
b'aabbcc'
>>> with open(filename, 'wb') as f:
... Iter(data).map(lambda x: x * 2 ).write_bytes_to_stream(f)
... with open(filename, 'rb') as f2:
... Iter.read_bytes(f2, size=1).collect()
[b'a', b'a', b'b', b'b', b'c', b'c']
>>> with open(filename, 'wb') as f:
... Iter(data).map(lambda x: x * 2 ).write_bytes_to_stream(f)
... with open(filename, 'rb') as f2:
... Iter.read_bytes(f2, size=2).map(bytes.decode).collect()
['aa', 'bb', 'cc']
Flushing can be delayed if multiple parts are to be written.
.. code-block:: python
>>> with open(filename, 'wb') as f:
... it = Iter(data)
... it.map(lambda x: x * 2 ).take(2).write_bytes_to_stream(f, flush=False)
... it.map(lambda x: x * 2 ).write_bytes_to_stream(f, flush=True)
... with open(filename, 'rb') as f2:
... Iter.read_bytes(f2, size=2).map(bytes.decode).collect()
['aa', 'bb', 'cc']
>>> td.cleanup()
.. _Iter.write_to_file:
|cool| |sink| ``Iter.write_to_file(self, file, mode="w", buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None, )``
===============================================================================================================================================
.. code-block:: python
>>> import tempfile
>>> with tempfile.TemporaryDirectory() as td:
... # Put some random text into a temporary file
... with open(td + 'text.txt', 'w') as f:
... f.writelines(['abc\n', 'def\n', 'ghi\n'])
...
... # Open the file, transform, write out to new file.
... Iter.open(td + 'text.txt').map(str.upper).write_to_file(td + 'test2.txt')
... # Read the new file, for the test
... Iter.open(td + 'test2.txt').collect()
['ABC\n', 'DEF\n', 'GHI\n']
.. _Iter.range:
|source| ``@classmethod Iter.range(cls, *args) -> "Iter[int]"``
===============================================================
The ``range`` function you all know and love.
.. code-block:: python
>>> Iter.range(3).collect()
[0, 1, 2]
>>> Iter.range(0).collect()
[]
.. _Iter.zip:
``Iter.zip(self, *iterables: Any) -> "Iter[Tuple[T, ...]]"``
============================================================
The ``zip`` function you all know and love. The only thing to
note here is that the first iterable is really what the Iter_
instance is wrapping. The Iter.zip_ invocation brings in the
other iterables.
Make an Iter_ instance, then call ``zip`` on that.
.. code-block:: python
>>> Iter('caleb').zip(range(10)).collect()
[('c', 0), ('a', 1), ('l', 2), ('e', 3), ('b', 4)]
Use a classmethod to get an infinite stream using Iter.count_
and zip against that with more finite iterators.
.. code-block:: python
>>> Iter.count().zip(range(5), range(3, 100, 2)).collect()
[(0, 0, 3), (1, 1, 5), (2, 2, 7), (3, 3, 9), (4, 4, 11)]
It takes a few minutes to get used to that but feels comfortable
pretty quickly.
Iter.take_ can be used to stop infinite zip sequences:
.. code-block:: python
>>> Iter('caleb').cycle().enumerate().take(8).collect()
[(0, 'c'), (1, 'a'), (2, 'l'), (3, 'e'), (4, 'b'), (5, 'c'), (6, 'a'), (7, 'l')]
While we're here (assuming you worked through the previous
example), note the difference if you switch the order of the
Iter.cycle_ and Iter.enumerate_ calls:
.. code-block:: python
>>> Iter('caleb').enumerate().cycle().take(8).collect()
[(0, 'c'), (1, 'a'), (2, 'l'), (3, 'e'), (4, 'b'), (0, 'c'), (1, 'a'), (2, 'l')]
If you understand how this works, everything else in _excitertools_
will be intuitive to use.
.. _Iter.any:
|sink| ``Iter.any(self) -> "bool"``
===================================
.. code-block:: python
>>> Iter([0, 0, 0]).any()
False
>>> Iter([0, 0, 1]).any()
True
>>> Iter([]).any()
False
.. _Iter.all:
|sink| ``Iter.all(self) -> "bool"``
===================================
.. code-block:: python
>>> Iter([0, 0, 0]).all()
False
>>> Iter([0, 0, 1]).all()
False
>>> Iter([1, 1, 1]).all()
True
Now pay attention:
.. code-block:: python
>>> Iter([]).all()
True
This behaviour has some controversy around it, but that's how the
``all()`` builtin works so that's what we do too. The way to
think about what ``all()`` does is this: it returns False if there
is at least one element that is falsy. Thus, if there are no elements
it follows that there are no elements that are falsy and that's why
``all([]) == True``.
.. _Iter.enumerate:
``Iter.enumerate(self) -> "Iter[Tuple[int, T]]"``
=================================================
.. code-block:: python
>>> Iter('abc').enumerate().collect()
[(0, 'a'), (1, 'b'), (2, 'c')]
>>> Iter([]).enumerate().collect()
[]
.. _Iter.dict:
``Iter.dict(self) -> "Dict"``
=============================
In regular Python a dict can be constructed through an iterable
of tuples:
.. code-block:: python
>>> dict([('a', 0), ('b', 1)])
{'a': 0, 'b': 1}
In *excitertools* we prefer chaining so this method is a shortcut
for that:
.. code-block:: python
>>> d = Iter('abc').zip(count()).dict()
>>> assert d == {'a': 0, 'b': 1, 'c': 2}
.. _Iter.map:
``Iter.map(self, func: Union[Callable[..., C], str]) -> "Iter[C]"``
===================================================================
The ``map`` function you all know and love.
.. code-block:: python
>>> Iter('abc').map(str.upper).collect()
['A', 'B', 'C']
>>> Iter(['abc', 'def']).map(str.upper).collect()
['ABC', 'DEF']
Using lambdas might seem convenient but in practice it turns
out that they make code difficult to read:
.. code-block:: python
>>> result = Iter('caleb').map(lambda x: (x, ord(x))).dict()
>>> assert result == {'a': 97, 'b': 98, 'c': 99, 'e': 101, 'l': 108}
It's recommended that you make a separate function instead:
.. code-block:: python
>>> def f(x):
... return x, ord(x)
>>> result = Iter('caleb').map(f).dict()
>>> assert result == {'a': 97, 'b': 98, 'c': 99, 'e': 101, 'l': 108}
I know many people prefer anonymous functions (often on
philosphical grounds) but in practice it's just easier to make
a separate, named function.
I've experimented with passing a string into the map, and using
``eval()`` to make a lambda internally. This simplifies the code
very slightly, at the cost of using strings-as-code. I'm pretty
sure this feature will be removed so don't use it.
.. code-block:: python
>>> result = Iter('caleb').map('x, ord(x)').dict()
>>> assert result == {'a': 97, 'b': 98, 'c': 99, 'e': 101, 'l': 108}
.. _Iter.filter:
``Iter.filter(self, function: "Optional[Callable[[T], bool]]" = None) -> "Iter[T]"``
====================================================================================
The ``map`` function you all know and love.
.. code-block:: python
>>> Iter('caleb').filter(lambda x: x in 'aeiou').collect()
['a', 'e']
There is a slight difference between this method signature and
the builtin ``filter``: how the identity function is handled.
This is a consquence of chaining. In the function signature above
it is possible for us to give the ``function`` parameter a
default value of ``None`` because the parameter appears towards
the end of the parameter list. Last, in fact. In the
`builtin filter signature <https://docs.python.org/3/library/functions.html#filter>`_
it doesn't allow for this because the predicate parameter appears
first.
This is a long way of saying: if you just want to filter out
falsy values, no parameter is needed:
.. code-block:: python
>>> Iter([0, 1, 0, 0, 0, 1, 1, 1, 0, 0]).filter().collect()
[1, 1, 1, 1]
Using the builtin, you'd have to do ``filter(None, iterable)``.
You'll find that Iter.map_ and Iter.filter_
(and Iter.reduce_, up next) work together very nicely:
.. code-block:: python
>>> def not_eve(x):
... return x != 'eve'
>>> Iter(['bob', 'eve', 'alice']).filter(not_eve).map(str.upper).collect()
['BOB', 'ALICE']
The long chains get unwieldy so let's rewrite that:
.. code-block:: python
>>> (
... Iter(['bob', 'eve', 'alice'])
... .filter(not_eve)
... .map(str.upper)
... .collect()
... )
['BOB', 'ALICE']
.. _Iter.starfilter:
|cool| ``Iter.starfilter(self, function: "Optional[Callable[[T, ...], bool]]" = None) -> "Iter[T]"``
====================================================================================================
Like Iter.filter_, but arg unpacking in lambdas will work.
With the normal ``filter``, this fails:
.. code-block:: python
>>> Iter('caleb').enumerate().filter(lambda i, x: i > 2).collect()
Traceback (most recent call last):
...
TypeError: <lambda>() missing 1 required positional argument: 'x'
This is a real buzzkill. ``starfilter`` is very similar to
``starmap`` in that tuples are unpacked when calling the function:
.. code-block:: python
>>> Iter('caleb').enumerate().starfilter(lambda i, x: i > 2).collect()
[(3, 'e'), (4, 'b')]
.. _Iter.filter_gt:
``Iter.filter_gt(self, value) -> "Iter[T]"``
============================================
Convenience method
.. code-block:: python
>>> Iter([1,2,3]).filter_gt(1).collect()
[2, 3]
.. _Iter.filter_ge:
``Iter.filter_ge(self, value) -> "Iter[T]"``
============================================
Convenience method
.. code-block:: python
>>> Iter([1,2,3]).filter_ge(2).collect()
[2, 3]
.. _Iter.filter_lt:
``Iter.filter_lt(self, value) -> "Iter[T]"``
============================================
Convenience method
.. code-block:: python
>>> Iter([1,2,3]).filter_lt(3).collect()
[1, 2]
.. _Iter.filter_le:
``Iter.filter_le(self, value) -> "Iter[T]"``
============================================
Convenience method
.. code-block:: python
>>> Iter([1,2,3]).filter_le(2).collect()
[1, 2]
.. _Iter.filter_eq:
``Iter.filter_eq(self, value) -> "Iter[T]"``
============================================
Convenience method
.. code-block:: python
>>> Iter([1,2,3]).filter_eq(2).collect()
[2]
.. _Iter.filter_ne:
``Iter.filter_ne(self, value) -> "Iter[T]"``
============================================
Convenience method
.. code-block:: python
>>> Iter([1,2,3]).filter_ne(2).collect()
[1, 3]
.. _Iter.filter_in:
``Iter.filter_in(self, value: Sized) -> "Iter[T]"``
===================================================
Convenience method for membership testing. Note that the value
parameter must be at least ``Sized`` because it gets reused
over and over for each pass of the iterator chain. For example,
passing in things like ``range()`` will not work properly because
it will become progressively exhausted.
.. code-block:: python
>>> Iter([1,2,3]).filter_in([2, 3, 4, 5]).collect()
[2, 3]
>>> Iter([1,2,3]).filter_in(range(2, 8).collect()).collect()
[2, 3]
>>> Iter([1,2,3]).filter_in({2, 3, 4, 5}).collect()
[2, 3]
>>> Iter([1,2,3]).filter_in(dict.fromkeys({2, 3, 4, 5})).collect()
[2, 3]
.. _Iter.filter_ni:
``Iter.filter_ni(self, value) -> "Iter[T]"``
============================================
Convenience method for membership testing. Note that the value
parameter must be at least ``Sized`` because it gets reused
over and over for each pass of the iterator chain. For example,
passing in things like ``range()`` will not work properly because
it will become progressively exhausted.
.. code-block:: python
>>> Iter([1,2,3]).filter_ni([2, 3, 4, 5]).collect()
[1]
>>> Iter([1,2,3]).filter_ni(range(2, 8).collect()).collect()
[1]
>>> Iter([1,2,3]).filter_ni({2, 3, 4, 5}).collect()
[1]
>>> Iter([1,2,3]).filter_ni(dict.fromkeys({2, 3, 4, 5})).collect()
[1]
.. _Iter.reduce:
|sink| ``Iter.reduce(self, func: Callable[..., T], *args) -> "T"``
==================================================================
The ``reduce`` function you all know and...hang on, actually
``reduce`` is rather unloved. In the past I've found it very complex
to reason about, when looking at a bunch of nested function calls
in typical ``itertools`` code. Hopefully iterable chaining makes
it easier to read code that uses ``reduce``?
Let's check, does this make sense?
.. code-block:: python
>>> payments = [
... ('bob', 100),
... ('alice', 50),
... ('eve', -100),
... ('bob', 19.95),
... ('bob', -5.50),
... ('eve', 11.95),
... ('eve', 200),
... ('alice', -45),
... ('alice', -67),
... ('bob', 1.99),
... ('alice', 89),
... ]
>>> (
... Iter(payments)
... .filter(lambda entry: entry[0] == 'bob')
... .map(lambda entry: entry[1])
... .reduce(lambda total, value: total + value, 0)
... )
116.44
I intentionally omitted comments above so that you can try the
"readability experiment", but in practice you would definitely
want to add some comments on these chains:
.. code-block:: python
>>> (
... # Iterate over all payments
... Iter(payments)
... # Only look at bob's payments
... .filter(lambda entry: entry[0] == 'bob')
... # Extract the value of the payment
... .map(lambda entry: entry[1])
... # Add all those payments together
... .reduce(lambda total, value: total + value, 0)
... )
116.44
``reduce`` is a quite crude low-level tool. In many cases you'll
find that there are other functions and methods better suited
to the situations you'll encounter most often. For example,
there is already Iter.sum_ if you just want to add up numbers,
and it's much easier to use Iter.groupby_ for grouping than
to try to make that work with Iter.reduce_. You *can* make it
work but it'll be easier to use Iter.groupby_.
.. _Iter.starreduce:
|sink| ``Iter.starreduce(self, function: Callable[..., T], initializer=0) -> "T"``
==================================================================================
Iter.starreduce_ is the same as Iter.reduce_ except that args are
star-unpacked when passed into ``function``. This is frequently
more convenient than the default behaviour.
We can see this using the same example shown for Iter.reduce_.
The star unpacking makes it easier to just do the filtering
directly inside the reducer function.
.. code-block:: python
>>> payments = [
... ('bob', 100),
... ('alice', 50),
... ('eve', -100),
... ('bob', 19.95),
... ('bob', -5.50),
... ('eve', 11.95),
... ('eve', 200),
... ('alice', -45),
... ('alice', -67),
... ('bob', 1.99),
... ('alice', 89),
... ]
>>> (
... Iter(payments)
... .starreduce(
... lambda tot, name, value: tot + value if name == 'bob' else tot,
... 0
... )
... )
116.44
This is how that looks if you avoid a lambda:
.. code-block:: python
>>> def f(tot, name, value):
... if name == 'bob':
... return tot + value
... else:
... return tot
>>> Iter(payments).starreduce(f)
116.44
.. _Iter.sum:
|sink| ``Iter.sum(self)``
=========================
Exactly what you expect:
.. code-block:: python
>>> Iter(range(10)).sum()
45
.. _Iter.concat:
|sink| ``Iter.concat(self, glue: AnyStr = "") -> "AnyStr"``
===========================================================
Joining strings (and bytes).
.. code-block:: python
>>> Iter(['hello', 'there']).concat()
'hellothere'
>>> Iter(['hello', 'there']).concat(' ')
'hello there'
>>> Iter(['hello', 'there']).concat(',')
'hello,there'
>>> Iter([b'hello', b'there']).concat(b',')
b'hello,there'
.. _Iter.insert:
``Iter.insert(self, glue: C) -> "Iter[Union[C, T]]"``
=====================================================
Docstring TBD
.. _Iter.count:
|source| ``@classmethod Iter.count(cls, *args) -> "Iter[int]"``
===============================================================
.. code-block:: python
>>> Iter.count().take(3).collect()
[0, 1, 2]
>>> Iter.count(100).take(3).collect()
[100, 101, 102]
>>> Iter.count(100, 2).take(3).collect()
[100, 102, 104]
.. _Iter.cycle:
|inf| ``Iter.cycle(self) -> "Iter[T]"``
=======================================
.. code-block:: python
>>> Iter('abc').cycle().take(8).collect()
['a', 'b', 'c', 'a', 'b', 'c', 'a', 'b']
>>> Iter('abc').cycle().take(8).concat('')
'abcabcab'
.. _Iter.repeat:
|source| |inf| ``@classmethod Iter.repeat(cls, elem: C, times=None) -> "Iter[C]"``
==================================================================================
.. code-block:: python
>>> Iter.repeat('c', times=3).collect()
['c', 'c', 'c']
.. _Iter.accumulate:
``Iter.accumulate(self, func=None, *, initial=None)``
=====================================================
Docstring TBD
.. code-block:: python
>>> Iter([1, 2, 3, 4, 5]).accumulate().collect()
[1, 3, 6, 10, 15]
>>> if sys.version_info >= (3, 8):
... out = Iter([1, 2, 3, 4, 5]).accumulate(initial=100).collect()
... assert out == [100, 101, 103, 106, 110, 115]
>>> Iter([1, 2, 3, 4, 5]).accumulate(operator.mul).collect()
[1, 2, 6, 24, 120]
.. _Iter.chain:
``Iter.chain(self, *iterables: Iterable[T]) -> "Iter[T]"``
==========================================================
Docstring TBD
.. code-block:: python
>>> Iter('ABC').chain('DEF').collect()
['A', 'B', 'C', 'D', 'E', 'F']
>>> Iter('ABC').chain().collect()
['A', 'B', 'C']
.. _Iter.chain_from_iterable:
``Iter.chain_from_iterable(self) -> "Iter[T]"``
===============================================
Docstring TBD
.. code-block:: python
>>> Iter(['ABC', 'DEF']).chain_from_iterable().collect()
['A', 'B', 'C', 'D', 'E', 'F']
.. _Iter.compress:
``Iter.compress(self, selectors)``
==================================
Replacement for the itertools ``compress`` function. This version returns
an instance of Iter_ to allow further iterable chaining.
.. code-block:: python
>>> Iter('ABCDEF').compress([1, 0, 1, 0, 1, 1]).collect()
['A', 'C', 'E', 'F']
.. _Iter.dropwhile:
``Iter.dropwhile(self, pred)``
==============================
Docstring TBD
.. _Iter.filterfalse:
``Iter.filterfalse(self, pred)``
================================
Docstring TBD
.. _Iter.groupby:
``Iter.groupby(self, key=None)``
================================
Docstring TBD
.. _Iter.islice:
``Iter.islice(self, *args) -> "Iter"``
======================================
Docstring TBD
.. _Iter.starmap:
``Iter.starmap(self, func)``
============================
Docstring TBD
.. _Iter.takewhile:
``Iter.takewhile(self, pred)``
==============================
Docstring TBD
.. _Iter.tee:
``Iter.tee(self, n=2)``
=======================
Docstring TBD
.. _Iter.zip_longest:
``Iter.zip_longest(self, *iterables, fillvalue=None)``
======================================================
Docstring TBD
.. _Iter.chunked:
``Iter.chunked(self, n: int) -> "Iter"``
========================================
Docstring TBD
.. _Iter.ichunked:
``Iter.ichunked(self, n: int) -> "Iter"``
=========================================
Docstring TBD
.. _Iter.sliced:
``@classmethod Iter.sliced(cls, seq: Sequence, n: int) -> "Iter"``
==================================================================
Docstring TBD
.. _Iter.distribute:
``Iter.distribute(self, n: int) -> "Iter"``
===========================================
Docstring TBD
.. _Iter.divide:
``Iter.divide(self, n: int) -> "Iter"``
=======================================
Docstring TBD
.. _Iter.split_at:
``Iter.split_at(self, pred)``
=============================
Docstring TBD
.. _Iter.split_before:
``Iter.split_before(self, pred)``
=================================
Docstring TBD
.. _Iter.split_after:
``Iter.split_after(self, pred)``
================================
Docstring TBD
.. _Iter.split_into:
``Iter.split_into(self, sizes)``
================================
Docstring TBD
.. _Iter.split_when:
``Iter.split_when(self, pred)``
===============================
Docstring TBD
.. _Iter.bucket:
``Iter.bucket(self, key, validator=None)``
==========================================
This is the basic example, copied from the more-itertools
docs:
.. code-block:: python
>>> iterable = ['a1', 'b1', 'c1', 'a2', 'b2', 'c2', 'b3']
>>> b = Iter(iterable).bucket(key=lambda x: x[0])
>>> sorted(b)
['a', 'b', 'c']
>>> list(b['a'])
['a1', 'a2']
Note that once consumed, you can't iterate over the contents
of a group again.
.. _Iter.unzip:
``Iter.unzip(self)``
====================
Docstring TBD
.. _Iter.grouper:
``Iter.grouper(self, n: int, fillvalue=None) -> "Iter"``
========================================================
Docstring TBD
.. _Iter.partition:
``Iter.partition(self, pred) -> "Iter"``
========================================
Docstring TBD
.. _Iter.spy:
``Iter.spy(self, n=1) -> "Tuple[Iter, Iter]"``
==============================================
Docstring TBD
.. _Iter.peekable:
``Iter.peekable(self) -> "more_itertools.peekable"``
====================================================
Docstring TBD
.. code-block:: python
>>> p = Iter(['a', 'b']).peekable()
>>> p.peek()
'a'
>>> next(p)
'a'
The peekable can be used to inspect what will be coming up.
But if you then want to resume iterator chaining, pass the
peekable back into an Iter_ instance.
.. code-block:: python
>>> p = Iter(range(10)).peekable()
>>> p.peek()
0
>>> Iter(p).take(3).collect()
[0, 1, 2]
A peekable is not an Iter_ instance so it doesn't provide
the iterator chaining methods. But if you want to get into
chaining, use the ``iter()`` method.
.. code-block:: python
>>> p = Iter(range(5)).peekable()
>>> p.peek()
0
>>> p[1]
1
>>> p.iter().take(3).collect()
[0, 1, 2]
Peekables can be prepended. But then you usually want to go
right back to iterator chaining. Thus, the ``prepend`` method
(on the returned ``peekable`` instance) returns an Iter_ instance.
.. code-block:: python
>>> p = Iter(range(3)).peekable()
>>> p.peek()
0
>>> p.prepend('a', 'b').take(4).collect()
['a', 'b', 0, 1]
.. _Iter.seekable:
``Iter.seekable(self) -> "more_itertools.seekable"``
====================================================
Docstring TBD
.. _Iter.windowed:
``Iter.windowed(self, n, fillvalue=None, step=1) -> "Iter"``
============================================================
Docstring TBD
.. _Iter.substrings:
``Iter.substrings(self)``
=========================
Docstring TBD
.. _Iter.substrings_indexes:
``Iter.substrings_indexes(self, reverse=False)``
================================================
Docstring TBD
.. _Iter.stagger:
``Iter.stagger(self, offsets=(-1, 0, 1), longest=False, fillvalue=None)``
=========================================================================
.. code-block:: python
>>> Iter([0, 1, 2, 3]).stagger().collect()
[(None, 0, 1), (0, 1, 2), (1, 2, 3)]
>>> Iter(range(8)).stagger(offsets=(0, 2, 4)).collect()
[(0, 2, 4), (1, 3, 5), (2, 4, 6), (3, 5, 7)]
>>> Iter([0, 1, 2, 3]).stagger(longest=True).collect()
[(None, 0, 1), (0, 1, 2), (1, 2, 3), (2, 3, None), (3, None, None)]
.. _Iter.pairwise:
``Iter.pairwise(self)``
=======================
Reference `more_itertools.pairwise <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.pairwise>`_
.. code-block:: python
>>> Iter.count().pairwise().take(4).collect()
[(0, 1), (1, 2), (2, 3), (3, 4)]
.. _Iter.count_cycle:
``Iter.count_cycle(self, n=None) -> "Iter"``
============================================
Reference: `more_itertools.count_cycle <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.count_cycle>`_
.. code-block:: python
>>> Iter('AB').count_cycle(3).collect()
[(0, 'A'), (0, 'B'), (1, 'A'), (1, 'B'), (2, 'A'), (2, 'B')]
.. _Iter.intersperse:
``Iter.intersperse(self, e, n=1) -> "Iter"``
============================================
Reference: `more_itertools.intersperse <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.intersperse>`_
.. code-block:: python
>>> Iter([1, 2, 3, 4, 5]).intersperse('!').collect()
[1, '!', 2, '!', 3, '!', 4, '!', 5]
>>> Iter([1, 2, 3, 4, 5]).intersperse(None, n=2).collect()
[1, 2, None, 3, 4, None, 5]
.. _Iter.padded:
``Iter.padded(self, fillvalue: Optional[C] = None, n: Optional[int] = None, next_multiple: bool = False, ) -> "Iter[Union[T, C]]"``
===================================================================================================================================
Reference: `more_itertools.padded <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.padded>`_
.. code-block:: python
>>> Iter([1, 2, 3]).padded('?', 5).collect()
[1, 2, 3, '?', '?']
>>> Iter([1, 2, 3, 4]).padded(n=3, next_multiple=True).collect()
[1, 2, 3, 4, None, None]
.. _Iter.repeat_last:
``Iter.repeat_last(self, default=None) -> "Iter[T]"``
=====================================================
Reference: `more_itertools.repeat_last <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.repeat_last>`_
.. code-block:: python
>>> Iter(range(3)).repeat_last().islice(5).collect()
[0, 1, 2, 2, 2]
>>> Iter(range(0)).repeat_last(42).islice(5).collect()
[42, 42, 42, 42, 42]
.. _Iter.adjacent:
``Iter.adjacent(self, pred, distance=1) -> "Iter[Tuple[bool, T]]"``
===================================================================
Reference: `more_itertools.adjacent <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.adjacent>`_
.. code-block:: python
>>> Iter(range(6)).adjacent(lambda x: x == 3).collect()
[(False, 0), (False, 1), (True, 2), (True, 3), (True, 4), (False, 5)]
>>> Iter(range(6)).adjacent(lambda x: x == 3, distance=2).collect()
[(False, 0), (True, 1), (True, 2), (True, 3), (True, 4), (True, 5)]
.. _Iter.groupby_transform:
``Iter.groupby_transform(self, keyfunc: Optional[Callable[..., K]] = None, valuefunc: Optional[Callable[..., V]] = None, ) -> "Iter[Tuple[K, Iterable[V]]]"``
=============================================================================================================================================================
Reference: `more_itertools.groupby_transform <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.groupby_transform>`_
This example has been modified somewhat from the original. We're using
``starmap`` here to "unzip" the tuples produced by the group
transform.
.. code-block:: python
>>> iterable = 'AaaABbBCcA'
>>> keyfunc = lambda x: x.upper()
>>> valuefunc = lambda x: x.lower()
>>> (
... Iter(iterable)
... .groupby_transform(keyfunc, valuefunc)
... .starmap(lambda k, g: (k, ''.join(g)))
... .collect()
... )
[('A', 'aaaa'), ('B', 'bbb'), ('C', 'cc'), ('A', 'a')]
>>> from operator import itemgetter
>>> keys = [0, 0, 1, 1, 1, 2, 2, 2, 3]
>>> values = 'abcdefghi'
>>> iterable = zip(keys, values)
>>> (
... Iter(iterable)
... .groupby_transform(itemgetter(0), itemgetter(1))
... .starmap(lambda k, g: (k, ''.join(g)))
... .collect()
... )
[(0, 'ab'), (1, 'cde'), (2, 'fgh'), (3, 'i')]
.. _Iter.padnone:
``Iter.padnone(self) -> "Iter[Union[T, None]]"``
================================================
Reference: `more_itertools.padnone <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.padnone>`_
.. code-block:: python
>>> Iter(range(3)).padnone().take(5).collect()
[0, 1, 2, None, None]
.. _Iter.ncycles:
``Iter.ncycles(self, n) -> "Iter[T]"``
======================================
Reference: `more_itertools.ncycles <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.ncycles>`_
.. code-block:: python
>>> Iter(['a', 'b']).ncycles(3).collect()
['a', 'b', 'a', 'b', 'a', 'b']
.. _Iter.collapse:
``Iter.collapse(self, base_type=None, levels=None) -> "Iter"``
==============================================================
Reference: `more_itertools.collapse <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.collapse>`_
.. code-block:: python
>>> iterable = [(1, 2), ([3, 4], [[5], [6]])]
>>> Iter(iterable).collapse().collect()
[1, 2, 3, 4, 5, 6]
>>> iterable = ['ab', ('cd', 'ef'), ['gh', 'ij']]
>>> Iter(iterable).collapse(base_type=tuple).collect()
['ab', ('cd', 'ef'), 'gh', 'ij']
>>> iterable = [('a', ['b']), ('c', ['d'])]
>>> Iter(iterable).collapse().collect() # Fully flattened
['a', 'b', 'c', 'd']
>>> Iter(iterable).collapse(levels=1).collect() # Only one level flattened
['a', ['b'], 'c', ['d']]
.. _Iter.sort_together:
``@class_or_instancemethod Iter.sort_together(self_or_cls, iterables, key_list=(0,), reverse=False)``
=====================================================================================================
Reference: `more_itertools.sort_together <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.sort_together>`_
This can be called either as an instance method or a class method.
The classmethod form is more convenient if all the iterables are
already available. The instancemethod form is more convenient if
one of the iterables already goes through some transformation.
Here are examples from the classmethod form, which mirror the
examples in the more-itertools_ documentation:
.. code-block:: python
>>> iterables = [(4, 3, 2, 1), ('a', 'b', 'c', 'd')]
>>> Iter.sort_together(iterables).collect()
[(1, 2, 3, 4), ('d', 'c', 'b', 'a')]
>>> iterables = [(3, 1, 2), (0, 1, 0), ('c', 'b', 'a')]
>>> Iter.sort_together(iterables, key_list=(1, 2)).collect()
[(2, 3, 1), (0, 0, 1), ('a', 'c', 'b')]
>>> Iter.sort_together([(1, 2, 3), ('c', 'b', 'a')], reverse=True).collect()
[(3, 2, 1), ('a', 'b', 'c')]
Here is an examples using the instancemethod form:
.. code-block:: python
>>> iterables = [('a', 'b', 'c', 'd')]
>>> Iter([4, 3, 2, 1]).sort_together(iterables).collect()
[(1, 2, 3, 4), ('d', 'c', 'b', 'a')]
.. _Iter.interleave:
``@class_or_instancemethod Iter.interleave(self_or_cls, *iterables) -> "Iter"``
===============================================================================
Reference: `more_itertools.interleave <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.interleave>`_
Classmethod form:
.. code-block:: python
>>> Iter.interleave([1, 2, 3], [4, 5], [6, 7, 8]).collect()
[1, 4, 6, 2, 5, 7]
Instancemethod form:
.. code-block:: python
>>> Iter([1, 2, 3]).interleave([4, 5], [6, 7, 8]).collect()
[1, 4, 6, 2, 5, 7]
.. _Iter.interleave_longest:
``@class_or_instancemethod Iter.interleave_longest(self_or_cls, *iterables) -> "Iter"``
=======================================================================================
Reference: `more_itertools.interleave_longest <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.interleave_longest>`_
Classmethod form:
.. code-block:: python
>>> Iter.interleave_longest([1, 2, 3], [4, 5], [6, 7, 8]).collect()
[1, 4, 6, 2, 5, 7, 3, 8]
Instancemethod form:
.. code-block:: python
>>> Iter([1, 2, 3]).interleave_longest([4, 5], [6, 7, 8]).collect()
[1, 4, 6, 2, 5, 7, 3, 8]
.. _Iter.zip_offset:
``@classmethod Iter.zip_offset(cls, *iterables, offsets, longest=False, fillvalue=None) -> "Iter"``
===================================================================================================
Reference: `more_itertools.zip_offset <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.zip_offset>`_
.. code-block:: python
>>> Iter.zip_offset('0123', 'abcdef', offsets=(0, 1)).collect()
[('0', 'b'), ('1', 'c'), ('2', 'd'), ('3', 'e')]
>>> Iter.zip_offset('0123', 'abcdef', offsets=(0, 1), longest=True).collect()
[('0', 'b'), ('1', 'c'), ('2', 'd'), ('3', 'e'), (None, 'f')]
.. _Iter.dotproduct:
``Iter.dotproduct(self, vec2: Iterable)``
=========================================
Reference: `more_itertools.dotproduct <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.dotproduct>`_
.. code-block:: python
>>> Iter([10, 10]).dotproduct([20, 20])
400
.. _Iter.flatten:
``Iter.flatten(self) -> "Iter[T]"``
===================================
Reference: `more_itertools.flatten <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.flatten>`_
.. code-block:: python
>>> Iter([[0, 1], [2, 3]]).flatten().collect()
[0, 1, 2, 3]
.. _Iter.roundrobin:
``@class_or_instancemethod Iter.roundrobin(self_or_cls: Union[Type[T], T], *iterables: C) -> "Iter[Union[T, C]]"``
==================================================================================================================
Reference: `more_itertools.roundrobin <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.roundrobin>`_
Classmethod form:
.. code-block:: python
>>> Iter.roundrobin('ABC', 'D', 'EF').collect()
['A', 'D', 'E', 'B', 'F', 'C']
Instancemethod form:
.. code-block:: python
>>> Iter('ABC').roundrobin('D', 'EF').collect()
['A', 'D', 'E', 'B', 'F', 'C']
.. _Iter.prepend:
``Iter.prepend(self, value: C) -> "Iter[Union[T, C]]"``
=======================================================
Reference: `more_itertools.prepend <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.prepend>`_
.. code-block:: python
>>> value = '0'
>>> iterator = ['1', '2', '3']
>>> Iter(iterator).prepend(value).collect()
['0', '1', '2', '3']
.. _Iter.ilen:
|sink| ``Iter.ilen(self) -> "int"``
===================================
Reference: `more_itertools.ilen <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.ilen>`_
.. code-block:: python
>>> Iter(x for x in range(1000000) if x % 3 == 0).ilen()
333334
.. _Iter.unique_to_each:
``Iter.unique_to_each(self) -> "Iter[T]"``
==========================================
Reference: `more_itertools.unique_to_each <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.unique_to_each>`_
.. code-block:: python
>>> Iter([{'A', 'B'}, {'B', 'C'}, {'B', 'D'}]).unique_to_each().collect()
[['A'], ['C'], ['D']]
>>> Iter(["mississippi", "missouri"]).unique_to_each().collect()
[['p', 'p'], ['o', 'u', 'r']]
.. _Iter.sample:
``Iter.sample(self, k=1, weights=None) -> "Iter"``
==================================================
Reference: `more_itertools.sample <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.sample>`_
.. code-block:: python
>>> iterable = range(100)
>>> Iter(iterable).sample(5).collect()
[81, 60, 96, 16, 4]
>>> iterable = range(100)
>>> weights = (i * i + 1 for i in range(100))
>>> Iter(iterable).sample(5, weights=weights)
[79, 67, 74, 66, 78]
>>> data = "abcdefgh"
>>> weights = range(1, len(data) + 1)
>>> Iter(data).sample(k=len(data), weights=weights)
['c', 'a', 'b', 'e', 'g', 'd', 'h', 'f']
>>> # This one just to let the doctest run
>>> iterable = range(100)
>>> Iter(iterable).sample(5).map(lambda x: 0 <= x < 100).all()
True
.. _Iter.consecutive_groups:
``Iter.consecutive_groups(self, ordering=lambda x: x)``
=======================================================
Reference: `more_itertools.consecutive_groups <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.consecutive_groups>`_
.. code-block:: python
>>> iterable = [1, 10, 11, 12, 20, 30, 31, 32, 33, 40]
>>> Iter(iterable).consecutive_groups().map(lambda g: list(g)).print('{v}').consume()
[1]
[10, 11, 12]
[20]
[30, 31, 32, 33]
[40]
.. _Iter.run_length_encode:
``Iter.run_length_encode(self) -> "Iter[Tuple[T, int]]"``
=========================================================
Reference: `more_itertools.run_length <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.run_length>`_
.. code-block:: python
>>> uncompressed = 'abbcccdddd'
>>> Iter(uncompressed).run_length_encode().collect()
[('a', 1), ('b', 2), ('c', 3), ('d', 4)]
.. _Iter.run_length_decode:
``Iter.run_length_decode(self) -> "Iter"``
==========================================
Reference: `more_itertools.run_length <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.run_length>`_
.. code-block:: python
>>> compressed = [('a', 1), ('b', 2), ('c', 3), ('d', 4)]
>>> Iter(compressed).run_length_decode().collect()
['a', 'b', 'b', 'c', 'c', 'c', 'd', 'd', 'd', 'd']
.. _Iter.map_reduce:
``Iter.map_reduce(self, keyfunc, valuefunc=None, reducefunc=None) -> "Dict"``
=============================================================================
Reference: `more_itertools.map_reduce <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.map_reduce>`_
This interface mirrors what more-itertools_ does in that it returns
a dict. See ``map_reduce_it()`` for a slightly-modified interface
that returns the dict items as another iterator.
.. code-block:: python
>>> keyfunc = lambda x: x.upper()
>>> d = Iter('abbccc').map_reduce(keyfunc)
>>> sorted(d.items())
[('A', ['a']), ('B', ['b', 'b']), ('C', ['c', 'c', 'c'])]
>>> keyfunc = lambda x: x.upper()
>>> valuefunc = lambda x: 1
>>> d = Iter('abbccc').map_reduce(keyfunc, valuefunc)
>>> sorted(d.items())
[('A', [1]), ('B', [1, 1]), ('C', [1, 1, 1])]
>>> keyfunc = lambda x: x.upper()
>>> valuefunc = lambda x: 1
>>> reducefunc = sum
>>> d = Iter('abbccc').map_reduce(keyfunc, valuefunc, reducefunc)
>>> sorted(d.items())
[('A', 1), ('B', 2), ('C', 3)]
Note the warning given in the more-itertools_ docs about how
lists are created before the reduce step. This means you always want
to filter *before* applying map_reduce, not after.
.. code-block:: python
>>> all_items = _range(30)
>>> keyfunc = lambda x: x % 2 # Evens map to 0; odds to 1
>>> categories = Iter(all_items).filter(lambda x: 10<=x<=20).map_reduce(keyfunc=keyfunc)
>>> sorted(categories.items())
[(0, [10, 12, 14, 16, 18, 20]), (1, [11, 13, 15, 17, 19])]
>>> summaries = Iter(all_items).filter(lambda x: 10<=x<=20).map_reduce(keyfunc=keyfunc, reducefunc=sum)
>>> sorted(summaries.items())
[(0, 90), (1, 75)]
.. _Iter.map_reduce_it:
``Iter.map_reduce_it(self, keyfunc: Callable[..., K], valuefunc: Optional[Callable[..., V]] = None, reducefunc: Optional[Callable[..., R]] = None, ) -> "Iter[Tuple[K, R]]"``
=============================================================================================================================================================================
Reference: `more_itertools.map_reduce <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.map_reduce>`_
.. code-block:: python
>>> keyfunc = lambda x: x.upper()
>>> Iter('abbccc').map_reduce_it(keyfunc).collect()
[('A', ['a']), ('B', ['b', 'b']), ('C', ['c', 'c', 'c'])]
>>> keyfunc = lambda x: x.upper()
>>> valuefunc = lambda x: 1
>>> Iter('abbccc').map_reduce_it(keyfunc, valuefunc).collect()
[('A', [1]), ('B', [1, 1]), ('C', [1, 1, 1])]
>>> keyfunc = lambda x: x.upper()
>>> valuefunc = lambda x: 1
>>> reducefunc = sum
>>> Iter('abbccc').map_reduce_it(keyfunc, valuefunc, reducefunc).collect()
[('A', 1), ('B', 2), ('C', 3)]
.. _Iter.exactly_n:
|sink| ``Iter.exactly_n(self, n, predicate=bool) -> "bool"``
============================================================
Docstring TBD
.. code-block:: python
>>> Iter([True, True, False]).exactly_n(2)
True
.. _Iter.all_equal:
``Iter.all_equal(self)``
========================
.. _Iter.first_true:
``Iter.first_true(self)``
=========================
.. _Iter.quantify:
``Iter.quantify(self)``
=======================
.. _Iter.islice_extended:
``Iter.islice_extended(self, *args)``
=====================================
Reference: `more_itertools.islice_extended <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.islice_extended>`_
.. code-block:: python
>>> Iter('abcdefgh').islice_extended(-4, -1).collect()
['e', 'f', 'g']
.. code-block:: python
>>> Iter.count().islice_extended( 110, 99, -2).collect()
[110, 108, 106, 104, 102, 100]
.. _Iter.first:
``Iter.first(self)``
====================
Reference: `more_itertools.first <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.first>`_
.. _Iter.last:
``Iter.last(self)``
===================
Reference: `more_itertools.last <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.last>`_
.. _Iter.one:
``Iter.one(self)``
==================
Reference: `more_itertools.one <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.one>`_
.. _Iter.only:
``Iter.only(self, default=None, too_long=ValueError) -> "T"``
=============================================================
Reference: `more_itertools.one <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.one>`_
.. code-block:: python
>>> Iter([]).only(default='missing')
'missing'
>>> Iter([42]).only(default='missing')
42
>>> Iter([1, 2]).only()
Traceback (most recent call last):
...
ValueError: ...
.. _Iter.strip:
``Iter.strip(self, pred) -> "Iter[T]"``
=======================================
Reference: `more_itertools.strip <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.strip>`_
.. code-block:: python
>>> iterable = (None, False, None, 1, 2, None, 3, False, None)
>>> pred = lambda x: x in {None, False, ''}
>>> Iter(iterable).strip(pred).collect()
[1, 2, None, 3]
.. _Iter.lstrip:
``Iter.lstrip(self, pred) -> "Iter[T]"``
========================================
Reference: `more_itertools.lstrip <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.lstrip>`_
.. code-block:: python
>>> iterable = (None, False, None, 1, 2, None, 3, False, None)
>>> pred = lambda x: x in {None, False, ''}
>>> Iter(iterable).lstrip(pred).collect()
[1, 2, None, 3, False, None]
.. _Iter.rstrip:
``Iter.rstrip(self, pred) -> "Iter[T]"``
========================================
Reference: `more_itertools.rstrip <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.rstrip>`_
.. code-block:: python
>>> iterable = (None, False, None, 1, 2, None, 3, False, None)
>>> pred = lambda x: x in {None, False, ''}
>>> Iter(iterable).rstrip(pred).collect()
[None, False, None, 1, 2, None, 3]
.. _Iter.filter_except:
``Iter.filter_except(self, validator, *exceptions) -> "Iter[T]"``
=================================================================
Reference: `more_itertools.filter_except <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.filter_except>`_
.. code-block:: python
>>> iterable = ['1', '2', 'three', '4', None]
>>> Iter(iterable).filter_except(int, ValueError, TypeError).collect()
['1', '2', '4']
.. _Iter.map_except:
``Iter.map_except(self, function, *exceptions) -> "Iter"``
==========================================================
Reference: `more_itertools.map_except <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.map_except>`_
.. code-block:: python
>>> iterable = ['1', '2', 'three', '4', None]
>>> Iter(iterable).map_except(int, ValueError, TypeError).collect()
[1, 2, 4]
.. _Iter.nth_or_last:
``Iter.nth_or_last(self, n, default=_marker) -> "T"``
=====================================================
Reference: `more_itertools.nth_or_last <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.nth_or_last>`_
.. code-block:: python
>>> Iter([0, 1, 2, 3]).nth_or_last(2)
2
>>> Iter([0, 1]).nth_or_last(2)
1
>>> Iter([]).nth_or_last(0, 'some default')
'some default'
.. _Iter.nth:
``Iter.nth(self, n, default=None)``
===================================
Reference: `more_itertools.nth <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.nth>`_
.. _Iter.take:
``Iter.take(self, n: int) -> "Iter"``
=====================================
Reference: `more_itertools.take <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.take>`_
.. _Iter.tail:
``Iter.tail(self, n) -> "Iter[T]"``
===================================
Reference: `more_itertools.tail <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.tail>`_
.. code-block:: python
>>> Iter('ABCDEFG').tail(3).collect()
['E', 'F', 'G']
.. _Iter.unique_everseen:
``Iter.unique_everseen(self, key=None) -> "Iter[T]"``
=====================================================
Reference: `more_itertools.unique_everseen <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.unique_everseen>`_
.. code-block:: python
>>> Iter('AAAABBBCCDAABBB').unique_everseen().collect()
['A', 'B', 'C', 'D']
>>> Iter('ABBCcAD').unique_everseen(key=str.lower).collect()
['A', 'B', 'C', 'D']
Be sure to read the *more-itertools* docs whne using unhashable
items.
.. code-block:: python
>>> iterable = ([1, 2], [2, 3], [1, 2])
>>> Iter(iterable).unique_everseen().collect() # Slow
[[1, 2], [2, 3]]
>>> Iter(iterable).unique_everseen(key=tuple).collect() # Faster
[[1, 2], [2, 3]]
.. _Iter.unique_justseen:
``Iter.unique_justseen(self, key=None) -> "Iter[T]"``
=====================================================
Reference: `more_itertools.unique_justseen <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.unique_justseen>`_
.. code-block:: python
>>> Iter('AAAABBBCCDAABBB').unique_justseen().collect()
['A', 'B', 'C', 'D', 'A', 'B']
>>> Iter('ABBCcAD').unique_justseen(key=str.lower).collect()
['A', 'B', 'C', 'A', 'D']
.. _Iter.distinct_permutations:
``Iter.distinct_permutations(self)``
====================================
Reference: `more_itertools.distinct_permutations <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.distinct_permutations>`_
.. code-block:: python
>>> Iter([1, 0, 1]).distinct_permutations().sorted().collect()
[(0, 1, 1), (1, 0, 1), (1, 1, 0)]
.. _Iter.distinct_combinations:
``Iter.distinct_combinations(self, r) -> "Iter[T]"``
====================================================
Reference: `more_itertools.distinct_combinations <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.distinct_combinations>`_
.. code-block:: python
>>> Iter([0, 0, 1]).distinct_combinations(2).collect()
[(0, 0), (0, 1)]
.. _Iter.circular_shifts:
``Iter.circular_shifts(self) -> "Iter[T]"``
===========================================
Reference: `more_itertools.circular_shifts <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.circular_shifts>`_
.. code-block:: python
>>> Iter(range(4)).circular_shifts().collect()
[(0, 1, 2, 3), (1, 2, 3, 0), (2, 3, 0, 1), (3, 0, 1, 2)]
.. _Iter.partitions:
``Iter.partitions(self) -> "Iter[T]"``
======================================
Reference: `more_itertools.partitions <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.partitions>`_
.. code-block:: python
>>> Iter('abc').partitions().collect()
[[['a', 'b', 'c']], [['a'], ['b', 'c']], [['a', 'b'], ['c']], [['a'], ['b'], ['c']]]
>>> Iter('abc').partitions().print('{v}').consume()
[['a', 'b', 'c']]
[['a'], ['b', 'c']]
[['a', 'b'], ['c']]
[['a'], ['b'], ['c']]
>>> Iter('abc').partitions().map(lambda v: [''.join(p) for p in v]).print('{v}').consume()
['abc']
['a', 'bc']
['ab', 'c']
['a', 'b', 'c']
.. _Iter.set_partitions:
``Iter.set_partitions(self, k=None) -> "Iter[T]"``
==================================================
Reference: `more_itertools.set_partitions <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.set_partitions>`_
.. code-block:: python
>>> Iter('abc').set_partitions(2).collect()
[[['a'], ['b', 'c']], [['a', 'b'], ['c']], [['b'], ['a', 'c']]]
.. _Iter.powerset:
``Iter.powerset(self)``
=======================
Reference: `more_itertools.powerset <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.powerset>`_
.. code-block:: python
>>> Iter([1, 2, 3]).powerset().collect()
[(), (1,), (2,), (3,), (1, 2), (1, 3), (2, 3), (1, 2, 3)]
.. _Iter.random_product:
``@class_or_instancemethod Iter.random_product(self_or_cls, *args, repeat=1)``
==============================================================================
Reference: `more_itertools.random_product <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.random_product>`_
.. code-block:: python
>>> Iter('abc').random_product(range(4), 'XYZ').collect()
['c', 3, 'X']
>>> Iter.random_product('abc', range(4), 'XYZ').collect()
['c', 0, 'Z']
>>> Iter('abc').random_product(range(0)).collect()
Traceback (most recent call last):
...
IndexError: Cannot choose from an empty sequence
>>> Iter.random_product(range(0)).collect()
Traceback (most recent call last):
...
IndexError: Cannot choose from an empty sequence
.. _Iter.random_permutation:
``Iter.random_permutation(self, r=None)``
=========================================
Reference: `more_itertools.random_permutation <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.random_permutation>`_
.. code-block:: python
>>> Iter(range(5)).random_permutation().collect()
[2, 0, 4, 3, 1]
>>> Iter(range(0)).random_permutation().collect()
[]
.. _Iter.random_combination:
``Iter.random_combination(self, r)``
====================================
Reference: `more_itertools.random_combination <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.random_combination>`_
.. code-block:: python
>>> Iter(range(5)).random_combination(3).collect()
[0, 1, 4]
>>> Iter(range(5)).random_combination(0).collect()
[]
.. _Iter.random_combination_with_replacement:
``Iter.random_combination_with_replacement(self, r)``
=====================================================
Reference: `more_itertools.random_combination_with_replacement <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.random_combination_with_replacement>`_
.. code-block:: python
>>> Iter(range(3)).random_combination_with_replacement(5).collect()
[0, 0, 1, 2, 2]
>>> Iter(range(3)).random_combination_with_replacement(0).collect()
[]
.. _Iter.nth_combination:
``Iter.nth_combination(self, r, index)``
========================================
Reference: `more_itertools.nth_combination <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.nth_combination>`_
.. code-block:: python
>>> Iter(range(9)).nth_combination(3, 1).collect()
[0, 1, 3]
>>> Iter(range(9)).nth_combination(3, 2).collect()
[0, 1, 4]
>>> Iter(range(9)).nth_combination(3, 3).collect()
[0, 1, 5]
>>> Iter(range(9)).nth_combination(4, 3).collect()
[0, 1, 2, 6]
>>> Iter(range(9)).nth_combination(3, 7).collect()
[0, 2, 3]
.. _Iter.always_iterable:
``@classmethod Iter.always_iterable(cls, obj, base_type=(str, bytes)) -> "Iter"``
=================================================================================
Reference: `more_itertools.always_iterable <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.always_iterable>`_
.. code-block: python
.. code-block:: python
>>> Iter.always_iterable([1, 2, 3]).collect()
[1, 2, 3]
>>> Iter.always_iterable(1).collect()
[1]
>>> Iter.always_iterable(None).collect()
[]
>>> Iter.always_iterable('foo').collect()
['foo']
>>> Iter.always_iterable(dict(a=1), base_type=dict).collect()
[{'a': 1}]
.. _Iter.always_reversible:
``Iter.always_reversible(self)``
================================
Reference: `more_itertools.always_reversible <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.always_reversible>`_
This is like ``reversed()`` but it also operates on things that
wouldn't normally be reversible, like generators. It does this with
internal caching, so be careful with memory use.
.. code-block: python
>>> Iter('abc').always_reversible().collect()
['c', 'b', 'a']
>>> Iter(x for x in 'abc').always_reversible().collect()
['c', 'b', 'a']
.. _Iter.with_iter:
``@classmethod Iter.with_iter(cls, context_manager)``
=====================================================
Reference: `more_itertools.with_iter <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.with_iter>`_
Note: Any context manager which returns an iterable is a candidate for
Iter.with_iter_.
.. code-block:: python
>>> import tempfile
>>> with tempfile.TemporaryDirectory() as td:
... with open(td + 'text.txt', 'w') as f:
... f.writelines(['abc\n', 'def\n', 'ghi\n'])
... Iter.with_iter(open(td + 'text.txt')).map(lambda x: x.upper()).collect()
['ABC\n', 'DEF\n', 'GHI\n']
See also: Iter.open_
|flux| TODO: perhaps we should get rid of Iter.open_ and just use this?
.. _Iter.iter_except:
``@classmethod Iter.iter_except(cls, func, exception, first=None) -> "Iter"``
=============================================================================
Reference: `more_itertools.iter_except <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.iter_except>`_
.. code-block:: python
>>> l = [0, 1, 2]
>>> Iter.iter_except(l.pop, IndexError).collect()
[2, 1, 0]
.. _Iter.locate:
``Iter.locate(self, pred=bool, window_size=None) -> "Iter"``
============================================================
Reference: `more_itertools.locate <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.locate>`_
.. code-block:: python
>>> Iter([0, 1, 1, 0, 1, 0, 0]).locate().collect()
[1, 2, 4]
.. code-block:: python
>>> Iter(['a', 'b', 'c', 'b']).locate(lambda x: x == 'b').collect()
[1, 3]
.. code-block:: python
>>> iterable = [0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3]
>>> pred = lambda *args: args == (1, 2, 3)
>>> Iter(iterable).locate(pred=pred, window_size=3).collect()
[1, 5, 9]
.. code-block:: python
>>> from itertools import count
>>> from more_itertools import seekable
>>> source = (3 * n + 1 if (n % 2) else n // 2 for n in count())
>>> it = Iter(source).seekable()
>>> pred = lambda x: x > 100
>>> # TODO: can we avoid making two instances?
>>> indexes = Iter(it).locate(pred=pred)
>>> i = next(indexes)
>>> it.seek(i)
>>> next(it)
106
.. _Iter.rlocate:
``Iter.rlocate(self, pred=bool, window_size=None) -> "Iter"``
=============================================================
Reference: `more_itertools.rlocate <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.rlocate>`_
.. code-block:: python
>>> Iter([0, 1, 1, 0, 1, 0, 0]).rlocate().collect() # Truthy at 1, 2, and 4
[4, 2, 1]
.. code-block:: python
>>> pred = lambda x: x == 'b'
>>> Iter('abcb').rlocate(pred).collect()
[3, 1]
.. code-block:: python
>>> iterable = [0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3]
>>> pred = lambda *args: args == (1, 2, 3)
>>> Iter(iterable).rlocate(pred=pred, window_size=3).collect()
[9, 5, 1]
.. _Iter.replace:
``Iter.replace(self, pred, substitutes, count=None, window_size=1) -> "Iter"``
==============================================================================
Reference: `more_itertools.replace <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.replace>`_
.. code-block:: python
>>> iterable = [1, 1, 0, 1, 1, 0, 1, 1]
>>> pred = lambda x: x == 0
>>> substitutes = (2, 3)
>>> Iter(iterable).replace(pred, substitutes).collect()
[1, 1, 2, 3, 1, 1, 2, 3, 1, 1]
.. code-block:: python
>>> iterable = [1, 1, 0, 1, 1, 0, 1, 1, 0]
>>> pred = lambda x: x == 0
>>> substitutes = [None]
>>> Iter(iterable).replace(pred, substitutes, count=2).collect()
[1, 1, None, 1, 1, None, 1, 1, 0]
.. code-block:: python
>>> iterable = [0, 1, 2, 5, 0, 1, 2, 5]
>>> window_size = 3
>>> pred = lambda *args: args == (0, 1, 2) # 3 items passed to pred
>>> substitutes = [3, 4] # Splice in these items
>>> Iter(iterable).replace(
... pred, substitutes, window_size=window_size
... ).collect()
[3, 4, 5, 3, 4, 5]
.. _Iter.numeric_range:
``@classmethod Iter.numeric_range(cls, *args) -> "Iter"``
=========================================================
Reference: `more_itertools.numeric_range <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.numeric_range>`_
.. code-block:: python
>>> Iter.numeric_range(3.5).collect()
[0.0, 1.0, 2.0, 3.0]
.. code-block:: python
>>> from decimal import Decimal
>>> start = Decimal('2.1')
>>> stop = Decimal('5.1')
>>> Iter.numeric_range(start, stop).collect()
[Decimal('2.1'), Decimal('3.1'), Decimal('4.1')]
.. code-block:: python
>>> from fractions import Fraction
>>> start = Fraction(1, 2) # Start at 1/2
>>> stop = Fraction(5, 2) # End at 5/2
>>> step = Fraction(1, 2) # Count by 1/2
>>> Iter.numeric_range(start, stop, step).collect()
[Fraction(1, 2), Fraction(1, 1), Fraction(3, 2), Fraction(2, 1)]
.. code-block:: python
>>> Iter.numeric_range(3, -1, -1.0).collect()
[3.0, 2.0, 1.0, 0.0]
.. _Iter.side_effect:
``Iter.side_effect(self, func, *args, chunk_size=None, before=None, after=None)``
=================================================================================
Reference: `more_itertools.side_effect <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.side_effect>`_
.. code-block:: python
>>> def f(item):
... if item == 3:
... raise Exception('got 3')
>>> Iter.range(5).side_effect(f).consume()
Traceback (most recent call last):
...
Exception: got 3
.. code-block:: python
>>> func = lambda item: print('Received {}'.format(item))
>>> Iter.range(2).side_effect(func).consume()
Received 0
Received 1
This version of ``side_effect`` also allows extra args:
.. code-block:: python
>>> func = lambda item, format_str='Received {}': print(format_str.format(item))
>>> Iter.range(2).side_effect(func).consume()
Received 0
Received 1
>>> func = lambda item, format_str='Received {}': print(format_str.format(item))
>>> Iter.range(2).side_effect(func, 'Got {}').consume()
Got 0
Got 1
.. _Iter.iterate:
``Iter.iterate(self)``
======================
.. _Iter.difference:
``Iter.difference(self, func=operator.sub, *, initial=None)``
=============================================================
Reference: `more_itertools.difference <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=difference#more_itertools.difference>`_
.. code-block:: python
>>> iterable = [0, 1, 3, 6, 10]
>>> Iter(iterable).difference().collect()
[0, 1, 2, 3, 4]
.. code-block:: python
>>> iterable = [1, 2, 6, 24, 120] # Factorial sequence
>>> func = lambda x, y: x // y
>>> Iter(iterable).difference(func).collect()
[1, 2, 3, 4, 5]
.. _Iter.make_decorator:
``Iter.make_decorator(self)``
=============================
.. _Iter.SequenceView:
``Iter.SequenceView(self)``
===========================
.. _Iter.time_limited:
``Iter.time_limited(self, limit_seconds) -> "Iter"``
====================================================
Reference: `more_itertools.time_limited <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=time_limited#more_itertools.time_limited>`_
.. code-block:: python
>>> from time import sleep
>>> def generator():
... yield 1
... yield 2
... sleep(0.2)
... yield 3
>>> Iter(generator()).time_limited(0.1).collect()
[1, 2]
.. _Iter.consume:
|sink| ``Iter.consume(self, n: Optional[int] = None) -> "Optional[Iter[T]]"``
=============================================================================
If n is not provided, the entire iterator is consumed and
``None`` is returned. Otherwise, an iterator will *always* be
returned, even if n is greater than the number of items left in
the iterator.
In this example, the source has more elements than what we consume,
so there will still be data available on the chain:
.. code-block:: python
>>> range(10).consume(5).collect()
[5, 6, 7, 8, 9]
We can bump up the count of how many items can be consumed. Note that
even though ``n`` is greater than the number of items in the source,
it is still required to call Iter.collect_ to consume the remaining
items.
.. code-block:: python
>>> range(10).consume(50).collect()
[]
Finally, if ``n`` is not provided, the entire stream is consumed.
In this scenario, Iter.collect_ would fail since nothing is being
returned from the consume call.
.. code-block:: python
>>> assert range(10).consume() is None
.. _Iter.tabulate:
``Iter.tabulate(self)``
=======================
.. _Iter.repeatfunc:
|source| ``@classmethod Iter.repeatfunc(cls, func, *args, times=None)``
=======================================================================
Docstring TBD
.. code-block:: python
>>> Iter.repeatfunc(operator.add, 3, 5, times=4).collect()
[8, 8, 8, 8]
.. _Iter.wrap:
``Iter.wrap(self, ends: "Sequence[T, T]" = "()")``
==================================================
Other examples for ends: '"' * 2, or '`' * 2, or '[]' etc.
.. _Iter.print:
``Iter.print(self, template="{i}: {v}") -> "Iter[T]"``
======================================================
Printing during the execution of an iterator. Mostly useful
for debugging. Returns another iterator instance through which
the original data is passed unchanged. This means you can include
a `print()` step as necessary to observe data during iteration.
.. code-block:: python
>>> Iter('abc').print().collect()
0: a
1: b
2: c
['a', 'b', 'c']
>>> (
... Iter(range(5))
... .print('before filter {i}: {v}')
... .filter(lambda x: x > 2)
... .print('after filter {i}: {v}')
... .collect()
... )
before filter 0: 0
before filter 1: 1
before filter 2: 2
before filter 3: 3
after filter 0: 3
before filter 4: 4
after filter 1: 4
[3, 4]
.. _Iter.from_queue:
|source| ``@classmethod Iter.from_queue(cls, q: queue.Queue, timeout=None, sentinel=None)``
===========================================================================================
Wrap a queue with an iterator interface. This allows it to participate
in chaining operations. The iterator will block while waiting for
new values to appear on the queue. This is useful: it allows you
to easily and safely pass data between threads or processes, and
feed the incoming data into a pipeline.
The sentinel value, default ``None``, will terminate the iterator.
.. code-block:: python
>>> q = queue.Queue()
>>> # This line puts stuff onto a queue
>>> range(10).chain([None]).map(q.put).consume()
>>> # This is where we consume data from the queue:
>>> Iter.from_queue(q).filter(lambda x: 2 < x < 9).collect()
[3, 4, 5, 6, 7, 8]
If ``None`` had not been chained onto the data, the iterator would
have waited in Iter.collect_ forever.
.. _Iter.into_queue:
``Iter.into_queue(self, q: queue.Queue) -> "Iter[T]"``
======================================================
This is a sink, like Iter.collect_, that consumes data from
an iterator chain and puts the data into the given queue.
.. code-block:: python
>>> q = queue.Queue()
>>> # This demonstrates the queue sink
>>> range(5).into_queue(q).consume()
>>> # Code below is only for verification
>>> out = []
>>> finished = False
>>> while not finished:
... try:
... out.append(q.get_nowait())
... except queue.Empty:
... finished = True
>>> out
[0, 1, 2, 3, 4]
.. _Iter.send:
|sink| ``Iter.send(self, collector: Generator, close_collector_when_done=False) -> "None"``
===========================================================================================
See also: `more_itertools.consumer <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.consumer>`_
Send data into a generator. You do not have to first call ``next()``
on the generator. Iter.send_ will do this for you.
|warning| Look carefully at the examples below; you'll see that the
``yield`` keyword is wrapped in a second set of parens, e.g.
``output.append((yield))``. This is required!
Simple case:
.. code-block:: python
>>> output = []
>>> def collector():
... while True:
... output.append((yield))
>>> Iter.range(3).send(collector())
>>> output
[0, 1, 2]
Note that the generator is **not** closed by default after the iterable is
exhausted. But this can be changed. If you choose to close the
generator, use the parameter:
.. code-block:: python
>>> output = []
>>> def collector():
... while True:
... output.append((yield))
>>> g = collector()
>>> Iter.range(3).send(g, close_collector_when_done=True)
>>> Iter.range(3).send(g)
Traceback (most recent call last):
...
StopIteration
The default behaviour is that the generator is left open which means you
can keep using it for other iterators:
.. code-block:: python
>>> output = []
>>> def collector():
... while True:
... output.append((yield))
>>> g = collector()
>>> Iter.range(3).send(g)
>>> Iter.range(10, 13).send(g)
>>> Iter.range(100, 103).send(g)
>>> output
[0, 1, 2, 10, 11, 12, 100, 101, 102]
If the generator is closed before the iteration is complete,
you'll get a ``StopIteration`` exception:
.. code-block:: python
>>> output = []
>>> def collector():
... for i in range(3):
... output.append((yield))
>>> Iter.range(5).send(collector())
Traceback (most recent call last):
...
StopIteration
Note that Iter.send_ is a sink, so no further chaining is allowed.
.. _Iter.send_also:
``Iter.send_also(self, collector: Generator) -> "Iter"``
========================================================
Reference: `more_itertools.consumer <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.consumer>`_
Some ideas around a reverse iterator as a sink. Usually you have
first to "send" a ``None`` into a generator if you want to send
more values into it (or call ``next()`` on it), but we handle
that automatically.
Simple case:
.. code-block:: python
>>> output = []
>>> def collector():
... while True:
... output.append((yield))
>>> Iter.range(3).send_also(collector()).collect()
[0, 1, 2]
>>> output
[0, 1, 2]
However, if the caller already started the generator, that
works too:
.. code-block:: python
>>> output = []
>>> def collector():
... while True:
... output.append((yield))
>>> g = collector()
>>> next(g) # This "starts" the generator
>>> Iter.range(3).send_also(g).collect()
[0, 1, 2]
>>> output
[0, 1, 2]
If the generator is closed before the iteration is complete,
you'll get an exception (Python 3.7+):
.. code-block:: python
>>> output = []
>>> def collector():
... for i in builtins.range(3):
... output.append((yield))
>>> Iter.range(50).send_also(collector()).collect()
Traceback (most recent call last):
...
RuntimeError
Note that the above doesn't happen in Python < 3.7 (which includes
pypy 7.3.1 that matches Python 3.6.9 compatibility). Instead, you
collect out the items up to until the point that the collector
returns; in this case, you'd get [0, 1, 2]. This change was made
as part of `PEP 479 <https://www.python.org/dev/peps/pep-0479/>`_.
Regardless, for any Python it's recommended that your generator
live at least as long as the iterator feeding it.
.. _Iter.sorted:
|sink| |warning| ``Iter.sorted(self, key=None, reverse=False) -> "Iter[T]"``
============================================================================
Simple wrapper for the ``sorted`` builtin.
Calling this will read the entire stream before producing
results.
.. code-block:: python
>>> Iter('bac').sorted().collect()
['a', 'b', 'c']
>>> Iter('bac').sorted(reverse=True).collect()
['c', 'b', 'a']
>>> Iter('bac').zip([2, 1, 0]).sorted(key=lambda tup: tup[1]).collect()
[('c', 0), ('a', 1), ('b', 2)]
.. _Iter.reversed:
|sink| |warning| ``Iter.reversed(self) -> "Iter[T]"``
=====================================================
Simple wrapper for the ``reversed`` builtin.
Calling this will read the entire stream before producing
results.
.. code-block:: python
>>> Iter(range(4)).reversed().collect()
[3, 2, 1, 0]
Experiments and Provisional Ideas
#################################
.. _IterDict:
|flux| ``class IterDict(UserDict)``
***********************************
The idea here was to make a custom dict where several of
the standard dict methods return ``Iter`` instances, which can then
be chained. I'm not sure if this will be kept yet.
.. _IterDict.keys:
``IterDict.keys(self) -> "Iter"``
=================================
.. _IterDict.values:
``IterDict.values(self) -> "Iter"``
===================================
.. _IterDict.items:
``IterDict.items(self) -> "Iter"``
==================================
.. _IterDict.update:
``IterDict.update(self, *args, **kwargs) -> "IterDict"``
========================================================
.. _insert_separator:
``insert_separator(iterable: Iterable[Any], glue: Any) -> "Iterable[Any]"``
***************************************************************************
Similar functionality can be obtained with, e.g.,
interleave, as in
.. code-block:: python
>>> result = Iter('caleb').interleave(Iter.repeat('x')).collect()
>>> result == list('cxaxlxexbx')
True
But you'll see a trailing "x" there, which join avoids. join
makes sure to only add the glue separator if another element
has arrived.
It can handle strings without any special considerations, but it doesn't
do any special handling for bytes and bytearrays. For that, rather
look at `concat()`.
Related projects
################
It turns out the idea of chaining iterators is not new. There are many
libraries that offer similar features:
* My fork of a now-missing library: `chained-iterable <https://github.com/cjrh/chained-iterable>`_.
* `https://github.com/olirice/flupy <https://github.com/olirice/flupy>`_
* `https://github.com/ddstte/chiter <https://github.com/ddstte/chiter>`_
* `https://github.com/neverendingqs/pyiterable <https://github.com/neverendingqs/pyiterable>`_
* `https://github.com/alliefitter/iterable_collections <https://github.com/alliefitter/iterable_collections>`_
* `https://github.com/halprin/iterator-chain <https://github.com/halprin/iterator-chain>`_
* `https://github.com/jagill/python-chainz <https://github.com/jagill/python-chainz>`_
* `https://github.com/ZianVW/IterPipe <https://github.com/ZianVW/IterPipe>`_
* `https://github.com/Evelyn-H/iterchain <https://github.com/Evelyn-H/iterchain>`_
* `https://github.com/EntilZha/PyFunctional <https://github.com/EntilZha/PyFunctional>`_
* `https://github.com/dwt/fluent <https://github.com/dwt/fluent>`_
Somewhat related:
* `https://github.com/jreese/aioitertools <https://github.com/jreese/aioitertools>`_
Dev Instructions
################
Setup
*****
.. code-block:: shell
$ python -m venv venv
$ source venv/bin/activate
(venv) $ pip install -e .[dev,test]
Testing
*******
.. code-block:: shell
(venv) $ pytest --cov
Documentation
*************
To regenerate the documentation, file ``README.rst``:
.. code-block:: shell
(venv) $ python regenerate_readme.py -m excitertools.py > README.rst
Releasing
*********
To do a release, we're using `bumpymcbumpface <https://pypi.org/project/bumpymcbumpface/>`_.
Make sure that is set up correctly according to its own documentation. I
like to use `pipx <https://github.com/pipxproject/pipx>`_ to install and
manage these kinds of tools.
.. code-block:: shell
$ bumpymcbumpface --push-git --push-pypi
|
|
-----
|
|
Work is a necessary evil to be avoided.
*Mark Twain*
Raw data
{
"_id": null,
"home_page": "https://github.com/cjrh/excitertools",
"name": "excitertools",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "itertools",
"author": "Caleb Hattingh",
"author_email": "caleb.hattingh@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/57/04/67a2c1d23cb76e9888ab22e38e1d16ae9f02f6548c061d3b70a999b8dc52/excitertools-2024.11.3.tar.gz",
"platform": null,
"description": "\n\n.. image:: https://github.com/cjrh/excitertools/workflows/Python%20application/badge.svg\n :target: https://github.com/cjrh/excitertools/actions\n\n.. image:: https://coveralls.io/repos/github/cjrh/excitertools/badge.svg?branch=master\n :target: https://coveralls.io/github/cjrh/excitertools?branch=master\n\n.. image:: https://img.shields.io/pypi/pyversions/excitertools.svg\n :target: https://pypi.python.org/pypi/excitertools\n\n.. image:: https://img.shields.io/pypi/implementation/excitertools.svg\n :target: https://pypi.python.org/pypi/excitertools\n\n.. image:: https://img.shields.io/github/tag/cjrh/excitertools.svg\n :target: https://github.com/cjrh/excitertools\n\n.. image:: https://img.shields.io/badge/install-pip%20install%20excitertools-ff69b4.svg\n :target: https://img.shields.io/badge/install-pip%20install%20excitertools-ff69b4.svg\n\n.. image:: https://img.shields.io/badge/dependencies-more--itertools-4488ff.svg\n :target: https://more-itertools.readthedocs.io/en/stable/\n\n.. image:: https://img.shields.io/pypi/v/excitertools.svg\n :target: https://img.shields.io/pypi/v/excitertools.svg\n\n.. image:: https://img.shields.io/badge/calver-YYYY.MM.MINOR-22bfda.svg\n :target: http://calver.org/\n\n.. image:: https://img.shields.io/badge/code%20style-black-000000.svg\n :target: https://github.com/ambv/black\n\n\n.. _more-itertools: https://more-itertools.readthedocs.io/en/stable/index.html\n\n.. _excitertools:\n\nexcitertools\n############\n\n``itertools`` reimagined as a `fluent interface <https://en.wikipedia.org/wiki/Fluent_interface>`_.\n\n In software engineering, a fluent interface is an object-oriented API whose design\n relies extensively on method chaining. Its goal is to increase code legibility by\n creating a domain-specific language (DSL). The term was coined in 2005 by Eric\n Evans and Martin Fowler.\n\n `*Wikipedia - Fluent Interface* <https://en.wikipedia.org/wiki/Fluent_interface>`_\n\nNote that nearly all of the ``more-itertools`` extension library is included.\n\nDemo\n****\n\n.. code-block:: python\n\n >>> range(10).map(lambda x: x*7).filter(lambda x: x % 3 == 0).collect()\n [0, 21, 42, 63]\n >>> range(10).map(lambda x: x*7).filter(lambda x: x > 0 and x % 3 == 0).collect()\n [21, 42, 63]\n\nWhen the lines get long, parens can be used to split up each instruction:\n\n.. code-block:: python\n\n >>> (\n ... range(10)\n ... .map(lambda x: x*7)\n ... .filter(lambda x: x % 3 == 0)\n ... .collect()\n ... )\n [0, 21, 42, 63]\n\nWhat's also interesting about that is how lambda's can easily contain these\nprocessing chains, since an entire chain is a single expression. For\nexample:\n\n.. code-block:: python\n\n >>> names = ['caleb', 'james', 'gina']\n >>> Iter(names).map(\n ... lambda name: (\n ... Iter(name)\n ... .map(lambda c: c.upper() if c in 'aeiouy' else c)\n ... .collect(str)\n ... )\n ... ).collect()\n ['cAlEb', 'jAmEs', 'gInA']\n\nSomething I've noticed is that ``reduce`` seems easier to use and reason\nabout with this fluent interface, as compared to the conventional usage\nas a standalone function; also, the operator module makes ``reduce`` quite\nuseful for simple cases:\n\n.. code-block:: python\n\n >>> from operator import add, mul\n >>> (\n ... range(10)\n ... .map(lambda x: x*7)\n ... .filter(lambda x: x > 0 and x % 3 == 0)\n ... .reduce(add)\n ... )\n 126\n >>> (\n ... range(10)\n ... .map(lambda x: x*7)\n ... .filter(lambda x: x > 0 and x % 3 == 0)\n ... .reduce(mul)\n ... )\n 55566\n\n.. contents::\n :depth: 1\n\n\n.. |warning| unicode:: U+26A0\n.. |cool| unicode:: U+2728\n.. |flux| unicode:: U+1F6E0\n.. |source| unicode:: U+1F3A4\n.. |sink| unicode:: U+1F3A7\n.. |inf| unicode:: U+267E\n\n\nHow to Understand the API Documentation\n#######################################\n\nSeveral symbols are used to indicate things about parts of the API:\n\n- |source| This function is a *source*, meaning that it produces data\n that will be processed in an iterator chain.\n- |sink| This function is a *sink*, meaning that it consumes data that\n was processed in an iterator chain.\n- |inf| This function returns an infinite iterable\n- |warning| Warning - pay attention\n- |flux| This API is still in flux, and might be changed or\n removed in the future\n- |cool| Noteworthy; could be especially useful in many situations.\n\nThe API is arranged roughly with the module-level functions first, and\nthereafter the Iter_ class itself. It is the Iter_ class that does\nthe work to allow these iterators to be chained together. However, the\nmodule-level functions are more likely to be used directly and that's\nwhy they're presented first.\n\nThe API includes wrappers for the stdlib *itertools* module, including\nthe \"recipes\" given in the *itertools* docs, as well as wrappers for\nthe iterators from the more-itertools_ 3rd-party package.\n\nModule-level Replacements for Builtins\n######################################\n\nThe following module-level functions, like range_, zip_ and so on, are\nintended to be used as replacements for their homonymous builtins. The\nonly difference between these and the builtin versions is that these\nreturn instances of the Iter_ class. Note that because Iter_ is itself\niterable, it means that the functions here can be used as drop-in\nreplacements.\n\nOnce you have an Iter_ instance, all of its methods become available\nvia function call chaining, so these toplevel functions are really only\na convenience to \"get started\" using the chaining syntax with minimal\nupfront cost in your own code.\n\n.. contents::\n :local:\n\n\n\n.. _range:\n\n\n|source| ``range(*args) -> \"Iter[int]\"``\n****************************************\n\n\nReplacement for the builtin ``range`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\nAll the same calling variations work because this function merely wraps\nthe original function.\n\n.. code-block:: python\n\n >>> range(3).collect()\n [0, 1, 2]\n >>> range(1, 4).collect()\n [1, 2, 3]\n >>> range(1, 6, 2).collect()\n [1, 3, 5]\n >>> range(1, 101, 3).filter(lambda x: x % 7 == 0).collect()\n [7, 28, 49, 70, 91]\n\nThis example multiples, element by element, the series ``[0:5]`` with the\nseries ``[1:6]``. Two things to note: Firstly, Iter.zip_ is used to emit\nthe tuples from each series. Secondly, Iter.starmap_ is used to receive\nthose tuples into separate arguments in the ``lambda``.\n\n.. code-block:: python\n\n >>> range(5).zip(range(1, 6)).starmap(lambda x, y: x * y).collect()\n [0, 2, 6, 12, 20]\n\nWhen written in a single line as above, it can get difficult to follow\nthe chain of logic if there are many processing steps. Parentheses in\nPython allow grouping such that expressions can be spread over multiple\nlines.\n\nThis is the same example as the prior one, but formatted to be spread\nover several lines. This is much clearer:\n\n.. code-block:: python\n\n >>> # Written out differently\n >>> (\n ... range(5)\n ... .zip(range(1, 6))\n ... .starmap(lambda x, y: x * y)\n ... .collect()\n ... )\n [0, 2, 6, 12, 20]\n\nIf you wanted the sum instead, it isn't necessary to do the collection\nat all:\n\n.. code-block:: python\n\n >>> (\n ... range(5)\n ... .zip(range(1, 6))\n ... .starmap(lambda x, y: x * y)\n ... .sum()\n ... )\n 40\n\n\n\n.. _zip:\n\n\n``zip(*iterables: Any) -> \"Iter[Tuple[T, ...]]\"``\n*************************************************\nReplacement for the builtin ``zip`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. _enumerate:\n\n\n``enumerate(iterable) -> \"Iter[Tuple[int, T]]\"``\n************************************************\nReplacement for the builtin ``enumerate`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> import string\n >>> enumerate(string.ascii_lowercase).take(3).collect()\n [(0, 'a'), (1, 'b'), (2, 'c')]\n\n\n\n\n.. _map:\n\n\n``map(func: Union[Callable[..., C], str], iterable) -> \"Iter[C]\"``\n******************************************************************\nReplacement for the builtin ``map`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> result = map(lambda x: (x, ord(x)), 'caleb').dict()\n >>> assert result == {'a': 97, 'b': 98, 'c': 99, 'e': 101, 'l': 108}\n\n >>> result = map('x, ord(x)', 'caleb').dict()\n >>> assert result == {'a': 97, 'b': 98, 'c': 99, 'e': 101, 'l': 108}\n\n\n.. _filter:\n\n\n``filter(function: \"Callable[[Any], bool]\", iterable: \"Iterable[T]\") -> \"Iter[T]\"``\n***********************************************************************************\nReplacement for the builtin ``filter`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> filter(lambda x: x % 3 == 0, range(10)).collect()\n [0, 3, 6, 9]\n\n\n\n\n.. _count:\n\n\n|source| ``count(start=0, step: int = 1) -> \"Iter[int]\"``\n*********************************************************\n\n\nReplacement for the itertools ``count`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> count().take(5).collect()\n [0, 1, 2, 3, 4]\n >>> count(0).take(0).collect()\n []\n >>> count(10).take(0).collect()\n []\n >>> count(10).take(5).collect()\n [10, 11, 12, 13, 14]\n >>> count(1).filter(lambda x: x > 10).take(5).collect()\n [11, 12, 13, 14, 15]\n\n\n\n.. _cycle:\n\n\n``cycle(iterable) -> \"Iter[T]\"``\n********************************\nReplacement for the itertools ``count`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> cycle(range(3)).take(6).collect()\n [0, 1, 2, 0, 1, 2]\n >>> cycle([]).take(6).collect()\n []\n >>> cycle(range(3)).take(0).collect()\n []\n\n\n\n.. _repeat:\n\n\n|source| ``repeat(object: C, times=None) -> \"Iter[C]\"``\n*******************************************************\n\n\nReplacement for the itertools ``count`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> repeat('a').take(3).collect()\n ['a', 'a', 'a']\n >>> repeat([1, 2]).take(3).collect()\n [[1, 2], [1, 2], [1, 2]]\n >>> repeat([1, 2]).take(3).collapse().collect()\n [1, 2, 1, 2, 1, 2]\n >>> repeat([1, 2]).collapse().take(3).collect()\n [1, 2, 1]\n >>> repeat('a', times=3).collect()\n ['a', 'a', 'a']\n\n\n\n\nThis next set of functions return iterators that terminate on the shortest \ninput sequence.\n\n\n\n.. _accumulate:\n\n\n``accumulate(iterable, func=None, *, initial=None)``\n****************************************************\nReplacement for the itertools ``accumulate`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> accumulate([1, 2, 3, 4, 5]).collect()\n [1, 3, 6, 10, 15]\n >>> if sys.version_info >= (3, 8):\n ... output = accumulate([1, 2, 3, 4, 5], initial=100).collect()\n ... assert output == [100, 101, 103, 106, 110, 115]\n >>> accumulate([1, 2, 3, 4, 5], operator.mul).collect()\n [1, 2, 6, 24, 120]\n >>> accumulate([]).collect()\n []\n >>> accumulate('abc').collect()\n ['a', 'ab', 'abc']\n >>> accumulate(b'abc').collect()\n [97, 195, 294]\n >>> accumulate(bytearray(b'abc')).collect()\n [97, 195, 294]\n\n\n\n.. _chain:\n\n\n``chain(*iterables: Iterable[T]) -> \"Iter[T]\"``\n***********************************************\nReplacement for the itertools ``chain`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> chain('ABC', 'DEF').collect()\n ['A', 'B', 'C', 'D', 'E', 'F']\n >>> chain().collect()\n []\n\n\n\n.. _chain_from_iterable:\n\n\n``chain_from_iterable(iterable) -> \"Iter[T]\"``\n**********************************************\nReplacement for the itertools ``chain.from_iterable`` method.\nThis version returns an instance of Iter_ to allow\nfurther iterable chaining.\n\n.. code-block:: python\n\n >>> chain_from_iterable(['ABC', 'DEF']).collect()\n ['A', 'B', 'C', 'D', 'E', 'F']\n >>> chain_from_iterable([]).collect()\n []\n\n\n\n.. _compress:\n\n\n``compress(data, selectors)``\n*****************************\nReplacement for the itertools ``compress`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> compress('ABCDEF', [1, 0, 1, 0, 1, 1]).collect()\n ['A', 'C', 'E', 'F']\n\n\n\n\n.. _dropwhile:\n\n\n``dropwhile(pred, iterable)``\n*****************************\nReplacement for the itertools ``dropwhile`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> dropwhile(lambda x: x < 4, range(6)).collect()\n [4, 5]\n\n\n\n.. _filterfalse:\n\n\n``filterfalse(pred, iterable)``\n*******************************\nReplacement for the itertools ``filterfalse`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> filterfalse(None, [2, 0, 3, None, 4, 0]).collect()\n [0, None, 0]\n\n\n\n.. _groupby:\n\n\n``groupby(iterable, key=None)``\n*******************************\nReplacement for the itertools ``groupby`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\ngroupby_ returns an iterator of a key and \"grouper\" iterable. In the\nexample below, we use Iter.starmap_ to collect each grouper iterable\ninto a list, as this makes it neater for display here in the docstring.\n\n.. code-block:: python\n\n >>> (\n ... groupby(['john', 'jill', 'anne', 'jack'], key=lambda x: x[0])\n ... .starmap(lambda k, g: (k, list(g)))\n ... .collect()\n ... )\n [('j', ['john', 'jill']), ('a', ['anne']), ('j', ['jack'])]\n\n\n\n\n.. _islice:\n\n\n``islice(iterable, *args) -> \"Iter\"``\n*************************************\nReplacement for the itertools ``islice`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> islice('ABCDEFG', 2).collect()\n ['A', 'B']\n >>> islice('ABCDEFG', 2, 4).collect()\n ['C', 'D']\n >>> islice('ABCDEFG', 2, None).collect()\n ['C', 'D', 'E', 'F', 'G']\n >>> islice('ABCDEFG', 0, None, 2).collect()\n ['A', 'C', 'E', 'G']\n\n\n\n.. _starmap:\n\n\n``starmap(func, iterable)``\n***************************\nReplacement for the itertools ``starmap`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> starmap(pow, [(2, 5), (3, 2), (10, 3)]).collect()\n [32, 9, 1000]\n\n\n\n.. _takewhile:\n\n\n``takewhile(pred, iterable)``\n*****************************\nReplacement for the itertools ``takewhile`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> takewhile(lambda x: x < 5, [1, 4, 6, 4, 1]).collect()\n [1, 4]\n\n\n\n.. _tee:\n\n\n``tee(iterable, n=2)``\n**********************\nReplacement for the itertools ``tee`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> a, b = tee(range(5))\n >>> a.collect()\n [0, 1, 2, 3, 4]\n >>> b.sum()\n 10\n\nIt is also possible to operate on the returned iterators in the chain\nbut it gets quite difficult to understand:\n\n.. code-block:: python\n\n >>> tee(range(5)).map(lambda it: it.sum()).collect()\n [10, 10]\n\nIn the example above we passed in range_, but with excitertools_\nit's usually more natural to push data sources further left:\n\n.. code-block:: python\n\n >>> range(5).tee().map(lambda it: it.sum()).collect()\n [10, 10]\n\nPay close attention to the above. The map_ is acting on each of the\ncopied iterators.\n\n\n\n.. _zip_longest:\n\n\n``zip_longest(*iterables, fillvalue=None)``\n*******************************************\nReplacement for the itertools ``zip_longest`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> zip_longest('ABCD', 'xy', fillvalue='-').collect()\n [('A', 'x'), ('B', 'y'), ('C', '-'), ('D', '-')]\n >>> (\n ... zip_longest('ABCD', 'xy', fillvalue='-')\n ... .map(lambda tup: concat(tup, ''))\n ... .collect()\n ... )\n ['Ax', 'By', 'C-', 'D-']\n >>> (\n ... zip_longest('ABCD', 'xy', fillvalue='-')\n ... .starmap(operator.add)\n ... .collect()\n ... )\n ['Ax', 'By', 'C-', 'D-']\n\n\n\n.. _finditer_regex:\n\n\n``finditer_regex(pat: \"re.Pattern[AnyStr]\", s: AnyStr, flags: Union[int, re.RegexFlag] = 0) -> \"Iter[AnyStr]\"``\n***************************************************************************************************************\n\nWrapper for ``re.finditer``. Returns an instance of Iter_ to allow\nchaining.\n\n.. code-block:: python\n\n >>> pat = r\"\\w+\"\n >>> text = \"Well hello there! How ya doin!\"\n >>> finditer_regex(pat, text).map(str.lower).filter(lambda w: 'o' in w).collect()\n ['hello', 'how', 'doin']\n >>> finditer_regex(r\"[A-Za-z']+\", \"A programmer's RegEx test.\").collect()\n ['A', \"programmer's\", 'RegEx', 'test']\n >>> finditer_regex(r\"[A-Za-z']+\", \"\").collect()\n []\n >>> finditer_regex(\"\", \"\").collect()\n ['']\n >>> finditer_regex(\"\", \"\").filter(None).collect()\n []\n\n\n\n.. _splititer_regex:\n\n\n``splititer_regex(pat: \"re.Pattern[AnyStr]\", s: AnyStr, flags: Union[int, re.RegexFlag] = 0) -> \"Iter[AnyStr]\"``\n****************************************************************************************************************\n\nLazy string splitting using regular expressions.\n\nMost of the time you want ``str.split``. Really! That will almost\nalways be fastest. You might think that ``str.split`` is inefficient\nbecause it always has to build a list, but it can do this very, very\nquickly.\n\nThe lazy splitting shown here is more about supporting a particular\nkind of programming model, rather than performance.\n\nSee more discussion `here <https://stackoverflow.com/questions/3862010/is-there-a-generator-version-of-string-split-in-python>`_.\n\n.. code-block:: python\n\n >>> splititer_regex(r\"\\s\", \"A programmer's RegEx test.\").collect()\n ['A', \"programmer's\", 'RegEx', 'test.']\n\nNote that splitting at a single whitespace character will return blanks\nfor each found. This is different to how ``str.split()`` works.\n\n.. code-block:: python\n\n >>> splititer_regex(r\"\\s\", \"aaa bbb \\n ccc\\nddd\\teee\").collect()\n ['aaa', '', '', '', '', 'bbb', '', '', '', '', 'ccc', 'ddd', 'eee']\n\nTo match ``str.split()``, specify a sequence of whitespace as the\nregex pattern.\n\n.. code-block:: python\n\n >>> splititer_regex(r\"\\s+\", \"aaa bbb \\n ccc\\nddd\\teee\").collect()\n ['aaa', 'bbb', 'ccc', 'ddd', 'eee']\n\nCounting the whitespace\n\n.. code-block:: python\n\n >>> from collections import Counter\n >>> splititer_regex(r\"\\s\", \"aaa bbb \\n ccc\\nddd\\teee\").collect(Counter)\n Counter({'': 8, 'aaa': 1, 'bbb': 1, 'ccc': 1, 'ddd': 1, 'eee': 1})\n\nLazy splitting at newlines\n\n.. code-block:: python\n\n >>> splititer_regex(r\"\\n\", \"aaa bbb \\n ccc\\nddd\\teee\").collect()\n ['aaa bbb ', ' ccc', 'ddd\\teee']\n\nA few more examples:\n\n.. code-block:: python\n\n >>> splititer_regex(r\"\", \"aaa\").collect()\n ['', 'a', 'a', 'a', '']\n >>> splititer_regex(r\"\", \"\").collect()\n ['', '']\n >>> splititer_regex(r\"\\s\", \"\").collect()\n ['']\n >>> splititer_regex(r\"a\", \"\").collect()\n ['']\n >>> splititer_regex(r\"\\s\", \"aaa\").collect()\n ['aaa']\n\n\n\n.. _concat:\n\n\n``concat(iterable: Iterable[AnyStr], glue: AnyStr) -> \"AnyStr\"``\n****************************************************************\nConcatenate strings, bytes and bytearrays. It is careful to avoid the\nproblem with single bytes becoming integers, and it looks at the value\nof `glue` to know whether to handle bytes or strings.\n\nThis function can raise ``ValueError`` if called with something\nother than ``bytes``, ``bytearray`` or ``str``.\n\n.. _from_queue:\n\n\n|cool| |source| ``from_queue(q: queue.Queue, timeout=None, sentinel=None) -> \"Iter\"``\n*************************************************************************************\n\n\n\n\nWrap a queue with an iterator interface. This allows it to participate\nin chaining operations. The iterator will block while waiting for\nnew values to appear on the queue. This is useful: it allows you\nto easily and safely pass data between threads or processes, and\nfeed the incoming data into a pipeline.\n\nThe sentinel value, default ``None``, will terminate the iterator.\n\n.. code-block:: python\n\n >>> q = queue.Queue()\n >>> # This line puts stuff onto a queue\n >>> range(10).chain([None]).map(q.put).consume()\n >>> from_queue(q).filter(lambda x: 2 < x < 9).collect()\n [3, 4, 5, 6, 7, 8]\n\nThis can be used in the same way you would normally use a queue, in\nthat it will block while waiting for future input. This makes it\nconvenient to run in a thread and wait for work. Below is a rough\nsketch of how one might cobble together a thread pool using this\nfeature. Note the use of Iter.side_effect_ to call ``task_done()``\non the queue.\n\n.. code-block:: python\n\n import queue\n from threading import Thread\n import logging\n from excitertools import from_queue\n\n logger = logging.getLogger(__name__)\n\n def process_job(job):\n result = ...\n return result\n\n def worker(inputs: Queue, results: Queue):\n (\n from_queue(inputs)\n .side_effect(lambda job: logger.info(f\"Received job {job}\")\n .map(process_job)\n .side_effect(lambda result: logger.info(f\"Got result {job}\")\n .into_queue(results)\n # Note that we only mark the task as done after the result\n # is added to the results queue.\n .side_effect(lambda _: inputs.task_done()\n )\n\n def create_job_pool(n: int) -> Tuple[Queue, Queue, Callable]:\n \"\"\"Returns two queues, and a pool shutdown method. The\n shutdown function can be called to shut down the pool and\n the ``inputs`` queue. Caller is responsible for draining\n the ``results`` queue.\"\"\"\n\n # Independent control of the sizes of the input and output\n # queues is interesting: it lets you decide how to bias\n # backpressure depending on the details of your workload.\n inputs, results = Queue(maxsize=100), Queue(maxsize=3)\n\n kwargs = dict(target=worker, args=(inputs, results), daemon=True)\n threads = repeat(Thread).map(lambda T: T(**kwargs)).take(n).collect()\n\n def shutdown():\n # Shut down each thread\n repeat(None).map(inputs.put).take(n).consume()\n inputs.join()\n Iter(threads).map(lambda t: t.join()).consume()\n\n return inputs, results, shutdown\n\nNow the two queues ``inputs`` and ``results`` can be used in various\nother threads to supply and consume data.\n\n\n\n\nThe ``Iter`` Class\n##################\n\n.. contents::\n :backlinks: entry\n :local:\n\n\n\n.. _Iter:\n\n\n|cool| ``class Iter(Generic[T])``\n*********************************\n\n\nThis class is what allows chaining. Many of the methods in this class\nreturn an instance of Iter_, which allows further chaining. There\nare two exceptions to this: *sources* and *sinks*.\n\nA \"source\" is usually a ``classmethod`` which can be used as an\ninitializer to produce data via an iterable. For example, the Iter.range_\nclassmethod can be used to get a sequence of numbers:\n\n.. code-block:: python\n\n >>> Iter.range(1_000_000).take(3).collect()\n [0, 1, 2]\n\nEven though our range was a million elements, the iterator chaining\ntook only 3 of those elements before collecting.\n\nA \"sink\" is a method that is usually the last component of a processing\nchain and often (but not always!) consumes the entire iterator. In the\nexample above, the call to Iter.collect_ was a sink. Note that we still\ncall it a sink even though it did not consume the entire iterator.\n\nWe're using the term \"source\" to refer to a classmethod of Iter_ that\nproduces data; but, the most typical source is going to be data that\nyou provide. Iter_ can be called with anything that is iterable, including\nsequences, iterators, mappings, sets, generators and so on.\n\nExamples:\n\n.. code-block:: python\n\n List\n >>> Iter([1, 2, 3]).map(lambda x: x * 2).sum()\n 12\n\n Generator\n >>> Iter((1, 2, 3)).map(lambda x: x * 2).sum()\n 12\n >>> def g():\n ... for i in [1, 2, 3]:\n ... yield i\n >>> Iter(g()).map(lambda x: x * 2).sum()\n 12\n\n Iterator\n >>> Iter(iter([1, 2, 3])).map(lambda x: x * 2).sum()\n 12\n\n Dict\n >>> Iter(dict(a=1, b=2)).map(lambda x: x.upper()).collect()\n ['A', 'B']\n >>> d = dict(a=1, b=2, c=3)\n >>> Iter(d.items()).starmap(lambda k, v: v).map(lambda x: x * 2).sum()\n 12\n\nA common error with generators is forgetting to actually evaluate, i.e.,\ncall a generator function. If you do this there's a friendly error\npointing out the mistake:\n\n.. code-block:: python\n\n >>> def mygen(): yield 123\n >>> Iter(mygen).collect()\n Traceback (most recent call last):\n ...\n TypeError: It seems you passed a generator function, but you\n probably intended to pass a generator. Remember to evaluate the\n function to obtain a generator instance:\n \n def mygen():\n yield 123\n \n Iter(mygen) # ERROR - a generator function object is not iterable\n Iter(mygen()) # CORRECT - a generator instance is iterable.\n >>> Iter(mygen()).collect()\n [123]\n\nInstance of Iter_ are resumable. Once an instance it created, it can\nbe partially iterated in successive calls, like the following example\nshows:\n\n.. code-block:: python\n\n >>> it = Iter.range(1_000_000)\n >>> it.take(3).collect()\n [0, 1, 2]\n >>> it.take(4).collect()\n [3, 4, 5, 6]\n >>> # Consume most of the stream, collect the last few\n >>> it.consume(999_990).collect()\n [999997, 999998, 999999]\n\nThis class implements the chaining. However, the module-level functions\nin excitertools_, such as range_, zip_ and so on, also return\ninstances of Iter_, so they allow the chaining to continue. These are\nequivalent:\n\n.. code-block:: python\n\n >>> Iter.range(10).filter(lambda x: x > 7).collect()\n [8, 9]\n >>> range(10).filter(lambda x: x > 7).collect()\n [8, 9]\n\nIt is intended that the module-level functions can act as drop-in\nreplacements for the builtins they wrap:\n\n>>> import builtins\n>>> list(builtins.range(3))\n[0, 1, 2]\n>>> list(range(3)) # This is excitertools.range!\n[0, 1, 2]\n>>> list(Iter.range(3))\n[0, 1, 2]\n\nIn your own code where you might like to use the excitertools_ version of\nrange_ and the other functions, you can just import it and use it to access all the other\ncool stuff:\n\n.. code-block:: python\n\n # mymodule.py\n from excitertools import (\n range,\n map,\n filter,\n reduce,\n repeat,\n count,\n enumerate,\n zip,\n ...\n )\n\n def func(inputs):\n data = (\n map(lambda x: x + 2, inputs)\n .enumerate()\n .filter(lambda x: x[1] > 10)\n ...\n .collect()\n\n )\n\nAlternatively, if you don't want to hide the builtins you can do just\nfine with importing this class only, or even importing the module only:\n\n.. code-block:: python\n\n # mymodule.py - same example as before\n import excitertools\n\n def func(inputs):\n data = (\n excitertools.Iter(inputs)\n .map(lambda x: x + 2, inputs)\n .enumerate()\n .filter(lambda x: x[1] > 10)\n ...\n .collect()\n )\n\n # Do something with data\n\nThere are several valuable additions to the standard *itertools* and\nmore-itertools_ functions. These usually involve sources and sinks,\nwhich are ways of getting data into an iterator pipeline, and then\ngetting results out again. In the majority of documentation examples\nshown here, the Iter.collect_ method is used to collect all the\nremaining data on a stream into a list; but in practice this is not\nuseful because large lists consume memory.\n\nIn practice it is more useful to send iterator data to one of these\ncommon sinks:\n\n- files\n- sockets\n- queues\n- HTTP APIs\n- Cloud storage buckets\n- (Ideas for more to add here?)\n\nIter_ has support for these use-cases, both for reading and for writing.\n\n\n\n.. _Iter.register:\n\n\n``@classmethod Iter.register(cls, *func)``\n==========================================\n\nAdd a new method to Iter_. Sure, you could subclass Iter_ to get\nnew chaining features, but it would be neat to let all existing\nIter_ instance just immediately have the new registered function\navailable.\n\nThe new function must take ``iterable`` as the first parameter.\n\n.. code-block:: python\n\n >>> def up(iterable):\n ... for v in iterable:\n ... yield v.upper()\n >>> Iter.register(up)\n >>> Iter('abc').up().collect()\n ['A', 'B', 'C']\n >>> def poly(iterable, a, b, c):\n ... # Polynomials a.x^2 + b.x + c\n ... for x in iterable:\n ... yield a*x**2 + b*x + c\n >>> Iter.register(poly)\n >>> Iter(range(-5, 5, 1)).poly(1, -5, 6).collect()\n [56, 42, 30, 20, 12, 6, 2, 0, 0, 2]\n\nHere's a math round-trip rollercoaster.\n\n.. code-block:: python\n\n >>> import math\n >>> def log(iterable):\n ... for x in iterable:\n ... yield math.log(x)\n >>> def exp(iterable):\n ... for x in iterable:\n ... yield math.exp(x)\n >>> def rnd(iterable):\n ... for x in iterable:\n ... yield round(x)\n >>> Iter.register(log, exp, rnd)\n >>> Iter(range(5)).exp().log().rnd().collect()\n [0, 1, 2, 3, 4]\n\nThese are silly examples, but hopefully you get the idea.\n\n\n\n.. _Iter.collect:\n\n\n|sink| ``Iter.collect(self, container=list) -> \"List[T]\"``\n==========================================================\n\n\n\nThis is the most common way of \"realizing\" an interable chain\ninto a concrete data structure. It should be the case that this\nis where most of the memory allocation occurs.\n\nThe default container is a list and you'll see throughout this\ndocumentation that most examples produce lists. However,\nany container, and indeed any function, can be used as the sink.\n\nThe basic example:\n\n.. code-block:: python\n\n >>> Iter(range(3)).collect()\n [0, 1, 2]\n >>> Iter(range(3)).collect(tuple)\n (0, 1, 2)\n\nYou must pay attention to some things. For example, if your\niterable is a string, the characters of the string are what\nget iterated over, and when you collect you'll get a collection\nof those atoms. You can however use ``str`` as your \"container\nfunction\" and that will give you back a string. It's like a join\nwith blank joiner.\n\n.. code-block:: python\n\n >>> Iter('abc').collect()\n ['a', 'b', 'c']\n >>> Iter('abc').collect(str)\n 'abc'\n\nWith some types, things get a little more tricky. Take ``bytes``\nfor example:\n\n.. code-block:: python\n\n >>> Iter(b'abc').collect()\n [97, 98, 99]\n\nYou probably didn't expect to get the integers back right? Anyhow,\nyou can use ``bytes`` as the \"collection container\", just like\nwe did with strings and that will work:\n\n.. code-block:: python\n\n >>> Iter(b'abc').collect(bytes)\n b'abc'\n >>> Iter(b'abc').collect(bytearray)\n bytearray(b'abc')\n\nThe other standard collections also work, here's a set for\ncompleteness.\n\n.. code-block:: python\n\n >>> Iter('abcaaaabbbbccc').collect(set) == {'a', 'b', 'c'}\n True\n\n\n\n.. _Iter.open:\n\n\n|cool| |source| ``@classmethod Iter.open(cls, file, mode=\"r\", buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None, ) -> \"Iter\"``\n==============================================================================================================================================================\n\n\n\n\nWrap the ``open()`` builtin precisely, but return an ``Iter``\ninstance to allow function chaining on the result.\n\nI know you're thinking that we should always use a context\nmanager for files. Don't worry, there is one being used\ninternally. When the iterator chain is terminated the underlying\nfile will be closed.\n\n>>> import tempfile\n>>> with tempfile.TemporaryDirectory() as td:\n... # Put some random text into a temporary file\n... with open(td + 'text.txt', 'w') as f:\n... f.writelines(['abc\\n', 'def\\n', 'ghi\\n'])\n...\n... # Open the file, filter some lines, collect the result\n... Iter.open(td + 'text.txt').filter(lambda line: 'def' in line).collect()\n['def\\n']\n\nNote that this is a convenience method for *reading* from a file,\nnot for writing. The function signature includes the ``mode``\nparameter for parity with the builtin ``open()`` function, but\nonly reading is supported.\n\n\n\n.. _Iter.read_lines:\n\n\n|source| ``@classmethod Iter.read_lines(cls, stream: IO[str], rewind=True)``\n============================================================================\n\n\n\nRead lines from a file-like object.\n\nFirst, let's put some data in a file. We'll be using that\nfile in the examples that follow.\n\n.. code-block:: python\n\n >>> import tempfile\n >>> td = tempfile.TemporaryDirectory()\n ... # Put some random text into a temporary file\n >>> with open(td.name + 'text.txt', 'w') as f:\n ... f.writelines(['abc\\n', 'def\\n', 'ghi\\n'])\n ...\n\nUse read_lines to process the file data\n\n.. code-block:: python\n\n >>> with open(td.name + 'text.txt') as f:\n ... Iter.read_lines(f).filter(lambda line: 'def' in line).collect()\n ['def\\n']\n\nThe ``rewind`` parameter can be used to read sections of a file.\n\n.. code-block:: python\n\n >>> with open(td.name + 'text.txt') as f:\n ... part1 = Iter.read_lines(f).take(1).collect()\n ... part2 = Iter.read_lines(f, rewind=False).collect()\n >>> part1\n ['abc\\n']\n >>> part2\n ['def\\n', 'ghi\\n']\n >>> td.cleanup()\n\n\n\n.. _Iter.read_bytes:\n\n\n|source| ``@classmethod Iter.read_bytes(cls, stream: IO[bytes], size: Union[Callable[[], int], int] = -1, rewind=True)``\n========================================================================================================================\n\n\n\nThe ``size`` parameter can be used to control how many bytes are\nread for each advancement of the iterator chain. Here we set ``size=1``\nwhich means we'll get back one byte at a time.\n\n.. code-block:: python\n\n >>> import tempfile\n >>> td = tempfile.TemporaryDirectory()\n >>> filename = td.name + 'bytes.bin'\n\nPut some random text into a temporary file:\n\n.. code-block:: python\n\n >>> with open(filename, 'wb') as f:\n ... x = f.write(b'\\x00' * 100)\n ...\n >>> with open(filename, 'rb') as f:\n ... data = Iter.read_bytes(f, size=1).collect()\n ... len(data)\n 100\n >>> with open(filename, 'rb') as f:\n ... data = Iter.read_bytes(f).collect()\n ... len(data)\n 1\n\nA little more ambitious. Because ``size`` is a callable, we can use\na ``deque`` and a ``side_effect`` to pass information back into\nthe reader to control how many bytes are read in each chunk.\n\nIn this example we're reading 1 byte at a time. In a real example\nyou might have a sequence of headers and bodies, where headers\ngive size information about how many bytes are in the body\ncorresponding to that header. Then you can precisely read\neach body in sequence.\n\n.. code-block:: python\n\n >>> from collections import deque\n >>> read_sizes = deque([1])\n >>> with open(filename, 'rb') as f:\n ... data = (\n ... Iter\n ... .read_bytes(f, size=lambda: read_sizes.popleft())\n ... .side_effect(lambda bytes: read_sizes.append(1))\n ... .collect()\n ... )\n ... len(data)\n 100\n\nThe ``rewind`` parameter can be used to read sections of a file.\n\n.. code-block:: python\n\n >>> with open(filename, 'rb') as f:\n ... part1 = Iter.read_bytes(f, size=10).take(1).collect()\n ... part2 = Iter.read_bytes(f, rewind=False).collect()\n >>> part1\n [b'\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x00']\n >>> len(part2[0])\n 90\n >>> td.cleanup()\n\n\n\n.. _Iter.write_text_to_stream:\n\n\n|sink| ``Iter.write_text_to_stream(self, stream: IO[str], insert_newlines=True, flush=True)``\n=============================================================================================\n\n\n\n.. code-block:: python\n\n >>> import tempfile\n >>> td = tempfile.TemporaryDirectory()\n >>> filename = td.name + 'text.txt'\n\n >>> data = ['a', 'b', 'c']\n >>> with open(filename, 'w') as f:\n ... Iter(data).map(str.upper).write_text_to_stream(f)\n ... with open(filename) as f2:\n ... Iter.read_lines(f2).concat()\n 'A\\nB\\nC'\n\nIf some prior step adds newlines, or more commonly, newlines\noriginate with a data source and are simply carried through the\nprocessing chain unaltered, disable the insertion of newlines:\n\n.. code-block:: python\n\n >>> with open(filename, 'w') as f:\n ... Iter(data).map(str.upper).write_text_to_stream(f, insert_newlines=False)\n ... with open(filename) as f2:\n ... Iter.read_lines(f2).concat()\n 'ABC'\n\nMultiple successive writes may be slowed down by the default\n``flush=True`` parameter. In this case you can delay flushing until\neverything has been written.\n\n.. code-block:: python\n\n >>> with open(filename, 'w') as f:\n ... Iter(data).map(str.upper).write_text_to_stream(f, flush=False)\n ... Iter(data).map(str.upper).write_text_to_stream(f, flush=False)\n ... Iter(data).map(str.upper).write_text_to_stream(f, flush=True)\n ... with open(filename) as f2:\n ... Iter.read_lines(f2).concat()\n 'A\\nB\\nCA\\nB\\nCA\\nB\\nC'\n >>> td.cleanup()\n\n\n\n.. _Iter.write_bytes_to_stream:\n\n\n|sink| ``Iter.write_bytes_to_stream(self, stream: IO[bytes], flush=True)``\n==========================================================================\n\n\n\n.. code-block:: python\n\n >>> import tempfile\n >>> td = tempfile.TemporaryDirectory()\n >>> filename = td.name + 'bytes.bin'\n >>> data = [b'a', b'b', b'c']\n >>> with open(filename, 'wb') as f:\n ... Iter(data).map(lambda x: x * 2 ).write_bytes_to_stream(f)\n ... with open(filename, 'rb') as f2:\n ... Iter.read_bytes(f2).collect()\n [b'aabbcc']\n >>> with open(filename, 'wb') as f:\n ... Iter(data).map(lambda x: x * 2 ).write_bytes_to_stream(f)\n ... with open(filename, 'rb') as f2:\n ... Iter.read_bytes(f2).concat(b'')\n b'aabbcc'\n >>> with open(filename, 'wb') as f:\n ... Iter(data).map(lambda x: x * 2 ).write_bytes_to_stream(f)\n ... with open(filename, 'rb') as f2:\n ... Iter.read_bytes(f2, size=1).collect()\n [b'a', b'a', b'b', b'b', b'c', b'c']\n >>> with open(filename, 'wb') as f:\n ... Iter(data).map(lambda x: x * 2 ).write_bytes_to_stream(f)\n ... with open(filename, 'rb') as f2:\n ... Iter.read_bytes(f2, size=2).map(bytes.decode).collect()\n ['aa', 'bb', 'cc']\n\nFlushing can be delayed if multiple parts are to be written.\n\n.. code-block:: python\n\n >>> with open(filename, 'wb') as f:\n ... it = Iter(data)\n ... it.map(lambda x: x * 2 ).take(2).write_bytes_to_stream(f, flush=False)\n ... it.map(lambda x: x * 2 ).write_bytes_to_stream(f, flush=True)\n ... with open(filename, 'rb') as f2:\n ... Iter.read_bytes(f2, size=2).map(bytes.decode).collect()\n ['aa', 'bb', 'cc']\n >>> td.cleanup()\n\n\n\n.. _Iter.write_to_file:\n\n\n|cool| |sink| ``Iter.write_to_file(self, file, mode=\"w\", buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None, )``\n===============================================================================================================================================\n\n\n\n\n.. code-block:: python\n\n >>> import tempfile\n >>> with tempfile.TemporaryDirectory() as td:\n ... # Put some random text into a temporary file\n ... with open(td + 'text.txt', 'w') as f:\n ... f.writelines(['abc\\n', 'def\\n', 'ghi\\n'])\n ...\n ... # Open the file, transform, write out to new file.\n ... Iter.open(td + 'text.txt').map(str.upper).write_to_file(td + 'test2.txt')\n ... # Read the new file, for the test\n ... Iter.open(td + 'test2.txt').collect()\n ['ABC\\n', 'DEF\\n', 'GHI\\n']\n\n\n\n.. _Iter.range:\n\n\n|source| ``@classmethod Iter.range(cls, *args) -> \"Iter[int]\"``\n===============================================================\n\n\n\nThe ``range`` function you all know and love.\n\n.. code-block:: python\n\n >>> Iter.range(3).collect()\n [0, 1, 2]\n >>> Iter.range(0).collect()\n []\n\n\n\n.. _Iter.zip:\n\n\n``Iter.zip(self, *iterables: Any) -> \"Iter[Tuple[T, ...]]\"``\n============================================================\n\n\nThe ``zip`` function you all know and love. The only thing to\nnote here is that the first iterable is really what the Iter_\ninstance is wrapping. The Iter.zip_ invocation brings in the\nother iterables.\n\nMake an Iter_ instance, then call ``zip`` on that.\n\n.. code-block:: python\n\n >>> Iter('caleb').zip(range(10)).collect()\n [('c', 0), ('a', 1), ('l', 2), ('e', 3), ('b', 4)]\n\nUse a classmethod to get an infinite stream using Iter.count_\nand zip against that with more finite iterators.\n\n.. code-block:: python\n\n >>> Iter.count().zip(range(5), range(3, 100, 2)).collect()\n [(0, 0, 3), (1, 1, 5), (2, 2, 7), (3, 3, 9), (4, 4, 11)]\n\nIt takes a few minutes to get used to that but feels comfortable\npretty quickly.\n\nIter.take_ can be used to stop infinite zip sequences:\n\n.. code-block:: python\n\n >>> Iter('caleb').cycle().enumerate().take(8).collect()\n [(0, 'c'), (1, 'a'), (2, 'l'), (3, 'e'), (4, 'b'), (5, 'c'), (6, 'a'), (7, 'l')]\n\nWhile we're here (assuming you worked through the previous\nexample), note the difference if you switch the order of the\nIter.cycle_ and Iter.enumerate_ calls:\n\n.. code-block:: python\n\n >>> Iter('caleb').enumerate().cycle().take(8).collect()\n [(0, 'c'), (1, 'a'), (2, 'l'), (3, 'e'), (4, 'b'), (0, 'c'), (1, 'a'), (2, 'l')]\n\nIf you understand how this works, everything else in _excitertools_\nwill be intuitive to use.\n\n\n\n.. _Iter.any:\n\n\n|sink| ``Iter.any(self) -> \"bool\"``\n===================================\n\n\n\n.. code-block:: python\n\n >>> Iter([0, 0, 0]).any()\n False\n >>> Iter([0, 0, 1]).any()\n True\n >>> Iter([]).any()\n False\n\n\n\n.. _Iter.all:\n\n\n|sink| ``Iter.all(self) -> \"bool\"``\n===================================\n\n\n\n\n.. code-block:: python\n\n >>> Iter([0, 0, 0]).all()\n False\n >>> Iter([0, 0, 1]).all()\n False\n >>> Iter([1, 1, 1]).all()\n True\n\nNow pay attention:\n\n.. code-block:: python\n\n >>> Iter([]).all()\n True\n\nThis behaviour has some controversy around it, but that's how the\n``all()`` builtin works so that's what we do too. The way to\nthink about what ``all()`` does is this: it returns False if there\nis at least one element that is falsy. Thus, if there are no elements\nit follows that there are no elements that are falsy and that's why\n``all([]) == True``.\n\n\n\n.. _Iter.enumerate:\n\n\n``Iter.enumerate(self) -> \"Iter[Tuple[int, T]]\"``\n=================================================\n\n\n.. code-block:: python\n\n >>> Iter('abc').enumerate().collect()\n [(0, 'a'), (1, 'b'), (2, 'c')]\n >>> Iter([]).enumerate().collect()\n []\n\n\n\n.. _Iter.dict:\n\n\n``Iter.dict(self) -> \"Dict\"``\n=============================\n\nIn regular Python a dict can be constructed through an iterable\nof tuples:\n\n.. code-block:: python\n\n >>> dict([('a', 0), ('b', 1)]) \n {'a': 0, 'b': 1}\n\nIn *excitertools* we prefer chaining so this method is a shortcut\nfor that:\n\n.. code-block:: python\n\n >>> d = Iter('abc').zip(count()).dict()\n >>> assert d == {'a': 0, 'b': 1, 'c': 2}\n\n\n\n.. _Iter.map:\n\n\n``Iter.map(self, func: Union[Callable[..., C], str]) -> \"Iter[C]\"``\n===================================================================\n\nThe ``map`` function you all know and love.\n\n.. code-block:: python\n\n >>> Iter('abc').map(str.upper).collect()\n ['A', 'B', 'C']\n >>> Iter(['abc', 'def']).map(str.upper).collect()\n ['ABC', 'DEF']\n\nUsing lambdas might seem convenient but in practice it turns\nout that they make code difficult to read:\n\n.. code-block:: python\n\n >>> result = Iter('caleb').map(lambda x: (x, ord(x))).dict()\n >>> assert result == {'a': 97, 'b': 98, 'c': 99, 'e': 101, 'l': 108}\n\nIt's recommended that you make a separate function instead:\n\n.. code-block:: python\n\n >>> def f(x):\n ... return x, ord(x)\n >>> result = Iter('caleb').map(f).dict()\n >>> assert result == {'a': 97, 'b': 98, 'c': 99, 'e': 101, 'l': 108}\n\nI know many people prefer anonymous functions (often on\nphilosphical grounds) but in practice it's just easier to make\na separate, named function.\n\nI've experimented with passing a string into the map, and using\n``eval()`` to make a lambda internally. This simplifies the code\nvery slightly, at the cost of using strings-as-code. I'm pretty\nsure this feature will be removed so don't use it.\n\n.. code-block:: python\n\n >>> result = Iter('caleb').map('x, ord(x)').dict()\n >>> assert result == {'a': 97, 'b': 98, 'c': 99, 'e': 101, 'l': 108}\n\n\n\n.. _Iter.filter:\n\n\n``Iter.filter(self, function: \"Optional[Callable[[T], bool]]\" = None) -> \"Iter[T]\"``\n====================================================================================\n\nThe ``map`` function you all know and love.\n\n.. code-block:: python\n\n >>> Iter('caleb').filter(lambda x: x in 'aeiou').collect()\n ['a', 'e']\n\nThere is a slight difference between this method signature and\nthe builtin ``filter``: how the identity function is handled.\nThis is a consquence of chaining. In the function signature above\nit is possible for us to give the ``function`` parameter a\ndefault value of ``None`` because the parameter appears towards\nthe end of the parameter list. Last, in fact. In the\n`builtin filter signature <https://docs.python.org/3/library/functions.html#filter>`_\nit doesn't allow for this because the predicate parameter appears\nfirst.\n\nThis is a long way of saying: if you just want to filter out\nfalsy values, no parameter is needed:\n\n.. code-block:: python\n\n >>> Iter([0, 1, 0, 0, 0, 1, 1, 1, 0, 0]).filter().collect()\n [1, 1, 1, 1]\n\nUsing the builtin, you'd have to do ``filter(None, iterable)``.\n\nYou'll find that Iter.map_ and Iter.filter_\n(and Iter.reduce_, up next) work together very nicely:\n\n.. code-block:: python\n\n >>> def not_eve(x):\n ... return x != 'eve'\n >>> Iter(['bob', 'eve', 'alice']).filter(not_eve).map(str.upper).collect()\n ['BOB', 'ALICE']\n\nThe long chains get unwieldy so let's rewrite that:\n\n.. code-block:: python\n\n >>> (\n ... Iter(['bob', 'eve', 'alice'])\n ... .filter(not_eve)\n ... .map(str.upper)\n ... .collect()\n ... )\n ['BOB', 'ALICE']\n\n\n\n.. _Iter.starfilter:\n\n\n|cool| ``Iter.starfilter(self, function: \"Optional[Callable[[T, ...], bool]]\" = None) -> \"Iter[T]\"``\n====================================================================================================\n\n\nLike Iter.filter_, but arg unpacking in lambdas will work.\n\nWith the normal ``filter``, this fails:\n\n.. code-block:: python\n\n >>> Iter('caleb').enumerate().filter(lambda i, x: i > 2).collect()\n Traceback (most recent call last):\n ...\n TypeError: <lambda>() missing 1 required positional argument: 'x'\n\nThis is a real buzzkill. ``starfilter`` is very similar to\n``starmap`` in that tuples are unpacked when calling the function:\n\n.. code-block:: python\n\n >>> Iter('caleb').enumerate().starfilter(lambda i, x: i > 2).collect()\n [(3, 'e'), (4, 'b')]\n\n\n\n.. _Iter.filter_gt:\n\n\n``Iter.filter_gt(self, value) -> \"Iter[T]\"``\n============================================\n\nConvenience method\n\n.. code-block:: python\n\n >>> Iter([1,2,3]).filter_gt(1).collect()\n [2, 3]\n\n\n\n.. _Iter.filter_ge:\n\n\n``Iter.filter_ge(self, value) -> \"Iter[T]\"``\n============================================\n\nConvenience method\n\n.. code-block:: python\n\n >>> Iter([1,2,3]).filter_ge(2).collect()\n [2, 3]\n\n\n\n.. _Iter.filter_lt:\n\n\n``Iter.filter_lt(self, value) -> \"Iter[T]\"``\n============================================\n\nConvenience method\n\n.. code-block:: python\n\n >>> Iter([1,2,3]).filter_lt(3).collect()\n [1, 2]\n\n\n.. _Iter.filter_le:\n\n\n``Iter.filter_le(self, value) -> \"Iter[T]\"``\n============================================\n\nConvenience method\n\n.. code-block:: python\n\n >>> Iter([1,2,3]).filter_le(2).collect()\n [1, 2]\n\n\n.. _Iter.filter_eq:\n\n\n``Iter.filter_eq(self, value) -> \"Iter[T]\"``\n============================================\n\nConvenience method\n\n.. code-block:: python\n\n >>> Iter([1,2,3]).filter_eq(2).collect()\n [2]\n\n\n.. _Iter.filter_ne:\n\n\n``Iter.filter_ne(self, value) -> \"Iter[T]\"``\n============================================\n\nConvenience method\n\n.. code-block:: python\n\n >>> Iter([1,2,3]).filter_ne(2).collect()\n [1, 3]\n\n\n.. _Iter.filter_in:\n\n\n``Iter.filter_in(self, value: Sized) -> \"Iter[T]\"``\n===================================================\n\nConvenience method for membership testing. Note that the value\nparameter must be at least ``Sized`` because it gets reused\nover and over for each pass of the iterator chain. For example,\npassing in things like ``range()`` will not work properly because\nit will become progressively exhausted.\n\n.. code-block:: python\n\n >>> Iter([1,2,3]).filter_in([2, 3, 4, 5]).collect()\n [2, 3]\n >>> Iter([1,2,3]).filter_in(range(2, 8).collect()).collect()\n [2, 3]\n >>> Iter([1,2,3]).filter_in({2, 3, 4, 5}).collect()\n [2, 3]\n >>> Iter([1,2,3]).filter_in(dict.fromkeys({2, 3, 4, 5})).collect()\n [2, 3]\n\n\n.. _Iter.filter_ni:\n\n\n``Iter.filter_ni(self, value) -> \"Iter[T]\"``\n============================================\n\nConvenience method for membership testing. Note that the value\nparameter must be at least ``Sized`` because it gets reused\nover and over for each pass of the iterator chain. For example,\npassing in things like ``range()`` will not work properly because\nit will become progressively exhausted.\n\n.. code-block:: python\n\n >>> Iter([1,2,3]).filter_ni([2, 3, 4, 5]).collect()\n [1]\n >>> Iter([1,2,3]).filter_ni(range(2, 8).collect()).collect()\n [1]\n >>> Iter([1,2,3]).filter_ni({2, 3, 4, 5}).collect()\n [1]\n >>> Iter([1,2,3]).filter_ni(dict.fromkeys({2, 3, 4, 5})).collect()\n [1]\n\n\n.. _Iter.reduce:\n\n\n|sink| ``Iter.reduce(self, func: Callable[..., T], *args) -> \"T\"``\n==================================================================\n\n\nThe ``reduce`` function you all know and...hang on, actually\n``reduce`` is rather unloved. In the past I've found it very complex\nto reason about, when looking at a bunch of nested function calls\nin typical ``itertools`` code. Hopefully iterable chaining makes\nit easier to read code that uses ``reduce``?\n\nLet's check, does this make sense?\n\n.. code-block:: python\n\n >>> payments = [\n ... ('bob', 100),\n ... ('alice', 50),\n ... ('eve', -100),\n ... ('bob', 19.95),\n ... ('bob', -5.50),\n ... ('eve', 11.95),\n ... ('eve', 200),\n ... ('alice', -45),\n ... ('alice', -67),\n ... ('bob', 1.99),\n ... ('alice', 89),\n ... ]\n >>> (\n ... Iter(payments)\n ... .filter(lambda entry: entry[0] == 'bob')\n ... .map(lambda entry: entry[1])\n ... .reduce(lambda total, value: total + value, 0)\n ... )\n 116.44\n\nI intentionally omitted comments above so that you can try the\n\"readability experiment\", but in practice you would definitely\nwant to add some comments on these chains:\n\n.. code-block:: python\n\n >>> (\n ... # Iterate over all payments\n ... Iter(payments)\n ... # Only look at bob's payments\n ... .filter(lambda entry: entry[0] == 'bob')\n ... # Extract the value of the payment\n ... .map(lambda entry: entry[1])\n ... # Add all those payments together\n ... .reduce(lambda total, value: total + value, 0)\n ... )\n 116.44\n\n``reduce`` is a quite crude low-level tool. In many cases you'll\nfind that there are other functions and methods better suited\nto the situations you'll encounter most often. For example,\nthere is already Iter.sum_ if you just want to add up numbers,\nand it's much easier to use Iter.groupby_ for grouping than\nto try to make that work with Iter.reduce_. You *can* make it\nwork but it'll be easier to use Iter.groupby_.\n\n\n\n.. _Iter.starreduce:\n\n\n|sink| ``Iter.starreduce(self, function: Callable[..., T], initializer=0) -> \"T\"``\n==================================================================================\n\n\nIter.starreduce_ is the same as Iter.reduce_ except that args are\nstar-unpacked when passed into ``function``. This is frequently\nmore convenient than the default behaviour.\n\nWe can see this using the same example shown for Iter.reduce_.\nThe star unpacking makes it easier to just do the filtering\ndirectly inside the reducer function.\n\n.. code-block:: python\n\n >>> payments = [\n ... ('bob', 100),\n ... ('alice', 50),\n ... ('eve', -100),\n ... ('bob', 19.95),\n ... ('bob', -5.50),\n ... ('eve', 11.95),\n ... ('eve', 200),\n ... ('alice', -45),\n ... ('alice', -67),\n ... ('bob', 1.99),\n ... ('alice', 89),\n ... ]\n >>> (\n ... Iter(payments)\n ... .starreduce(\n ... lambda tot, name, value: tot + value if name == 'bob' else tot,\n ... 0\n ... )\n ... )\n 116.44\n\nThis is how that looks if you avoid a lambda:\n\n.. code-block:: python\n\n >>> def f(tot, name, value):\n ... if name == 'bob':\n ... return tot + value\n ... else:\n ... return tot\n >>> Iter(payments).starreduce(f)\n 116.44\n\n\n\n.. _Iter.sum:\n\n\n|sink| ``Iter.sum(self)``\n=========================\n\n\nExactly what you expect:\n\n.. code-block:: python\n\n >>> Iter(range(10)).sum()\n 45\n\n\n\n.. _Iter.concat:\n\n\n|sink| ``Iter.concat(self, glue: AnyStr = \"\") -> \"AnyStr\"``\n===========================================================\n\n\n\nJoining strings (and bytes).\n\n.. code-block:: python\n\n >>> Iter(['hello', 'there']).concat()\n 'hellothere'\n >>> Iter(['hello', 'there']).concat(' ')\n 'hello there'\n >>> Iter(['hello', 'there']).concat(',')\n 'hello,there'\n >>> Iter([b'hello', b'there']).concat(b',')\n b'hello,there'\n\n\n\n.. _Iter.insert:\n\n\n``Iter.insert(self, glue: C) -> \"Iter[Union[C, T]]\"``\n=====================================================\nDocstring TBD\n\n\n.. _Iter.count:\n\n\n|source| ``@classmethod Iter.count(cls, *args) -> \"Iter[int]\"``\n===============================================================\n\n\n\n.. code-block:: python\n\n >>> Iter.count().take(3).collect()\n [0, 1, 2]\n >>> Iter.count(100).take(3).collect()\n [100, 101, 102]\n >>> Iter.count(100, 2).take(3).collect()\n [100, 102, 104]\n\n\n\n.. _Iter.cycle:\n\n\n|inf| ``Iter.cycle(self) -> \"Iter[T]\"``\n=======================================\n\n\n\n.. code-block:: python\n\n >>> Iter('abc').cycle().take(8).collect()\n ['a', 'b', 'c', 'a', 'b', 'c', 'a', 'b']\n >>> Iter('abc').cycle().take(8).concat('')\n 'abcabcab'\n\n\n\n.. _Iter.repeat:\n\n\n|source| |inf| ``@classmethod Iter.repeat(cls, elem: C, times=None) -> \"Iter[C]\"``\n==================================================================================\n\n\n\n\n.. code-block:: python\n\n >>> Iter.repeat('c', times=3).collect()\n ['c', 'c', 'c']\n\n\n\n.. _Iter.accumulate:\n\n\n``Iter.accumulate(self, func=None, *, initial=None)``\n=====================================================\nDocstring TBD\n\n.. code-block:: python\n\n >>> Iter([1, 2, 3, 4, 5]).accumulate().collect()\n [1, 3, 6, 10, 15]\n >>> if sys.version_info >= (3, 8):\n ... out = Iter([1, 2, 3, 4, 5]).accumulate(initial=100).collect()\n ... assert out == [100, 101, 103, 106, 110, 115]\n >>> Iter([1, 2, 3, 4, 5]).accumulate(operator.mul).collect()\n [1, 2, 6, 24, 120]\n\n\n\n.. _Iter.chain:\n\n\n``Iter.chain(self, *iterables: Iterable[T]) -> \"Iter[T]\"``\n==========================================================\nDocstring TBD\n\n.. code-block:: python\n\n >>> Iter('ABC').chain('DEF').collect()\n ['A', 'B', 'C', 'D', 'E', 'F']\n >>> Iter('ABC').chain().collect()\n ['A', 'B', 'C']\n\n\n\n.. _Iter.chain_from_iterable:\n\n\n``Iter.chain_from_iterable(self) -> \"Iter[T]\"``\n===============================================\nDocstring TBD\n\n.. code-block:: python\n\n >>> Iter(['ABC', 'DEF']).chain_from_iterable().collect()\n ['A', 'B', 'C', 'D', 'E', 'F']\n\n\n\n.. _Iter.compress:\n\n\n``Iter.compress(self, selectors)``\n==================================\nReplacement for the itertools ``compress`` function. This version returns\nan instance of Iter_ to allow further iterable chaining.\n\n.. code-block:: python\n\n >>> Iter('ABCDEF').compress([1, 0, 1, 0, 1, 1]).collect()\n ['A', 'C', 'E', 'F']\n\n\n\n.. _Iter.dropwhile:\n\n\n``Iter.dropwhile(self, pred)``\n==============================\nDocstring TBD\n\n\n.. _Iter.filterfalse:\n\n\n``Iter.filterfalse(self, pred)``\n================================\nDocstring TBD\n\n\n.. _Iter.groupby:\n\n\n``Iter.groupby(self, key=None)``\n================================\nDocstring TBD\n\n\n.. _Iter.islice:\n\n\n``Iter.islice(self, *args) -> \"Iter\"``\n======================================\nDocstring TBD\n\n\n.. _Iter.starmap:\n\n\n``Iter.starmap(self, func)``\n============================\nDocstring TBD\n\n\n.. _Iter.takewhile:\n\n\n``Iter.takewhile(self, pred)``\n==============================\nDocstring TBD\n\n\n.. _Iter.tee:\n\n\n``Iter.tee(self, n=2)``\n=======================\nDocstring TBD\n\n\n.. _Iter.zip_longest:\n\n\n``Iter.zip_longest(self, *iterables, fillvalue=None)``\n======================================================\nDocstring TBD\n\n\n.. _Iter.chunked:\n\n\n``Iter.chunked(self, n: int) -> \"Iter\"``\n========================================\nDocstring TBD\n\n\n.. _Iter.ichunked:\n\n\n``Iter.ichunked(self, n: int) -> \"Iter\"``\n=========================================\nDocstring TBD\n\n\n.. _Iter.sliced:\n\n\n``@classmethod Iter.sliced(cls, seq: Sequence, n: int) -> \"Iter\"``\n==================================================================\nDocstring TBD\n\n\n.. _Iter.distribute:\n\n\n``Iter.distribute(self, n: int) -> \"Iter\"``\n===========================================\nDocstring TBD\n\n\n.. _Iter.divide:\n\n\n``Iter.divide(self, n: int) -> \"Iter\"``\n=======================================\nDocstring TBD\n\n\n.. _Iter.split_at:\n\n\n``Iter.split_at(self, pred)``\n=============================\nDocstring TBD\n\n\n.. _Iter.split_before:\n\n\n``Iter.split_before(self, pred)``\n=================================\nDocstring TBD\n\n\n.. _Iter.split_after:\n\n\n``Iter.split_after(self, pred)``\n================================\nDocstring TBD\n\n\n.. _Iter.split_into:\n\n\n``Iter.split_into(self, sizes)``\n================================\nDocstring TBD\n\n\n.. _Iter.split_when:\n\n\n``Iter.split_when(self, pred)``\n===============================\nDocstring TBD\n\n\n.. _Iter.bucket:\n\n\n``Iter.bucket(self, key, validator=None)``\n==========================================\n\nThis is the basic example, copied from the more-itertools\ndocs:\n\n.. code-block:: python\n\n >>> iterable = ['a1', 'b1', 'c1', 'a2', 'b2', 'c2', 'b3']\n >>> b = Iter(iterable).bucket(key=lambda x: x[0])\n >>> sorted(b)\n ['a', 'b', 'c']\n >>> list(b['a'])\n ['a1', 'a2']\n\nNote that once consumed, you can't iterate over the contents\nof a group again.\n\n\n.. _Iter.unzip:\n\n\n``Iter.unzip(self)``\n====================\nDocstring TBD\n\n\n.. _Iter.grouper:\n\n\n``Iter.grouper(self, n: int, fillvalue=None) -> \"Iter\"``\n========================================================\nDocstring TBD\n\n\n.. _Iter.partition:\n\n\n``Iter.partition(self, pred) -> \"Iter\"``\n========================================\nDocstring TBD\n\n\n.. _Iter.spy:\n\n\n``Iter.spy(self, n=1) -> \"Tuple[Iter, Iter]\"``\n==============================================\nDocstring TBD\n\n\n.. _Iter.peekable:\n\n\n``Iter.peekable(self) -> \"more_itertools.peekable\"``\n====================================================\n\nDocstring TBD\n\n.. code-block:: python\n\n >>> p = Iter(['a', 'b']).peekable()\n >>> p.peek()\n 'a'\n >>> next(p)\n 'a'\n\nThe peekable can be used to inspect what will be coming up.\nBut if you then want to resume iterator chaining, pass the\npeekable back into an Iter_ instance.\n\n.. code-block:: python\n\n >>> p = Iter(range(10)).peekable()\n >>> p.peek()\n 0\n >>> Iter(p).take(3).collect()\n [0, 1, 2]\n\nA peekable is not an Iter_ instance so it doesn't provide\nthe iterator chaining methods. But if you want to get into\nchaining, use the ``iter()`` method.\n\n.. code-block:: python\n\n >>> p = Iter(range(5)).peekable()\n >>> p.peek()\n 0\n >>> p[1]\n 1\n >>> p.iter().take(3).collect()\n [0, 1, 2]\n\nPeekables can be prepended. But then you usually want to go\nright back to iterator chaining. Thus, the ``prepend`` method\n(on the returned ``peekable`` instance) returns an Iter_ instance.\n\n.. code-block:: python\n\n >>> p = Iter(range(3)).peekable()\n >>> p.peek()\n 0\n >>> p.prepend('a', 'b').take(4).collect()\n ['a', 'b', 0, 1]\n\n\n\n.. _Iter.seekable:\n\n\n``Iter.seekable(self) -> \"more_itertools.seekable\"``\n====================================================\nDocstring TBD\n\n\n.. _Iter.windowed:\n\n\n``Iter.windowed(self, n, fillvalue=None, step=1) -> \"Iter\"``\n============================================================\nDocstring TBD\n\n\n.. _Iter.substrings:\n\n\n``Iter.substrings(self)``\n=========================\nDocstring TBD\n\n\n.. _Iter.substrings_indexes:\n\n\n``Iter.substrings_indexes(self, reverse=False)``\n================================================\nDocstring TBD\n\n\n.. _Iter.stagger:\n\n\n``Iter.stagger(self, offsets=(-1, 0, 1), longest=False, fillvalue=None)``\n=========================================================================\n\n.. code-block:: python\n\n >>> Iter([0, 1, 2, 3]).stagger().collect()\n [(None, 0, 1), (0, 1, 2), (1, 2, 3)]\n >>> Iter(range(8)).stagger(offsets=(0, 2, 4)).collect()\n [(0, 2, 4), (1, 3, 5), (2, 4, 6), (3, 5, 7)]\n >>> Iter([0, 1, 2, 3]).stagger(longest=True).collect()\n [(None, 0, 1), (0, 1, 2), (1, 2, 3), (2, 3, None), (3, None, None)]\n\n\n\n.. _Iter.pairwise:\n\n\n``Iter.pairwise(self)``\n=======================\n\nReference `more_itertools.pairwise <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.pairwise>`_\n\n.. code-block:: python\n\n >>> Iter.count().pairwise().take(4).collect()\n [(0, 1), (1, 2), (2, 3), (3, 4)]\n\n\n.. _Iter.count_cycle:\n\n\n``Iter.count_cycle(self, n=None) -> \"Iter\"``\n============================================\n\n\nReference: `more_itertools.count_cycle <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.count_cycle>`_\n\n.. code-block:: python\n\n >>> Iter('AB').count_cycle(3).collect()\n [(0, 'A'), (0, 'B'), (1, 'A'), (1, 'B'), (2, 'A'), (2, 'B')]\n\n\n\n.. _Iter.intersperse:\n\n\n``Iter.intersperse(self, e, n=1) -> \"Iter\"``\n============================================\n\nReference: `more_itertools.intersperse <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.intersperse>`_\n\n.. code-block:: python\n\n >>> Iter([1, 2, 3, 4, 5]).intersperse('!').collect()\n [1, '!', 2, '!', 3, '!', 4, '!', 5]\n\n >>> Iter([1, 2, 3, 4, 5]).intersperse(None, n=2).collect()\n [1, 2, None, 3, 4, None, 5]\n\n\n\n.. _Iter.padded:\n\n\n``Iter.padded(self, fillvalue: Optional[C] = None, n: Optional[int] = None, next_multiple: bool = False, ) -> \"Iter[Union[T, C]]\"``\n===================================================================================================================================\n\nReference: `more_itertools.padded <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.padded>`_\n\n.. code-block:: python\n\n >>> Iter([1, 2, 3]).padded('?', 5).collect()\n [1, 2, 3, '?', '?']\n\n >>> Iter([1, 2, 3, 4]).padded(n=3, next_multiple=True).collect()\n [1, 2, 3, 4, None, None]\n\n\n\n.. _Iter.repeat_last:\n\n\n``Iter.repeat_last(self, default=None) -> \"Iter[T]\"``\n=====================================================\n\nReference: `more_itertools.repeat_last <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.repeat_last>`_\n\n.. code-block:: python\n\n >>> Iter(range(3)).repeat_last().islice(5).collect()\n [0, 1, 2, 2, 2]\n\n >>> Iter(range(0)).repeat_last(42).islice(5).collect()\n [42, 42, 42, 42, 42]\n\n\n\n.. _Iter.adjacent:\n\n\n``Iter.adjacent(self, pred, distance=1) -> \"Iter[Tuple[bool, T]]\"``\n===================================================================\n\nReference: `more_itertools.adjacent <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.adjacent>`_\n\n.. code-block:: python\n\n >>> Iter(range(6)).adjacent(lambda x: x == 3).collect()\n [(False, 0), (False, 1), (True, 2), (True, 3), (True, 4), (False, 5)]\n\n >>> Iter(range(6)).adjacent(lambda x: x == 3, distance=2).collect()\n [(False, 0), (True, 1), (True, 2), (True, 3), (True, 4), (True, 5)]\n\n\n\n\n.. _Iter.groupby_transform:\n\n\n``Iter.groupby_transform(self, keyfunc: Optional[Callable[..., K]] = None, valuefunc: Optional[Callable[..., V]] = None, ) -> \"Iter[Tuple[K, Iterable[V]]]\"``\n=============================================================================================================================================================\n\nReference: `more_itertools.groupby_transform <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.groupby_transform>`_\n\nThis example has been modified somewhat from the original. We're using\n``starmap`` here to \"unzip\" the tuples produced by the group\ntransform.\n\n.. code-block:: python\n\n >>> iterable = 'AaaABbBCcA'\n >>> keyfunc = lambda x: x.upper()\n >>> valuefunc = lambda x: x.lower()\n >>> (\n ... Iter(iterable)\n ... .groupby_transform(keyfunc, valuefunc)\n ... .starmap(lambda k, g: (k, ''.join(g)))\n ... .collect()\n ... )\n [('A', 'aaaa'), ('B', 'bbb'), ('C', 'cc'), ('A', 'a')]\n\n >>> from operator import itemgetter\n >>> keys = [0, 0, 1, 1, 1, 2, 2, 2, 3]\n >>> values = 'abcdefghi'\n >>> iterable = zip(keys, values)\n >>> (\n ... Iter(iterable)\n ... .groupby_transform(itemgetter(0), itemgetter(1))\n ... .starmap(lambda k, g: (k, ''.join(g)))\n ... .collect()\n ... )\n [(0, 'ab'), (1, 'cde'), (2, 'fgh'), (3, 'i')]\n\n\n\n.. _Iter.padnone:\n\n\n``Iter.padnone(self) -> \"Iter[Union[T, None]]\"``\n================================================\n\nReference: `more_itertools.padnone <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.padnone>`_\n\n.. code-block:: python\n\n >>> Iter(range(3)).padnone().take(5).collect()\n [0, 1, 2, None, None]\n\n\n\n.. _Iter.ncycles:\n\n\n``Iter.ncycles(self, n) -> \"Iter[T]\"``\n======================================\n\nReference: `more_itertools.ncycles <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.ncycles>`_\n\n.. code-block:: python\n\n >>> Iter(['a', 'b']).ncycles(3).collect()\n ['a', 'b', 'a', 'b', 'a', 'b']\n\n\n\n.. _Iter.collapse:\n\n\n``Iter.collapse(self, base_type=None, levels=None) -> \"Iter\"``\n==============================================================\n\nReference: `more_itertools.collapse <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.collapse>`_\n\n.. code-block:: python\n\n >>> iterable = [(1, 2), ([3, 4], [[5], [6]])]\n >>> Iter(iterable).collapse().collect()\n [1, 2, 3, 4, 5, 6]\n\n >>> iterable = ['ab', ('cd', 'ef'), ['gh', 'ij']]\n >>> Iter(iterable).collapse(base_type=tuple).collect()\n ['ab', ('cd', 'ef'), 'gh', 'ij']\n\n >>> iterable = [('a', ['b']), ('c', ['d'])]\n >>> Iter(iterable).collapse().collect() # Fully flattened\n ['a', 'b', 'c', 'd']\n >>> Iter(iterable).collapse(levels=1).collect() # Only one level flattened\n ['a', ['b'], 'c', ['d']]\n\n\n\n.. _Iter.sort_together:\n\n\n``@class_or_instancemethod Iter.sort_together(self_or_cls, iterables, key_list=(0,), reverse=False)``\n=====================================================================================================\n\nReference: `more_itertools.sort_together <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.sort_together>`_\n\nThis can be called either as an instance method or a class method.\nThe classmethod form is more convenient if all the iterables are\nalready available. The instancemethod form is more convenient if\none of the iterables already goes through some transformation.\n\nHere are examples from the classmethod form, which mirror the\nexamples in the more-itertools_ documentation:\n\n.. code-block:: python\n\n >>> iterables = [(4, 3, 2, 1), ('a', 'b', 'c', 'd')]\n >>> Iter.sort_together(iterables).collect()\n [(1, 2, 3, 4), ('d', 'c', 'b', 'a')]\n\n >>> iterables = [(3, 1, 2), (0, 1, 0), ('c', 'b', 'a')]\n >>> Iter.sort_together(iterables, key_list=(1, 2)).collect()\n [(2, 3, 1), (0, 0, 1), ('a', 'c', 'b')]\n\n >>> Iter.sort_together([(1, 2, 3), ('c', 'b', 'a')], reverse=True).collect()\n [(3, 2, 1), ('a', 'b', 'c')]\n\nHere is an examples using the instancemethod form:\n\n.. code-block:: python\n\n >>> iterables = [('a', 'b', 'c', 'd')]\n >>> Iter([4, 3, 2, 1]).sort_together(iterables).collect()\n [(1, 2, 3, 4), ('d', 'c', 'b', 'a')]\n\n\n\n.. _Iter.interleave:\n\n\n``@class_or_instancemethod Iter.interleave(self_or_cls, *iterables) -> \"Iter\"``\n===============================================================================\n\nReference: `more_itertools.interleave <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.interleave>`_\n\nClassmethod form:\n\n.. code-block:: python\n\n >>> Iter.interleave([1, 2, 3], [4, 5], [6, 7, 8]).collect()\n [1, 4, 6, 2, 5, 7]\n\nInstancemethod form:\n\n.. code-block:: python\n\n >>> Iter([1, 2, 3]).interleave([4, 5], [6, 7, 8]).collect()\n [1, 4, 6, 2, 5, 7]\n\n\n\n.. _Iter.interleave_longest:\n\n\n``@class_or_instancemethod Iter.interleave_longest(self_or_cls, *iterables) -> \"Iter\"``\n=======================================================================================\n\nReference: `more_itertools.interleave_longest <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.interleave_longest>`_\n\nClassmethod form:\n\n.. code-block:: python\n\n >>> Iter.interleave_longest([1, 2, 3], [4, 5], [6, 7, 8]).collect()\n [1, 4, 6, 2, 5, 7, 3, 8]\n\nInstancemethod form:\n\n.. code-block:: python\n\n >>> Iter([1, 2, 3]).interleave_longest([4, 5], [6, 7, 8]).collect()\n [1, 4, 6, 2, 5, 7, 3, 8]\n\n\n\n.. _Iter.zip_offset:\n\n\n``@classmethod Iter.zip_offset(cls, *iterables, offsets, longest=False, fillvalue=None) -> \"Iter\"``\n===================================================================================================\n\nReference: `more_itertools.zip_offset <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.zip_offset>`_\n\n.. code-block:: python\n\n >>> Iter.zip_offset('0123', 'abcdef', offsets=(0, 1)).collect()\n [('0', 'b'), ('1', 'c'), ('2', 'd'), ('3', 'e')]\n\n >>> Iter.zip_offset('0123', 'abcdef', offsets=(0, 1), longest=True).collect()\n [('0', 'b'), ('1', 'c'), ('2', 'd'), ('3', 'e'), (None, 'f')]\n\n\n.. _Iter.dotproduct:\n\n\n``Iter.dotproduct(self, vec2: Iterable)``\n=========================================\n\nReference: `more_itertools.dotproduct <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.dotproduct>`_\n\n.. code-block:: python\n\n >>> Iter([10, 10]).dotproduct([20, 20])\n 400\n\n\n.. _Iter.flatten:\n\n\n``Iter.flatten(self) -> \"Iter[T]\"``\n===================================\n\nReference: `more_itertools.flatten <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.flatten>`_\n\n.. code-block:: python\n\n >>> Iter([[0, 1], [2, 3]]).flatten().collect()\n [0, 1, 2, 3]\n\n\n\n.. _Iter.roundrobin:\n\n\n``@class_or_instancemethod Iter.roundrobin(self_or_cls: Union[Type[T], T], *iterables: C) -> \"Iter[Union[T, C]]\"``\n==================================================================================================================\n\nReference: `more_itertools.roundrobin <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.roundrobin>`_\n\nClassmethod form:\n\n.. code-block:: python\n\n >>> Iter.roundrobin('ABC', 'D', 'EF').collect()\n ['A', 'D', 'E', 'B', 'F', 'C']\n\nInstancemethod form:\n\n.. code-block:: python\n\n >>> Iter('ABC').roundrobin('D', 'EF').collect()\n ['A', 'D', 'E', 'B', 'F', 'C']\n\n\n\n.. _Iter.prepend:\n\n\n``Iter.prepend(self, value: C) -> \"Iter[Union[T, C]]\"``\n=======================================================\n\nReference: `more_itertools.prepend <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.prepend>`_\n\n.. code-block:: python\n\n >>> value = '0'\n >>> iterator = ['1', '2', '3']\n >>> Iter(iterator).prepend(value).collect()\n ['0', '1', '2', '3']\n\n\n\n.. _Iter.ilen:\n\n\n|sink| ``Iter.ilen(self) -> \"int\"``\n===================================\n\n\n\nReference: `more_itertools.ilen <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.ilen>`_\n\n.. code-block:: python\n\n >>> Iter(x for x in range(1000000) if x % 3 == 0).ilen()\n 333334\n\n\n\n.. _Iter.unique_to_each:\n\n\n``Iter.unique_to_each(self) -> \"Iter[T]\"``\n==========================================\n\nReference: `more_itertools.unique_to_each <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.unique_to_each>`_\n\n.. code-block:: python\n\n >>> Iter([{'A', 'B'}, {'B', 'C'}, {'B', 'D'}]).unique_to_each().collect()\n [['A'], ['C'], ['D']]\n\n >>> Iter([\"mississippi\", \"missouri\"]).unique_to_each().collect()\n [['p', 'p'], ['o', 'u', 'r']]\n\n\n.. _Iter.sample:\n\n\n``Iter.sample(self, k=1, weights=None) -> \"Iter\"``\n==================================================\n\nReference: `more_itertools.sample <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.sample>`_\n\n.. code-block:: python\n\n >>> iterable = range(100)\n >>> Iter(iterable).sample(5).collect() \n [81, 60, 96, 16, 4]\n\n >>> iterable = range(100)\n >>> weights = (i * i + 1 for i in range(100))\n >>> Iter(iterable).sample(5, weights=weights) \n [79, 67, 74, 66, 78]\n\n >>> data = \"abcdefgh\"\n >>> weights = range(1, len(data) + 1)\n >>> Iter(data).sample(k=len(data), weights=weights) \n ['c', 'a', 'b', 'e', 'g', 'd', 'h', 'f']\n\n\n >>> # This one just to let the doctest run\n >>> iterable = range(100)\n >>> Iter(iterable).sample(5).map(lambda x: 0 <= x < 100).all()\n True\n\n\n\n.. _Iter.consecutive_groups:\n\n\n``Iter.consecutive_groups(self, ordering=lambda x: x)``\n=======================================================\n\nReference: `more_itertools.consecutive_groups <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.consecutive_groups>`_\n\n.. code-block:: python\n\n >>> iterable = [1, 10, 11, 12, 20, 30, 31, 32, 33, 40]\n >>> Iter(iterable).consecutive_groups().map(lambda g: list(g)).print('{v}').consume()\n [1]\n [10, 11, 12]\n [20]\n [30, 31, 32, 33]\n [40]\n\n\n\n.. _Iter.run_length_encode:\n\n\n``Iter.run_length_encode(self) -> \"Iter[Tuple[T, int]]\"``\n=========================================================\n\nReference: `more_itertools.run_length <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.run_length>`_\n\n.. code-block:: python\n\n >>> uncompressed = 'abbcccdddd'\n >>> Iter(uncompressed).run_length_encode().collect()\n [('a', 1), ('b', 2), ('c', 3), ('d', 4)]\n\n\n\n.. _Iter.run_length_decode:\n\n\n``Iter.run_length_decode(self) -> \"Iter\"``\n==========================================\n\nReference: `more_itertools.run_length <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.run_length>`_\n\n.. code-block:: python\n\n >>> compressed = [('a', 1), ('b', 2), ('c', 3), ('d', 4)]\n >>> Iter(compressed).run_length_decode().collect()\n ['a', 'b', 'b', 'c', 'c', 'c', 'd', 'd', 'd', 'd']\n\n\n\n.. _Iter.map_reduce:\n\n\n``Iter.map_reduce(self, keyfunc, valuefunc=None, reducefunc=None) -> \"Dict\"``\n=============================================================================\n\nReference: `more_itertools.map_reduce <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.map_reduce>`_\n\nThis interface mirrors what more-itertools_ does in that it returns\na dict. See ``map_reduce_it()`` for a slightly-modified interface\nthat returns the dict items as another iterator.\n\n.. code-block:: python\n\n >>> keyfunc = lambda x: x.upper()\n >>> d = Iter('abbccc').map_reduce(keyfunc)\n >>> sorted(d.items())\n [('A', ['a']), ('B', ['b', 'b']), ('C', ['c', 'c', 'c'])]\n\n >>> keyfunc = lambda x: x.upper()\n >>> valuefunc = lambda x: 1\n >>> d = Iter('abbccc').map_reduce(keyfunc, valuefunc)\n >>> sorted(d.items())\n [('A', [1]), ('B', [1, 1]), ('C', [1, 1, 1])]\n\n >>> keyfunc = lambda x: x.upper()\n >>> valuefunc = lambda x: 1\n >>> reducefunc = sum\n >>> d = Iter('abbccc').map_reduce(keyfunc, valuefunc, reducefunc)\n >>> sorted(d.items())\n [('A', 1), ('B', 2), ('C', 3)]\n\nNote the warning given in the more-itertools_ docs about how\nlists are created before the reduce step. This means you always want\nto filter *before* applying map_reduce, not after.\n\n.. code-block:: python\n\n >>> all_items = _range(30)\n >>> keyfunc = lambda x: x % 2 # Evens map to 0; odds to 1\n >>> categories = Iter(all_items).filter(lambda x: 10<=x<=20).map_reduce(keyfunc=keyfunc)\n >>> sorted(categories.items())\n [(0, [10, 12, 14, 16, 18, 20]), (1, [11, 13, 15, 17, 19])]\n >>> summaries = Iter(all_items).filter(lambda x: 10<=x<=20).map_reduce(keyfunc=keyfunc, reducefunc=sum)\n >>> sorted(summaries.items())\n [(0, 90), (1, 75)]\n\n\n\n.. _Iter.map_reduce_it:\n\n\n``Iter.map_reduce_it(self, keyfunc: Callable[..., K], valuefunc: Optional[Callable[..., V]] = None, reducefunc: Optional[Callable[..., R]] = None, ) -> \"Iter[Tuple[K, R]]\"``\n=============================================================================================================================================================================\n\nReference: `more_itertools.map_reduce <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.map_reduce>`_\n\n.. code-block:: python\n\n >>> keyfunc = lambda x: x.upper()\n >>> Iter('abbccc').map_reduce_it(keyfunc).collect()\n [('A', ['a']), ('B', ['b', 'b']), ('C', ['c', 'c', 'c'])]\n\n >>> keyfunc = lambda x: x.upper()\n >>> valuefunc = lambda x: 1\n >>> Iter('abbccc').map_reduce_it(keyfunc, valuefunc).collect()\n [('A', [1]), ('B', [1, 1]), ('C', [1, 1, 1])]\n\n >>> keyfunc = lambda x: x.upper()\n >>> valuefunc = lambda x: 1\n >>> reducefunc = sum\n >>> Iter('abbccc').map_reduce_it(keyfunc, valuefunc, reducefunc).collect()\n [('A', 1), ('B', 2), ('C', 3)]\n\n\n\n.. _Iter.exactly_n:\n\n\n|sink| ``Iter.exactly_n(self, n, predicate=bool) -> \"bool\"``\n============================================================\n\n\n\nDocstring TBD\n\n.. code-block:: python\n\n >>> Iter([True, True, False]).exactly_n(2)\n True\n\n\n\n.. _Iter.all_equal:\n\n\n``Iter.all_equal(self)``\n========================\n\n.. _Iter.first_true:\n\n\n``Iter.first_true(self)``\n=========================\n\n.. _Iter.quantify:\n\n\n``Iter.quantify(self)``\n=======================\n\n.. _Iter.islice_extended:\n\n\n``Iter.islice_extended(self, *args)``\n=====================================\n\nReference: `more_itertools.islice_extended <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.islice_extended>`_\n\n.. code-block:: python\n\n >>> Iter('abcdefgh').islice_extended(-4, -1).collect()\n ['e', 'f', 'g']\n\n.. code-block:: python\n\n >>> Iter.count().islice_extended( 110, 99, -2).collect()\n [110, 108, 106, 104, 102, 100]\n\n\n\n.. _Iter.first:\n\n\n``Iter.first(self)``\n====================\n\nReference: `more_itertools.first <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.first>`_\n\n\n.. _Iter.last:\n\n\n``Iter.last(self)``\n===================\n\nReference: `more_itertools.last <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.last>`_\n\n\n.. _Iter.one:\n\n\n``Iter.one(self)``\n==================\n\nReference: `more_itertools.one <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.one>`_\n\n\n\n.. _Iter.only:\n\n\n``Iter.only(self, default=None, too_long=ValueError) -> \"T\"``\n=============================================================\n\nReference: `more_itertools.one <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.one>`_\n\n.. code-block:: python\n\n >>> Iter([]).only(default='missing')\n 'missing'\n >>> Iter([42]).only(default='missing')\n 42\n >>> Iter([1, 2]).only()\n Traceback (most recent call last):\n ...\n ValueError: ...\n\n\n\n.. _Iter.strip:\n\n\n``Iter.strip(self, pred) -> \"Iter[T]\"``\n=======================================\n\nReference: `more_itertools.strip <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.strip>`_\n\n.. code-block:: python\n\n >>> iterable = (None, False, None, 1, 2, None, 3, False, None)\n >>> pred = lambda x: x in {None, False, ''}\n >>> Iter(iterable).strip(pred).collect()\n [1, 2, None, 3]\n\n\n\n.. _Iter.lstrip:\n\n\n``Iter.lstrip(self, pred) -> \"Iter[T]\"``\n========================================\n\nReference: `more_itertools.lstrip <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.lstrip>`_\n\n.. code-block:: python\n\n >>> iterable = (None, False, None, 1, 2, None, 3, False, None)\n >>> pred = lambda x: x in {None, False, ''}\n >>> Iter(iterable).lstrip(pred).collect()\n [1, 2, None, 3, False, None]\n\n\n\n.. _Iter.rstrip:\n\n\n``Iter.rstrip(self, pred) -> \"Iter[T]\"``\n========================================\n\nReference: `more_itertools.rstrip <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.rstrip>`_\n\n.. code-block:: python\n\n >>> iterable = (None, False, None, 1, 2, None, 3, False, None)\n >>> pred = lambda x: x in {None, False, ''}\n >>> Iter(iterable).rstrip(pred).collect()\n [None, False, None, 1, 2, None, 3]\n\n\n\n.. _Iter.filter_except:\n\n\n``Iter.filter_except(self, validator, *exceptions) -> \"Iter[T]\"``\n=================================================================\n\nReference: `more_itertools.filter_except <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.filter_except>`_\n\n.. code-block:: python\n\n >>> iterable = ['1', '2', 'three', '4', None]\n >>> Iter(iterable).filter_except(int, ValueError, TypeError).collect()\n ['1', '2', '4']\n\n\n\n.. _Iter.map_except:\n\n\n``Iter.map_except(self, function, *exceptions) -> \"Iter\"``\n==========================================================\n\nReference: `more_itertools.map_except <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.map_except>`_\n\n.. code-block:: python\n\n >>> iterable = ['1', '2', 'three', '4', None]\n >>> Iter(iterable).map_except(int, ValueError, TypeError).collect()\n [1, 2, 4]\n\n\n\n.. _Iter.nth_or_last:\n\n\n``Iter.nth_or_last(self, n, default=_marker) -> \"T\"``\n=====================================================\n\nReference: `more_itertools.nth_or_last <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.nth_or_last>`_\n\n.. code-block:: python\n\n >>> Iter([0, 1, 2, 3]).nth_or_last(2)\n 2\n >>> Iter([0, 1]).nth_or_last(2)\n 1\n >>> Iter([]).nth_or_last(0, 'some default')\n 'some default'\n\n\n\n.. _Iter.nth:\n\n\n``Iter.nth(self, n, default=None)``\n===================================\n\nReference: `more_itertools.nth <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.nth>`_\n\n\n.. _Iter.take:\n\n\n``Iter.take(self, n: int) -> \"Iter\"``\n=====================================\n\nReference: `more_itertools.take <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.take>`_\n\n\n.. _Iter.tail:\n\n\n``Iter.tail(self, n) -> \"Iter[T]\"``\n===================================\n\nReference: `more_itertools.tail <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.tail>`_\n\n.. code-block:: python\n\n >>> Iter('ABCDEFG').tail(3).collect()\n ['E', 'F', 'G']\n\n\n\n.. _Iter.unique_everseen:\n\n\n``Iter.unique_everseen(self, key=None) -> \"Iter[T]\"``\n=====================================================\n\nReference: `more_itertools.unique_everseen <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.unique_everseen>`_\n\n.. code-block:: python\n\n >>> Iter('AAAABBBCCDAABBB').unique_everseen().collect()\n ['A', 'B', 'C', 'D']\n >>> Iter('ABBCcAD').unique_everseen(key=str.lower).collect()\n ['A', 'B', 'C', 'D']\n\nBe sure to read the *more-itertools* docs whne using unhashable\nitems.\n\n.. code-block:: python\n\n >>> iterable = ([1, 2], [2, 3], [1, 2])\n >>> Iter(iterable).unique_everseen().collect() # Slow\n [[1, 2], [2, 3]]\n >>> Iter(iterable).unique_everseen(key=tuple).collect() # Faster\n [[1, 2], [2, 3]]\n\n\n\n.. _Iter.unique_justseen:\n\n\n``Iter.unique_justseen(self, key=None) -> \"Iter[T]\"``\n=====================================================\n\nReference: `more_itertools.unique_justseen <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.unique_justseen>`_\n\n.. code-block:: python\n\n >>> Iter('AAAABBBCCDAABBB').unique_justseen().collect()\n ['A', 'B', 'C', 'D', 'A', 'B']\n >>> Iter('ABBCcAD').unique_justseen(key=str.lower).collect()\n ['A', 'B', 'C', 'A', 'D']\n\n\n\n.. _Iter.distinct_permutations:\n\n\n``Iter.distinct_permutations(self)``\n====================================\n\nReference: `more_itertools.distinct_permutations <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.distinct_permutations>`_\n\n.. code-block:: python\n\n >>> Iter([1, 0, 1]).distinct_permutations().sorted().collect()\n [(0, 1, 1), (1, 0, 1), (1, 1, 0)]\n\n\n\n.. _Iter.distinct_combinations:\n\n\n``Iter.distinct_combinations(self, r) -> \"Iter[T]\"``\n====================================================\n\nReference: `more_itertools.distinct_combinations <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.distinct_combinations>`_\n\n.. code-block:: python\n\n >>> Iter([0, 0, 1]).distinct_combinations(2).collect()\n [(0, 0), (0, 1)]\n\n\n\n.. _Iter.circular_shifts:\n\n\n``Iter.circular_shifts(self) -> \"Iter[T]\"``\n===========================================\n\nReference: `more_itertools.circular_shifts <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.circular_shifts>`_\n\n.. code-block:: python\n\n >>> Iter(range(4)).circular_shifts().collect()\n [(0, 1, 2, 3), (1, 2, 3, 0), (2, 3, 0, 1), (3, 0, 1, 2)]\n\n\n\n.. _Iter.partitions:\n\n\n``Iter.partitions(self) -> \"Iter[T]\"``\n======================================\n\nReference: `more_itertools.partitions <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.partitions>`_\n\n.. code-block:: python\n\n >>> Iter('abc').partitions().collect()\n [[['a', 'b', 'c']], [['a'], ['b', 'c']], [['a', 'b'], ['c']], [['a'], ['b'], ['c']]]\n >>> Iter('abc').partitions().print('{v}').consume()\n [['a', 'b', 'c']]\n [['a'], ['b', 'c']]\n [['a', 'b'], ['c']]\n [['a'], ['b'], ['c']]\n >>> Iter('abc').partitions().map(lambda v: [''.join(p) for p in v]).print('{v}').consume()\n ['abc']\n ['a', 'bc']\n ['ab', 'c']\n ['a', 'b', 'c']\n\n\n\n.. _Iter.set_partitions:\n\n\n``Iter.set_partitions(self, k=None) -> \"Iter[T]\"``\n==================================================\n\nReference: `more_itertools.set_partitions <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.set_partitions>`_\n\n.. code-block:: python\n\n >>> Iter('abc').set_partitions(2).collect()\n [[['a'], ['b', 'c']], [['a', 'b'], ['c']], [['b'], ['a', 'c']]]\n\n\n\n.. _Iter.powerset:\n\n\n``Iter.powerset(self)``\n=======================\n\nReference: `more_itertools.powerset <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.powerset>`_\n\n.. code-block:: python\n\n >>> Iter([1, 2, 3]).powerset().collect()\n [(), (1,), (2,), (3,), (1, 2), (1, 3), (2, 3), (1, 2, 3)]\n\n\n\n.. _Iter.random_product:\n\n\n``@class_or_instancemethod Iter.random_product(self_or_cls, *args, repeat=1)``\n==============================================================================\n\nReference: `more_itertools.random_product <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.random_product>`_\n\n.. code-block:: python\n\n >>> Iter('abc').random_product(range(4), 'XYZ').collect() \n ['c', 3, 'X']\n >>> Iter.random_product('abc', range(4), 'XYZ').collect() \n ['c', 0, 'Z']\n >>> Iter('abc').random_product(range(0)).collect()\n Traceback (most recent call last):\n ...\n IndexError: Cannot choose from an empty sequence\n >>> Iter.random_product(range(0)).collect()\n Traceback (most recent call last):\n ...\n IndexError: Cannot choose from an empty sequence\n\n\n\n.. _Iter.random_permutation:\n\n\n``Iter.random_permutation(self, r=None)``\n=========================================\n\nReference: `more_itertools.random_permutation <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.random_permutation>`_\n\n.. code-block:: python\n\n >>> Iter(range(5)).random_permutation().collect() \n [2, 0, 4, 3, 1]\n >>> Iter(range(0)).random_permutation().collect()\n []\n\n\n\n.. _Iter.random_combination:\n\n\n``Iter.random_combination(self, r)``\n====================================\n\nReference: `more_itertools.random_combination <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.random_combination>`_\n\n.. code-block:: python\n\n >>> Iter(range(5)).random_combination(3).collect() \n [0, 1, 4]\n >>> Iter(range(5)).random_combination(0).collect()\n []\n\n\n\n.. _Iter.random_combination_with_replacement:\n\n\n``Iter.random_combination_with_replacement(self, r)``\n=====================================================\n\nReference: `more_itertools.random_combination_with_replacement <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.random_combination_with_replacement>`_\n\n.. code-block:: python\n\n >>> Iter(range(3)).random_combination_with_replacement(5).collect() \n [0, 0, 1, 2, 2]\n >>> Iter(range(3)).random_combination_with_replacement(0).collect()\n []\n\n\n\n.. _Iter.nth_combination:\n\n\n``Iter.nth_combination(self, r, index)``\n========================================\n\nReference: `more_itertools.nth_combination <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.nth_combination>`_\n\n.. code-block:: python\n\n >>> Iter(range(9)).nth_combination(3, 1).collect()\n [0, 1, 3]\n >>> Iter(range(9)).nth_combination(3, 2).collect()\n [0, 1, 4]\n >>> Iter(range(9)).nth_combination(3, 3).collect()\n [0, 1, 5]\n >>> Iter(range(9)).nth_combination(4, 3).collect()\n [0, 1, 2, 6]\n >>> Iter(range(9)).nth_combination(3, 7).collect()\n [0, 2, 3]\n\n\n\n.. _Iter.always_iterable:\n\n\n``@classmethod Iter.always_iterable(cls, obj, base_type=(str, bytes)) -> \"Iter\"``\n=================================================================================\n\nReference: `more_itertools.always_iterable <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.always_iterable>`_\n\n.. code-block: python\n\n.. code-block:: python\n\n >>> Iter.always_iterable([1, 2, 3]).collect()\n [1, 2, 3]\n >>> Iter.always_iterable(1).collect()\n [1]\n >>> Iter.always_iterable(None).collect()\n []\n >>> Iter.always_iterable('foo').collect()\n ['foo']\n >>> Iter.always_iterable(dict(a=1), base_type=dict).collect()\n [{'a': 1}]\n\n\n\n.. _Iter.always_reversible:\n\n\n``Iter.always_reversible(self)``\n================================\n\nReference: `more_itertools.always_reversible <https://more-itertools.readthedocs.io/en/stable/api.html#more_itertools.always_reversible>`_\n\nThis is like ``reversed()`` but it also operates on things that\nwouldn't normally be reversible, like generators. It does this with\ninternal caching, so be careful with memory use.\n\n.. code-block: python\n\n >>> Iter('abc').always_reversible().collect()\n ['c', 'b', 'a']\n >>> Iter(x for x in 'abc').always_reversible().collect()\n ['c', 'b', 'a']\n\n\n\n.. _Iter.with_iter:\n\n\n``@classmethod Iter.with_iter(cls, context_manager)``\n=====================================================\n\nReference: `more_itertools.with_iter <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.with_iter>`_\n\nNote: Any context manager which returns an iterable is a candidate for\nIter.with_iter_.\n\n.. code-block:: python\n\n >>> import tempfile\n >>> with tempfile.TemporaryDirectory() as td:\n ... with open(td + 'text.txt', 'w') as f:\n ... f.writelines(['abc\\n', 'def\\n', 'ghi\\n'])\n ... Iter.with_iter(open(td + 'text.txt')).map(lambda x: x.upper()).collect()\n ['ABC\\n', 'DEF\\n', 'GHI\\n']\n\nSee also: Iter.open_\n\n|flux| TODO: perhaps we should get rid of Iter.open_ and just use this?\n\n\n\n.. _Iter.iter_except:\n\n\n``@classmethod Iter.iter_except(cls, func, exception, first=None) -> \"Iter\"``\n=============================================================================\n\nReference: `more_itertools.iter_except <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.iter_except>`_\n\n.. code-block:: python\n\n >>> l = [0, 1, 2]\n >>> Iter.iter_except(l.pop, IndexError).collect()\n [2, 1, 0]\n\n\n\n.. _Iter.locate:\n\n\n``Iter.locate(self, pred=bool, window_size=None) -> \"Iter\"``\n============================================================\n\nReference: `more_itertools.locate <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.locate>`_\n\n.. code-block:: python\n\n >>> Iter([0, 1, 1, 0, 1, 0, 0]).locate().collect()\n [1, 2, 4]\n\n.. code-block:: python\n\n >>> Iter(['a', 'b', 'c', 'b']).locate(lambda x: x == 'b').collect()\n [1, 3]\n\n.. code-block:: python\n\n >>> iterable = [0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3]\n >>> pred = lambda *args: args == (1, 2, 3)\n >>> Iter(iterable).locate(pred=pred, window_size=3).collect()\n [1, 5, 9]\n\n.. code-block:: python\n\n >>> from itertools import count\n >>> from more_itertools import seekable\n >>> source = (3 * n + 1 if (n % 2) else n // 2 for n in count())\n >>> it = Iter(source).seekable()\n >>> pred = lambda x: x > 100\n >>> # TODO: can we avoid making two instances?\n >>> indexes = Iter(it).locate(pred=pred)\n >>> i = next(indexes)\n >>> it.seek(i)\n >>> next(it)\n 106\n\n\n\n.. _Iter.rlocate:\n\n\n``Iter.rlocate(self, pred=bool, window_size=None) -> \"Iter\"``\n=============================================================\n\nReference: `more_itertools.rlocate <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.rlocate>`_\n\n.. code-block:: python\n\n >>> Iter([0, 1, 1, 0, 1, 0, 0]).rlocate().collect() # Truthy at 1, 2, and 4\n [4, 2, 1]\n\n.. code-block:: python\n\n >>> pred = lambda x: x == 'b'\n >>> Iter('abcb').rlocate(pred).collect()\n [3, 1]\n\n.. code-block:: python\n\n >>> iterable = [0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3]\n >>> pred = lambda *args: args == (1, 2, 3)\n >>> Iter(iterable).rlocate(pred=pred, window_size=3).collect()\n [9, 5, 1]\n\n\n\n.. _Iter.replace:\n\n\n``Iter.replace(self, pred, substitutes, count=None, window_size=1) -> \"Iter\"``\n==============================================================================\n\nReference: `more_itertools.replace <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.replace>`_\n\n.. code-block:: python\n\n >>> iterable = [1, 1, 0, 1, 1, 0, 1, 1]\n >>> pred = lambda x: x == 0\n >>> substitutes = (2, 3)\n >>> Iter(iterable).replace(pred, substitutes).collect()\n [1, 1, 2, 3, 1, 1, 2, 3, 1, 1]\n\n.. code-block:: python\n\n >>> iterable = [1, 1, 0, 1, 1, 0, 1, 1, 0]\n >>> pred = lambda x: x == 0\n >>> substitutes = [None]\n >>> Iter(iterable).replace(pred, substitutes, count=2).collect()\n [1, 1, None, 1, 1, None, 1, 1, 0]\n\n.. code-block:: python\n\n >>> iterable = [0, 1, 2, 5, 0, 1, 2, 5]\n >>> window_size = 3\n >>> pred = lambda *args: args == (0, 1, 2) # 3 items passed to pred\n >>> substitutes = [3, 4] # Splice in these items\n >>> Iter(iterable).replace(\n ... pred, substitutes, window_size=window_size\n ... ).collect()\n [3, 4, 5, 3, 4, 5]\n\n\n\n.. _Iter.numeric_range:\n\n\n``@classmethod Iter.numeric_range(cls, *args) -> \"Iter\"``\n=========================================================\n\nReference: `more_itertools.numeric_range <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.numeric_range>`_\n\n.. code-block:: python\n\n >>> Iter.numeric_range(3.5).collect()\n [0.0, 1.0, 2.0, 3.0]\n\n.. code-block:: python\n\n >>> from decimal import Decimal\n >>> start = Decimal('2.1')\n >>> stop = Decimal('5.1')\n >>> Iter.numeric_range(start, stop).collect()\n [Decimal('2.1'), Decimal('3.1'), Decimal('4.1')]\n\n.. code-block:: python\n\n >>> from fractions import Fraction\n >>> start = Fraction(1, 2) # Start at 1/2\n >>> stop = Fraction(5, 2) # End at 5/2\n >>> step = Fraction(1, 2) # Count by 1/2\n >>> Iter.numeric_range(start, stop, step).collect()\n [Fraction(1, 2), Fraction(1, 1), Fraction(3, 2), Fraction(2, 1)]\n\n.. code-block:: python\n\n >>> Iter.numeric_range(3, -1, -1.0).collect()\n [3.0, 2.0, 1.0, 0.0]\n\n\n\n.. _Iter.side_effect:\n\n\n``Iter.side_effect(self, func, *args, chunk_size=None, before=None, after=None)``\n=================================================================================\n\nReference: `more_itertools.side_effect <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.side_effect>`_\n\n.. code-block:: python\n\n >>> def f(item):\n ... if item == 3:\n ... raise Exception('got 3')\n >>> Iter.range(5).side_effect(f).consume()\n Traceback (most recent call last):\n ...\n Exception: got 3\n\n.. code-block:: python\n\n >>> func = lambda item: print('Received {}'.format(item))\n >>> Iter.range(2).side_effect(func).consume()\n Received 0\n Received 1\n\nThis version of ``side_effect`` also allows extra args:\n\n.. code-block:: python\n\n >>> func = lambda item, format_str='Received {}': print(format_str.format(item))\n >>> Iter.range(2).side_effect(func).consume()\n Received 0\n Received 1\n >>> func = lambda item, format_str='Received {}': print(format_str.format(item))\n >>> Iter.range(2).side_effect(func, 'Got {}').consume()\n Got 0\n Got 1\n\n\n\n\n.. _Iter.iterate:\n\n\n``Iter.iterate(self)``\n======================\n\n.. _Iter.difference:\n\n\n``Iter.difference(self, func=operator.sub, *, initial=None)``\n=============================================================\n\nReference: `more_itertools.difference <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=difference#more_itertools.difference>`_\n\n.. code-block:: python\n\n >>> iterable = [0, 1, 3, 6, 10]\n >>> Iter(iterable).difference().collect()\n [0, 1, 2, 3, 4]\n\n.. code-block:: python\n\n >>> iterable = [1, 2, 6, 24, 120] # Factorial sequence\n >>> func = lambda x, y: x // y\n >>> Iter(iterable).difference(func).collect()\n [1, 2, 3, 4, 5]\n\n\n\n.. _Iter.make_decorator:\n\n\n``Iter.make_decorator(self)``\n=============================\n\n.. _Iter.SequenceView:\n\n\n``Iter.SequenceView(self)``\n===========================\n\n.. _Iter.time_limited:\n\n\n``Iter.time_limited(self, limit_seconds) -> \"Iter\"``\n====================================================\n\nReference: `more_itertools.time_limited <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=time_limited#more_itertools.time_limited>`_\n\n.. code-block:: python\n\n >>> from time import sleep\n >>> def generator():\n ... yield 1\n ... yield 2\n ... sleep(0.2)\n ... yield 3\n >>> Iter(generator()).time_limited(0.1).collect()\n [1, 2]\n\n\n\n.. _Iter.consume:\n\n\n|sink| ``Iter.consume(self, n: Optional[int] = None) -> \"Optional[Iter[T]]\"``\n=============================================================================\n\n\nIf n is not provided, the entire iterator is consumed and\n``None`` is returned. Otherwise, an iterator will *always* be\nreturned, even if n is greater than the number of items left in\nthe iterator.\n\nIn this example, the source has more elements than what we consume,\nso there will still be data available on the chain:\n\n.. code-block:: python\n\n >>> range(10).consume(5).collect()\n [5, 6, 7, 8, 9]\n\nWe can bump up the count of how many items can be consumed. Note that\neven though ``n`` is greater than the number of items in the source,\nit is still required to call Iter.collect_ to consume the remaining\nitems.\n\n.. code-block:: python\n\n >>> range(10).consume(50).collect()\n []\n\nFinally, if ``n`` is not provided, the entire stream is consumed.\nIn this scenario, Iter.collect_ would fail since nothing is being\nreturned from the consume call.\n\n.. code-block:: python\n\n >>> assert range(10).consume() is None\n\n\n\n.. _Iter.tabulate:\n\n\n``Iter.tabulate(self)``\n=======================\n\n.. _Iter.repeatfunc:\n\n\n|source| ``@classmethod Iter.repeatfunc(cls, func, *args, times=None)``\n=======================================================================\n\n\nDocstring TBD\n\n.. code-block:: python\n\n >>> Iter.repeatfunc(operator.add, 3, 5, times=4).collect()\n [8, 8, 8, 8]\n\n\n\n.. _Iter.wrap:\n\n\n``Iter.wrap(self, ends: \"Sequence[T, T]\" = \"()\")``\n==================================================\nOther examples for ends: '\"' * 2, or '`' * 2, or '[]' etc.\n\n\n.. _Iter.print:\n\n\n``Iter.print(self, template=\"{i}: {v}\") -> \"Iter[T]\"``\n======================================================\n\nPrinting during the execution of an iterator. Mostly useful\nfor debugging. Returns another iterator instance through which\nthe original data is passed unchanged. This means you can include\na `print()` step as necessary to observe data during iteration.\n\n.. code-block:: python\n\n >>> Iter('abc').print().collect()\n 0: a\n 1: b\n 2: c\n ['a', 'b', 'c']\n\n >>> (\n ... Iter(range(5))\n ... .print('before filter {i}: {v}')\n ... .filter(lambda x: x > 2)\n ... .print('after filter {i}: {v}')\n ... .collect()\n ... )\n before filter 0: 0\n before filter 1: 1\n before filter 2: 2\n before filter 3: 3\n after filter 0: 3\n before filter 4: 4\n after filter 1: 4\n [3, 4]\n\n\n\n.. _Iter.from_queue:\n\n\n|source| ``@classmethod Iter.from_queue(cls, q: queue.Queue, timeout=None, sentinel=None)``\n===========================================================================================\n\n\nWrap a queue with an iterator interface. This allows it to participate\nin chaining operations. The iterator will block while waiting for\nnew values to appear on the queue. This is useful: it allows you\nto easily and safely pass data between threads or processes, and\nfeed the incoming data into a pipeline.\n\nThe sentinel value, default ``None``, will terminate the iterator.\n\n.. code-block:: python\n\n >>> q = queue.Queue()\n >>> # This line puts stuff onto a queue\n >>> range(10).chain([None]).map(q.put).consume()\n >>> # This is where we consume data from the queue:\n >>> Iter.from_queue(q).filter(lambda x: 2 < x < 9).collect()\n [3, 4, 5, 6, 7, 8]\n\nIf ``None`` had not been chained onto the data, the iterator would\nhave waited in Iter.collect_ forever.\n\n\n\n.. _Iter.into_queue:\n\n\n``Iter.into_queue(self, q: queue.Queue) -> \"Iter[T]\"``\n======================================================\n\nThis is a sink, like Iter.collect_, that consumes data from\nan iterator chain and puts the data into the given queue.\n\n.. code-block:: python\n\n >>> q = queue.Queue()\n >>> # This demonstrates the queue sink\n >>> range(5).into_queue(q).consume()\n >>> # Code below is only for verification\n >>> out = []\n >>> finished = False\n >>> while not finished:\n ... try:\n ... out.append(q.get_nowait())\n ... except queue.Empty:\n ... finished = True\n >>> out\n [0, 1, 2, 3, 4]\n\n\n\n.. _Iter.send:\n\n\n|sink| ``Iter.send(self, collector: Generator, close_collector_when_done=False) -> \"None\"``\n===========================================================================================\n\n\nSee also: `more_itertools.consumer <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.consumer>`_\n\nSend data into a generator. You do not have to first call ``next()``\non the generator. Iter.send_ will do this for you.\n\n|warning| Look carefully at the examples below; you'll see that the\n``yield`` keyword is wrapped in a second set of parens, e.g.\n``output.append((yield))``. This is required!\n\nSimple case:\n\n.. code-block:: python\n\n >>> output = []\n >>> def collector():\n ... while True:\n ... output.append((yield))\n >>> Iter.range(3).send(collector())\n >>> output\n [0, 1, 2]\n\nNote that the generator is **not** closed by default after the iterable is\nexhausted. But this can be changed. If you choose to close the\ngenerator, use the parameter:\n\n.. code-block:: python\n\n >>> output = []\n >>> def collector():\n ... while True:\n ... output.append((yield))\n >>> g = collector()\n >>> Iter.range(3).send(g, close_collector_when_done=True)\n >>> Iter.range(3).send(g)\n Traceback (most recent call last):\n ...\n StopIteration\n\nThe default behaviour is that the generator is left open which means you\ncan keep using it for other iterators:\n\n.. code-block:: python\n\n >>> output = []\n >>> def collector():\n ... while True:\n ... output.append((yield))\n >>> g = collector()\n >>> Iter.range(3).send(g)\n >>> Iter.range(10, 13).send(g)\n >>> Iter.range(100, 103).send(g)\n >>> output\n [0, 1, 2, 10, 11, 12, 100, 101, 102]\n\n\nIf the generator is closed before the iteration is complete,\nyou'll get a ``StopIteration`` exception:\n\n.. code-block:: python\n\n >>> output = []\n >>> def collector():\n ... for i in range(3):\n ... output.append((yield))\n >>> Iter.range(5).send(collector())\n Traceback (most recent call last):\n ...\n StopIteration\n\nNote that Iter.send_ is a sink, so no further chaining is allowed.\n\n\n\n.. _Iter.send_also:\n\n\n``Iter.send_also(self, collector: Generator) -> \"Iter\"``\n========================================================\n\nReference: `more_itertools.consumer <https://more-itertools.readthedocs.io/en/stable/api.html?highlight=numeric_range#more_itertools.consumer>`_\n\nSome ideas around a reverse iterator as a sink. Usually you have\nfirst to \"send\" a ``None`` into a generator if you want to send\nmore values into it (or call ``next()`` on it), but we handle\nthat automatically.\n\nSimple case:\n\n.. code-block:: python\n\n >>> output = []\n >>> def collector():\n ... while True:\n ... output.append((yield))\n >>> Iter.range(3).send_also(collector()).collect()\n [0, 1, 2]\n >>> output\n [0, 1, 2]\n\nHowever, if the caller already started the generator, that\nworks too:\n\n.. code-block:: python\n\n >>> output = []\n >>> def collector():\n ... while True:\n ... output.append((yield))\n >>> g = collector()\n >>> next(g) # This \"starts\" the generator\n >>> Iter.range(3).send_also(g).collect()\n [0, 1, 2]\n >>> output\n [0, 1, 2]\n\nIf the generator is closed before the iteration is complete,\nyou'll get an exception (Python 3.7+):\n\n.. code-block:: python\n\n >>> output = []\n >>> def collector():\n ... for i in builtins.range(3):\n ... output.append((yield))\n >>> Iter.range(50).send_also(collector()).collect() \n Traceback (most recent call last):\n ...\n RuntimeError\n\nNote that the above doesn't happen in Python < 3.7 (which includes\npypy 7.3.1 that matches Python 3.6.9 compatibility). Instead, you\ncollect out the items up to until the point that the collector\nreturns; in this case, you'd get [0, 1, 2]. This change was made\nas part of `PEP 479 <https://www.python.org/dev/peps/pep-0479/>`_.\n\nRegardless, for any Python it's recommended that your generator\nlive at least as long as the iterator feeding it.\n\n\n\n.. _Iter.sorted:\n\n\n|sink| |warning| ``Iter.sorted(self, key=None, reverse=False) -> \"Iter[T]\"``\n============================================================================\n\n\n\nSimple wrapper for the ``sorted`` builtin.\n\n\nCalling this will read the entire stream before producing\nresults.\n\n.. code-block:: python\n\n >>> Iter('bac').sorted().collect()\n ['a', 'b', 'c']\n >>> Iter('bac').sorted(reverse=True).collect()\n ['c', 'b', 'a']\n >>> Iter('bac').zip([2, 1, 0]).sorted(key=lambda tup: tup[1]).collect()\n [('c', 0), ('a', 1), ('b', 2)]\n\n\n\n.. _Iter.reversed:\n\n\n|sink| |warning| ``Iter.reversed(self) -> \"Iter[T]\"``\n=====================================================\n\n\n\nSimple wrapper for the ``reversed`` builtin.\n\n\nCalling this will read the entire stream before producing\nresults.\n\n.. code-block:: python\n\n >>> Iter(range(4)).reversed().collect()\n [3, 2, 1, 0]\n\n\n\n\nExperiments and Provisional Ideas\n#################################\n\n\n\n.. _IterDict:\n\n\n|flux| ``class IterDict(UserDict)``\n***********************************\n\n\n\nThe idea here was to make a custom dict where several of\nthe standard dict methods return ``Iter`` instances, which can then\nbe chained. I'm not sure if this will be kept yet.\n\n\n.. _IterDict.keys:\n\n\n``IterDict.keys(self) -> \"Iter\"``\n=================================\n\n.. _IterDict.values:\n\n\n``IterDict.values(self) -> \"Iter\"``\n===================================\n\n.. _IterDict.items:\n\n\n``IterDict.items(self) -> \"Iter\"``\n==================================\n\n.. _IterDict.update:\n\n\n``IterDict.update(self, *args, **kwargs) -> \"IterDict\"``\n========================================================\n\n.. _insert_separator:\n\n\n``insert_separator(iterable: Iterable[Any], glue: Any) -> \"Iterable[Any]\"``\n***************************************************************************\nSimilar functionality can be obtained with, e.g.,\ninterleave, as in\n\n.. code-block:: python\n\n >>> result = Iter('caleb').interleave(Iter.repeat('x')).collect()\n >>> result == list('cxaxlxexbx')\n True\n\nBut you'll see a trailing \"x\" there, which join avoids. join\nmakes sure to only add the glue separator if another element\nhas arrived.\n\nIt can handle strings without any special considerations, but it doesn't\ndo any special handling for bytes and bytearrays. For that, rather\nlook at `concat()`.\n\n\n\nRelated projects\n################\n\nIt turns out the idea of chaining iterators is not new. There are many\nlibraries that offer similar features:\n\n* My fork of a now-missing library: `chained-iterable <https://github.com/cjrh/chained-iterable>`_.\n\n* `https://github.com/olirice/flupy <https://github.com/olirice/flupy>`_\n\n* `https://github.com/ddstte/chiter <https://github.com/ddstte/chiter>`_\n\n* `https://github.com/neverendingqs/pyiterable <https://github.com/neverendingqs/pyiterable>`_\n\n* `https://github.com/alliefitter/iterable_collections <https://github.com/alliefitter/iterable_collections>`_\n\n* `https://github.com/halprin/iterator-chain <https://github.com/halprin/iterator-chain>`_\n\n* `https://github.com/jagill/python-chainz <https://github.com/jagill/python-chainz>`_\n\n* `https://github.com/ZianVW/IterPipe <https://github.com/ZianVW/IterPipe>`_\n\n* `https://github.com/Evelyn-H/iterchain <https://github.com/Evelyn-H/iterchain>`_\n\n* `https://github.com/EntilZha/PyFunctional <https://github.com/EntilZha/PyFunctional>`_\n\n* `https://github.com/dwt/fluent <https://github.com/dwt/fluent>`_\n\nSomewhat related:\n\n* `https://github.com/jreese/aioitertools <https://github.com/jreese/aioitertools>`_\n\n\nDev Instructions\n################\n\nSetup\n*****\n\n.. code-block:: shell\n\n $ python -m venv venv\n $ source venv/bin/activate\n (venv) $ pip install -e .[dev,test]\n\nTesting\n*******\n\n.. code-block:: shell\n\n (venv) $ pytest --cov\n\nDocumentation\n*************\n\nTo regenerate the documentation, file ``README.rst``:\n\n.. code-block:: shell\n\n (venv) $ python regenerate_readme.py -m excitertools.py > README.rst\n\nReleasing\n*********\n\nTo do a release, we're using `bumpymcbumpface <https://pypi.org/project/bumpymcbumpface/>`_.\nMake sure that is set up correctly according to its own documentation. I \nlike to use `pipx <https://github.com/pipxproject/pipx>`_ to install and \nmanage these kinds of tools.\n\n.. code-block:: shell\n\n $ bumpymcbumpface --push-git --push-pypi\n\n|\n|\n\n-----\n\n|\n|\n\n Work is a necessary evil to be avoided. \n *Mark Twain*\n\n\n\n",
"bugtrack_url": null,
"license": null,
"summary": "itertools with function chaining",
"version": "2024.11.3",
"project_urls": {
"Homepage": "https://github.com/cjrh/excitertools"
},
"split_keywords": [
"itertools"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "0afe3d024011346e04f50b7cfafa2610964a861e32a6e085b311663ade373e0b",
"md5": "42f2023636cac61537e6434af1cf1598",
"sha256": "c3fd9f4bd5ce4f9e4f10c9bbdb7e6f785f77c2fbba735a4ffc61df459bb36190"
},
"downloads": -1,
"filename": "excitertools-2024.11.3-py3-none-any.whl",
"has_sig": false,
"md5_digest": "42f2023636cac61537e6434af1cf1598",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 60567,
"upload_time": "2024-11-24T13:00:01",
"upload_time_iso_8601": "2024-11-24T13:00:01.450796Z",
"url": "https://files.pythonhosted.org/packages/0a/fe/3d024011346e04f50b7cfafa2610964a861e32a6e085b311663ade373e0b/excitertools-2024.11.3-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "570467a2c1d23cb76e9888ab22e38e1d16ae9f02f6548c061d3b70a999b8dc52",
"md5": "01922cbd6b0232bf293c5a11a2bfabcc",
"sha256": "34b099ab1dfeee2e424682297985b7f0db12afbc33153be3db1d261ac0cce61f"
},
"downloads": -1,
"filename": "excitertools-2024.11.3.tar.gz",
"has_sig": false,
"md5_digest": "01922cbd6b0232bf293c5a11a2bfabcc",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 116366,
"upload_time": "2024-11-24T13:00:05",
"upload_time_iso_8601": "2024-11-24T13:00:05.506900Z",
"url": "https://files.pythonhosted.org/packages/57/04/67a2c1d23cb76e9888ab22e38e1d16ae9f02f6548c061d3b70a999b8dc52/excitertools-2024.11.3.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-24 13:00:05",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cjrh",
"github_project": "excitertools",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"lcname": "excitertools"
}
JSON
