<p align="center">
<img src="https://github.com/kaliv0/pyrio/blob/main/assets/Pyrio.jpg?raw=true" width="400" alt="Pyrio">
</p>
# PYRIO
[](https://github.com/kaliv0/pyrio/actions/workflows/ci.yml)
[](https://codecov.io/gh/kaliv0/pyrio)
[](https://pyrio.readthedocs.io/en/latest)
[](https://pypi.org/project/pyrio/)
[](https://pepy.tech/projects/pyrio)
<br><b>Functional-style Streams API library</b><br>
<br>Facilitates processing of collections and iterables using fluent APIs.
<br>Gives access to files of various types (<i>json</i>, <i>toml</i>, <i>yaml</i>, <i>xml</i>, <i>csv</i> and <i>tsv</i>) for reading and executing complex queries
<br>Provides easy integration with <i>itertools</i>
<br>(NB: Commonly used <i>itertools 'recipes'</i> are included as part of the main APIs)
## How to use
### Creating streams
- stream from iterable
```python
Stream([1, 2, 3])
```
- from variadic arguments
```python
Stream.of(1, 2, 3)
```
- empty stream
```python
Stream.empty()
```
- infinite ordered stream
```python
Stream.iterate(0, lambda x: x + 1)
```
- infinite unordered stream
```python
import random
Stream.generate(lambda: random.random())
```
- infinite stream with given value
```python
Stream.constant(42)
```
- concat
<br>(concatenate new streams/iterables with the current one)
```python
Stream.of(1, 2, 3).concat(Stream.of(4, 5)).to_list()
Stream([1, 2, 3]).concat([5, 6]).to_list()
```
- prepend
<br>(prepend new stream/iterable to the current one)
```python
Stream([2, 3, 4]).prepend(0, 1).to_list()
Stream.of(3, 4, 5).prepend(Stream.of([0, 1], 2)).to_list()
```
--------------------------------------------
### Intermediate operations
- filter
```python
Stream([1, 2, 3]).filter(lambda x: x % 2 == 0)
```
- map
```python
Stream([1, 2, 3]).map(str).to_list()
Stream([1, 2, 3]).map(lambda x: x + 5).to_list()
```
- filter_map
<br>(filter out all None or falsy values (if falsy=True) and applies mapper function to the elements of the stream)
```python
Stream.of(None, "foo", "", "bar", 0, []).filter_map(str.upper, falsy=True).to_list()
```
```shell
["FOO", "BAR"]
```
- flat_map
<br>(map each element of the stream and yields the elements of the produced iterators)
```python
Stream([[1, 2], [3, 4], [5]]).flat_map(lambda x: Stream(x)).to_list()
```
```shell
[1, 2, 3, 4, 5]
```
- flatten
```python
Stream([[1, 2], [3, 4], [5]]).flatten().to_list()
```
```shell
[1, 2, 3, 4, 5]
```
- reduce
<br>(returns Optional)
```python
Stream([1, 2, 3]).reduce(lambda acc, val: acc + val, identity=3).get()
```
- peek
<br>(perform the provided operation on each element of the stream without consuming it)
```python
(Stream([1, 2, 3, 4])
.filter(lambda x: x > 2)
.peek(lambda x: print(f"{x} ", end=""))
.map(lambda x: x * 20)
.to_list())
```
- view
<br>(provides access to a selected part of the stream)
```python
Stream([1, 2, 3, 4, 5, 6, 7, 8, 9]).view(start=1, stop=-3, step=2).to_list()
```
```shell
[2, 4, 6]
```
- distinct
<br>(returns a stream with the distinct elements of the current one)
```python
Stream([1, 1, 2, 2, 2, 3]).distinct().to_list()
```
- skip
<br>(discards the first n elements of the stream and returns a new stream with the remaining ones)
```python
Stream.iterate(0, lambda x: x + 1).skip(5).limit(5).to_list()
```
- limit / head
<br>(returns a stream with the first n elements, or fewer if the underlying iterator ends sooner)
```python
Stream([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).limit(3).to_tuple()
Stream([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).head(3).to_tuple()
```
- tail
<br>(returns a stream with the last n elements, or fewer if the underlying iterator ends sooner)
```python
Stream([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).tail(3).to_tuple()
```
- take_while
<br>(returns a stream that yields elements based on a predicate)
```python
Stream.of(1, 2, 3, 4, 5, 6, 7, 2, 3).take_while(lambda x: x < 5).to_list()
```
```shell
[1, 2, 3, 4]
```
- drop_while
<br>(returns a stream that skips elements based on a predicate and yields the remaining ones)
```python
Stream.of(1, 2, 3, 5, 6, 7, 2).drop_while(lambda x: x < 5).to_list()
```
```shell
[5, 6, 7, 2]
```
- sort
<br>(sorts the elements of the current stream according to natural order or based on the given comparator;
<br>if 'reverse' flag is True, the elements are sorted in descending order)
```python
(Stream.of((3, 30), (2, 30), (2, 20), (1, 20), (1, 10))
.sort(lambda x: (x[0], x[1]), reverse=True)
.to_list())
```
```shell
[(3, 30), (2, 30), (2, 20), (1, 20), (1, 10)]
```
- reverse
<br>(sorts the elements of the current stream in reverse order;
<br>alias for <i>'sort(collector, reverse=True)'</i>)
```python
(Stream.of((3, 30), (2, 30), (2, 20), (1, 20), (1, 10))
.reverse(lambda x: (x[0], x[1]))
.to_list())
```
```shell
[(3, 30), (2, 30), (2, 20), (1, 20), (1, 10)]
```
<br>NB: in case of stream of dicts all key-value pairs are represented internally as <i>DictItem</i> objects
<br>(including recursively for nested Mapping structures)
<br>to provide more convenient intermediate operations syntax e.g.
```python
first_dict = {"a": 1, "b": 2}
second_dict = {"x": 3, "y": 4}
(Stream(first_dict).concat(second_dict)
.filter(lambda x: x.value % 2 == 0)
.map(lambda x: x.key)
.to_list())
```
--------------------------------------------
### Terminal operations
#### Collectors
- collecting result into list, tuple, set
```python
Stream([1, 2, 3]).to_list()
Stream([1, 2, 3]).to_tuple()
Stream([1, 2, 3]).to_set()
```
- into dict
```python
class Foo:
def __init__(self, name, num):
self.name = name
self.num = num
Stream([Foo("fizz", 1), Foo("buzz", 2)]).to_dict(lambda x: (x.name, x.num))
```
```shell
{"fizz": 1, "buzz": 2}
```
In the case of a collision (duplicate keys) the 'merger' functions indicates which entry should be kept
```python
collection = [Foo("fizz", 1), Foo("fizz", 2), Foo("buzz", 2)]
Stream(collection).to_dict(collector=lambda x: (x.name, x.num), merger=lambda old, new: old)
```
```shell
{"fizz": 1, "buzz": 2}
```
<i>to_dict</i> method also supports creating dictionaries from dict DictItem objects
```python
first_dict = {"x": 1, "y": 2}
second_dict = {"p": 33, "q": 44, "r": None}
Stream(first_dict).concat(Stream(second_dict)).to_dict(lambda x: DictItem(x.key, x.value or 0))
```
```shell
{"x": 1, "y": 2, "p": 33, "q": 44, "r": 0}
```
e.g. you could combine streams of dicts by writing:
```python
Stream(first_dict).concat(Stream(second_dict)).to_dict()
```
(simplified from <i>'.to_dict(lambda x: x)'</i>)
- into string
```python
Stream({"a": 1, "b": [2, 3]}).to_string()
```
```shell
"Stream(DictItem(key=a, value=1), DictItem(key=b, value=[2, 3]))"
```
```python
Stream({"a": 1, "b": [2, 3]}).map(lambda x: {x.key: x.value}).to_string(delimiter=" | ")
```
```shell
"Stream({'a': 1} | {'b': [2, 3]})"
```
- alternative for working with collectors is using the <i>collect</i> method
```python
Stream([1, 2, 3]).collect(tuple)
Stream.of(1, 2, 3).collect(list)
Stream.of(1, 1, 2, 2, 2, 3).collect(set)
Stream.of(1, 2, 3, 4).collect(dict, lambda x: (str(x), x * 10))
```
- grouping
```python
Stream("AAAABBBCCD").group_by(collector=lambda key, grouper: (key, len(grouper)))
```
```shell
{"A": 4, "B": 3, "C": 2, "D": 1}
```
```python
coll = [Foo("fizz", 1), Foo("fizz", 2), Foo("fizz", 3), Foo("buzz", 2), Foo("buzz", 3), Foo("buzz", 4), Foo("buzz", 5)]
Stream(coll).group_by(
classifier=lambda obj: obj.name,
collector=lambda key, grouper: (key, [(obj.name, obj.num) for obj in list(grouper)]))
```
```shell
{
"fizz": [("fizz", 1), ("fizz", 2), ("fizz", 3)],
"buzz": [("buzz", 2), ("buzz", 3), ("buzz", 4), ("buzz", 5)],
}
```
#### Other terminal operations
- for_each
```python
Stream([1, 2, 3, 4]).for_each(lambda x: print(f"{'#' * x} ", end=""))
```
- count
<br>(returns the count of elements in the stream)
```python
Stream([1, 2, 3, 4]).filter(lambda x: x % 2 == 0).count()
```
- sum
```python
Stream.of(1, 2, 3, 4).sum()
```
- find_first
<br>(search for an element of the stream that satisfies a predicate,
returns an Optional with the first found value, if any, or None)
```python
Stream.of(1, 2, 3, 4).filter(lambda x: x % 2 == 0).find_first().get()
```
- find_any
<br>(search for an element of the stream that satisfies a predicate,
returns an Optional with some of the found values, if any, or None)
```python
Stream.of(1, 2, 3, 4).filter(lambda x: x % 2 == 0).find_any().get()
```
- any_match
<br>(returns whether any elements of the stream match the given predicate)
```python
Stream.of(1, 2, 3, 4).any_match(lambda x: x > 2)
```
- all_match
<br>(returns whether all elements of the stream match the given predicate)
```python
Stream.of(1, 2, 3, 4).all_match(lambda x: x > 2)
```
- none_match
<br>(returns whether no elements of the stream match the given predicate)
```python
Stream.of(1, 2, 3, 4).none_match(lambda x: x < 0)
```
- min
<br>(returns Optional with the minimum element of the stream)
```python
Stream.of(2, 1, 3, 4).min().get()
```
- max
<br>(returns Optional with the maximum element of the stream)
```python
Stream.of(2, 1, 3, 4).max().get()
```
- compare_with
<br>(compares linearly the contents of two streams based on a given comparator)
```python
fizz = Foo("fizz", 1)
buzz = Foo("buzz", 2)
Stream([buzz, fizz]).compare_with(Stream([fizz, buzz]), lambda x, y: x.num == y.num)
```
- quantify
<br>(count how many of the elements are Truthy or evaluate to True based on a given predicate)
```python
Stream([2, 3, 4, 5, 6]).quantify(predicate=lambda x: x % 2 == 0)
```
--------------------------------------------
### Itertools integration
Invoke <i>use</i> method by passing the itertools function and it's arguments as **kwargs
```python
import itertools
import operator
Stream([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).use(itertools.islice, start=3, stop=8)
Stream.of(1, 2, 3, 4, 5).use(itertools.accumulate, func=operator.mul).to_list()
Stream(range(3)).use(itertools.permutations, r=3).to_list()
```
#### Itertools 'recipes'
Invoke the 'recipes' described [here](https://docs.python.org/3/library/itertools.html#itertools-recipes) as stream methods and pass required key-word arguments
```python
Stream([1, 2, 3]).ncycles(count=2).to_list()
Stream.of(2, 3, 4).take_nth(10, default=66).get()
Stream(["ABC", "D", "EF"]).round_robin().to_list()
```
--------------------------------------------
### FileStreams
#### Querying files
- working with <i>json</i>, <i>toml</i>, <i>yaml</i>, <i>xml</i> files
<br>NB: FileStream reads data as series of DictItem objects from underlying dict_items view
```python
FileStream("path/to/file").map(lambda x: f"{x.key}=>{x.value}").to_tuple()
```
```shell
(
"abc=>xyz",
"qwerty=>42",
)
```
```python
from operator import attrgetter
from pyrio import DictItem
(FileStream("path/to/file")
.filter(lambda x: "a" in x.key)
.map(lambda x: DictItem(x.key, sum(x.value) * 10))
.sort(attrgetter("value"), reverse=True)
.map(lambda x: f"{str(x.value)}::{x.key}")
.to_list())
```
```shell
["230::xza", "110::abba", "30::a"]
```
- querying <i>csv</i> and <i>tsv</i> files
<br>(each row is read as a dict with keys taken from the header)
```python
FileStream("path/to/file").map(lambda x: f"fizz: {x['fizz']}, buzz: {x['buzz']}").to_tuple()
```
```shell
(
"fizz: 42, buzz: 45",
"fizz: aaa, buzz: bbb",
)
```
```python
from operator import itemgetter
FileStream("path/to/file").map(itemgetter('fizz', 'buzz')).to_tuple()
```
```shell
(('42', '45'), ('aaa', 'bbb'))
```
You could query the nested dicts by creating streams out of them
```python
(FileStream("path/to/file")
.map(lambda x: (Stream(x).to_dict(lambda y: DictItem(y.key, y.value or "Unknown"))))
.save())
```
- reading a file with <i>process()</i> method
- use extra <i>f_open_options</i> (for the underlying <i>open file</i> function)
- <i>f_read_options</i> (to be passed to the corresponding library function that is loading the file content e.g. tomllib, json)
```python
from decimal import Decimal
(FileStream.process(
file_path="path/to/file.json",
f_open_options={"encoding": "utf-8"},
f_read_options={"parse_float": Decimal})
.map(lambda x:x.value).to_list())
```
```shell
['foo', True, Decimal('1.22'), Decimal('5.456367654369698986')]
```
To include the <i>root</i> tag when loading an <i>.xml</i> file pass <i>'include_root=True'</i>
```python
FileStream.process("path/to/custom_root.xml", include_root=True).map(
lambda x: f"root={x.key}: inner_records={str(x.value)}"
).to_list()
```
```shell
["root=custom-root: inner_records={'abc': 'xyz', 'qwerty': '42'}"]
```
--------------------------------------------
#### Saving to a file
- write the contents of a FileStream by passing a <i>file_path</i> to the <i>save()</i> method
```python
in_memory_dict = Stream(json_dict).filter(lambda x: len(x.key) < 6).to_tuple()
FileStream("path/to/file.json").prepend(in_memory_dict).save("./tests/resources/updated.json")
```
If no path is given, the source file for the FileStream will be updated
```python
FileStream("path/to/file.json").concat(in_memory_dict).save()
```
NB: if while updating the file something goes wrong, the original content will be restored/preserved
- handle null values
<br>(pass <i>null_handler</i> function to replace null values)
```python
FileStream("path/to/test.toml").save(null_handler=lambda x: DictItem(x.key, x.value or "N/A"))
```
NB: useful for writing <i>.toml</i> files which don't allow None values
- passing advanced <i>file open</i> and <i>write</i> options
<br>similarly to the <i>process</i> method you could provide
- <i>f_open_options</i> (for the underlying <i>open</i> function)
- <i>f_write_options</i> (passed to the corresponding library that will 'dump' the contents of the stream e.g. tomli-w, pyyaml)
```python
FileStream("path/to/file.json").concat(in_memory_dict).save(
file_path="merged.xml",
f_open_options={"encoding": "utf-8"},
f_write_options={"indent": 4},
)
```
To add <i>custom root</i> tag when saving an <i>.xml</i> file pass <i>'xml_root="my-custom-root"'</i>
```python
FileStream("path/to/file.json").concat(in_memory_dict).save(
file_path="path/to/custom.xml",
f_open_options={"encoding": "utf-8"},
f_write_options={"indent": 4},
xml_root="my-custom-root",
)
```
--------------------------------------------
- how far can we actually push it?
```python
(
FileStream("path/to/file.csv")
.concat(
FileStream("path/to/other/file.json")
.filter(
lambda x: (
Stream(x.value)
.find_first(lambda y: y.key == "name" and y.value != "Snake")
.or_else_get(lambda: None)
)
is not None
)
.map(lambda x: x.value)
)
.map(lambda x: (Stream(x).to_dict(lambda y: DictItem(y.key, y.value or "N/A"))))
.save("path/to/third/file.tsv")
)
```
or how hideous can it get?
<p align="center">
<img src="https://github.com/kaliv0/pyrio/blob/main/assets/Chubby.jpg?raw=true" width="400" alt="Chubby">
</p>
Raw data
{
"_id": null,
"home_page": "https://github.com/kaliv0/pyrio",
"name": "pyrio",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.12",
"maintainer_email": null,
"keywords": "stream, functional, processing, collections, fluent API, file processing, queries",
"author": "kaliv0",
"author_email": "kaloyan.ivanov88@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/62/05/e090cb70bb75431f1c9c93e896a5abc447d40c9f8e2e24680c5d0f2f8502/pyrio-1.3.4.tar.gz",
"platform": null,
"description": "<p align=\"center\">\n <img src=\"https://github.com/kaliv0/pyrio/blob/main/assets/Pyrio.jpg?raw=true\" width=\"400\" alt=\"Pyrio\">\n</p>\n\n# PYRIO\n\n\n\n[](https://github.com/kaliv0/pyrio/actions/workflows/ci.yml)\n[](https://codecov.io/gh/kaliv0/pyrio)\n[](https://pyrio.readthedocs.io/en/latest)\n[](https://pypi.org/project/pyrio/)\n[](https://pepy.tech/projects/pyrio)\n\n<br><b>Functional-style Streams API library</b><br>\n<br>Facilitates processing of collections and iterables using fluent APIs.\n<br>Gives access to files of various types (<i>json</i>, <i>toml</i>, <i>yaml</i>, <i>xml</i>, <i>csv</i> and <i>tsv</i>) for reading and executing complex queries\n<br>Provides easy integration with <i>itertools</i>\n<br>(NB: Commonly used <i>itertools 'recipes'</i> are included as part of the main APIs)\n\n## How to use\n### Creating streams\n- stream from iterable\n```python\nStream([1, 2, 3])\n```\n\n- from variadic arguments\n```python\nStream.of(1, 2, 3)\n```\n\n- empty stream\n```python\nStream.empty()\n```\n\n- infinite ordered stream\n```python\nStream.iterate(0, lambda x: x + 1)\n```\n\n- infinite unordered stream\n```python\nimport random\n\nStream.generate(lambda: random.random())\n```\n\n- infinite stream with given value\n```python\nStream.constant(42)\n```\n\n- concat\n<br>(concatenate new streams/iterables with the current one)\n```python\nStream.of(1, 2, 3).concat(Stream.of(4, 5)).to_list()\nStream([1, 2, 3]).concat([5, 6]).to_list()\n```\n\n- prepend\n<br>(prepend new stream/iterable to the current one)\n```python\nStream([2, 3, 4]).prepend(0, 1).to_list()\nStream.of(3, 4, 5).prepend(Stream.of([0, 1], 2)).to_list()\n\n```\n--------------------------------------------\n### Intermediate operations\n- filter\n```python\nStream([1, 2, 3]).filter(lambda x: x % 2 == 0)\n```\n\n- map\n```python\nStream([1, 2, 3]).map(str).to_list()\nStream([1, 2, 3]).map(lambda x: x + 5).to_list()\n```\n\n- filter_map\n<br>(filter out all None or falsy values (if falsy=True) and applies mapper function to the elements of the stream)\n```python\nStream.of(None, \"foo\", \"\", \"bar\", 0, []).filter_map(str.upper, falsy=True).to_list()\n```\n```shell\n[\"FOO\", \"BAR\"]\n```\n\n- flat_map\n<br>(map each element of the stream and yields the elements of the produced iterators)\n```python\nStream([[1, 2], [3, 4], [5]]).flat_map(lambda x: Stream(x)).to_list()\n```\n```shell\n[1, 2, 3, 4, 5]\n```\n\n- flatten\n```python\nStream([[1, 2], [3, 4], [5]]).flatten().to_list()\n```\n```shell\n[1, 2, 3, 4, 5]\n```\n\n- reduce \n<br>(returns Optional)\n```python\nStream([1, 2, 3]).reduce(lambda acc, val: acc + val, identity=3).get()\n```\n\n- peek\n<br>(perform the provided operation on each element of the stream without consuming it)\n```python\n(Stream([1, 2, 3, 4])\n .filter(lambda x: x > 2)\n .peek(lambda x: print(f\"{x} \", end=\"\"))\n .map(lambda x: x * 20)\n .to_list())\n```\n\n- view\n<br>(provides access to a selected part of the stream)\n```python\nStream([1, 2, 3, 4, 5, 6, 7, 8, 9]).view(start=1, stop=-3, step=2).to_list()\n```\n```shell\n[2, 4, 6]\n```\n\n- distinct\n<br>(returns a stream with the distinct elements of the current one)\n```python\nStream([1, 1, 2, 2, 2, 3]).distinct().to_list()\n```\n\n- skip\n<br>(discards the first n elements of the stream and returns a new stream with the remaining ones)\n```python\nStream.iterate(0, lambda x: x + 1).skip(5).limit(5).to_list()\n```\n\n- limit / head\n<br>(returns a stream with the first n elements, or fewer if the underlying iterator ends sooner)\n```python\nStream([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).limit(3).to_tuple()\nStream([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).head(3).to_tuple()\n```\n\n- tail\n<br>(returns a stream with the last n elements, or fewer if the underlying iterator ends sooner)\n```python\nStream([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).tail(3).to_tuple()\n```\n\n- take_while\n<br>(returns a stream that yields elements based on a predicate)\n```python\nStream.of(1, 2, 3, 4, 5, 6, 7, 2, 3).take_while(lambda x: x < 5).to_list()\n```\n```shell\n[1, 2, 3, 4]\n```\n\n- drop_while\n<br>(returns a stream that skips elements based on a predicate and yields the remaining ones)\n```python\nStream.of(1, 2, 3, 5, 6, 7, 2).drop_while(lambda x: x < 5).to_list()\n```\n```shell\n[5, 6, 7, 2]\n```\n\n- sort\n<br>(sorts the elements of the current stream according to natural order or based on the given comparator;\n<br>if 'reverse' flag is True, the elements are sorted in descending order)\n```python\n(Stream.of((3, 30), (2, 30), (2, 20), (1, 20), (1, 10))\n .sort(lambda x: (x[0], x[1]), reverse=True)\n .to_list())\n```\n```shell\n[(3, 30), (2, 30), (2, 20), (1, 20), (1, 10)]\n```\n\n- reverse\n<br>(sorts the elements of the current stream in reverse order;\n<br>alias for <i>'sort(collector, reverse=True)'</i>)\n```python\n(Stream.of((3, 30), (2, 30), (2, 20), (1, 20), (1, 10))\n .reverse(lambda x: (x[0], x[1]))\n .to_list())\n```\n```shell\n[(3, 30), (2, 30), (2, 20), (1, 20), (1, 10)]\n```\n\n<br>NB: in case of stream of dicts all key-value pairs are represented internally as <i>DictItem</i> objects \n<br>(including recursively for nested Mapping structures)\n<br>to provide more convenient intermediate operations syntax e.g.\n```python\nfirst_dict = {\"a\": 1, \"b\": 2}\nsecond_dict = {\"x\": 3, \"y\": 4}\n(Stream(first_dict).concat(second_dict)\n .filter(lambda x: x.value % 2 == 0)\n .map(lambda x: x.key)\n .to_list()) \n```\n\n--------------------------------------------\n### Terminal operations\n#### Collectors\n- collecting result into list, tuple, set\n```python\nStream([1, 2, 3]).to_list()\nStream([1, 2, 3]).to_tuple()\nStream([1, 2, 3]).to_set()\n```\n\n- into dict\n```python\nclass Foo:\n def __init__(self, name, num):\n self.name = name\n self.num = num\n \nStream([Foo(\"fizz\", 1), Foo(\"buzz\", 2)]).to_dict(lambda x: (x.name, x.num))\n```\n```shell\n{\"fizz\": 1, \"buzz\": 2}\n```\n\nIn the case of a collision (duplicate keys) the 'merger' functions indicates which entry should be kept\n```python\ncollection = [Foo(\"fizz\", 1), Foo(\"fizz\", 2), Foo(\"buzz\", 2)]\nStream(collection).to_dict(collector=lambda x: (x.name, x.num), merger=lambda old, new: old)\n```\n```shell\n{\"fizz\": 1, \"buzz\": 2}\n```\n\n<i>to_dict</i> method also supports creating dictionaries from dict DictItem objects\n```python\nfirst_dict = {\"x\": 1, \"y\": 2}\nsecond_dict = {\"p\": 33, \"q\": 44, \"r\": None}\nStream(first_dict).concat(Stream(second_dict)).to_dict(lambda x: DictItem(x.key, x.value or 0)) \n```\n```shell\n{\"x\": 1, \"y\": 2, \"p\": 33, \"q\": 44, \"r\": 0}\n```\ne.g. you could combine streams of dicts by writing:\n```python\nStream(first_dict).concat(Stream(second_dict)).to_dict() \n```\n(simplified from <i>'.to_dict(lambda x: x)'</i>)\n\n- into string\n```python\nStream({\"a\": 1, \"b\": [2, 3]}).to_string()\n```\n```shell\n\"Stream(DictItem(key=a, value=1), DictItem(key=b, value=[2, 3]))\"\n```\n```python\nStream({\"a\": 1, \"b\": [2, 3]}).map(lambda x: {x.key: x.value}).to_string(delimiter=\" | \")\n```\n```shell\n\"Stream({'a': 1} | {'b': [2, 3]})\"\n```\n\n- alternative for working with collectors is using the <i>collect</i> method\n```python\nStream([1, 2, 3]).collect(tuple)\nStream.of(1, 2, 3).collect(list)\nStream.of(1, 1, 2, 2, 2, 3).collect(set)\nStream.of(1, 2, 3, 4).collect(dict, lambda x: (str(x), x * 10))\n```\n\n- grouping\n```python\nStream(\"AAAABBBCCD\").group_by(collector=lambda key, grouper: (key, len(grouper)))\n```\n```shell\n{\"A\": 4, \"B\": 3, \"C\": 2, \"D\": 1}\n```\n\n```python\ncoll = [Foo(\"fizz\", 1), Foo(\"fizz\", 2), Foo(\"fizz\", 3), Foo(\"buzz\", 2), Foo(\"buzz\", 3), Foo(\"buzz\", 4), Foo(\"buzz\", 5)]\nStream(coll).group_by(\n classifier=lambda obj: obj.name,\n collector=lambda key, grouper: (key, [(obj.name, obj.num) for obj in list(grouper)]))\n```\n```shell\n{\n \"fizz\": [(\"fizz\", 1), (\"fizz\", 2), (\"fizz\", 3)],\n \"buzz\": [(\"buzz\", 2), (\"buzz\", 3), (\"buzz\", 4), (\"buzz\", 5)],\n}\n```\n#### Other terminal operations\n- for_each\n```python\nStream([1, 2, 3, 4]).for_each(lambda x: print(f\"{'#' * x} \", end=\"\"))\n```\n\n- count\n<br>(returns the count of elements in the stream)\n```python\nStream([1, 2, 3, 4]).filter(lambda x: x % 2 == 0).count()\n```\n\n- sum\n```python\nStream.of(1, 2, 3, 4).sum() \n```\n\n- find_first\n<br>(search for an element of the stream that satisfies a predicate,\nreturns an Optional with the first found value, if any, or None)\n```python\nStream.of(1, 2, 3, 4).filter(lambda x: x % 2 == 0).find_first().get()\n```\n\n- find_any\n<br>(search for an element of the stream that satisfies a predicate,\nreturns an Optional with some of the found values, if any, or None)\n```python\nStream.of(1, 2, 3, 4).filter(lambda x: x % 2 == 0).find_any().get()\n```\n\n- any_match\n<br>(returns whether any elements of the stream match the given predicate)\n```python\nStream.of(1, 2, 3, 4).any_match(lambda x: x > 2)\n```\n\n- all_match\n<br>(returns whether all elements of the stream match the given predicate)\n```python\nStream.of(1, 2, 3, 4).all_match(lambda x: x > 2)\n```\n\n- none_match\n<br>(returns whether no elements of the stream match the given predicate)\n```python\nStream.of(1, 2, 3, 4).none_match(lambda x: x < 0)\n```\n\n- min\n<br>(returns Optional with the minimum element of the stream)\n```python\nStream.of(2, 1, 3, 4).min().get()\n```\n\n- max\n<br>(returns Optional with the maximum element of the stream)\n```python\nStream.of(2, 1, 3, 4).max().get()\n```\n\n- compare_with\n<br>(compares linearly the contents of two streams based on a given comparator)\n```python\nfizz = Foo(\"fizz\", 1)\nbuzz = Foo(\"buzz\", 2)\nStream([buzz, fizz]).compare_with(Stream([fizz, buzz]), lambda x, y: x.num == y.num)\n```\n\n- quantify\n<br>(count how many of the elements are Truthy or evaluate to True based on a given predicate)\n```python\nStream([2, 3, 4, 5, 6]).quantify(predicate=lambda x: x % 2 == 0)\n```\n--------------------------------------------\n### Itertools integration\nInvoke <i>use</i> method by passing the itertools function and it's arguments as **kwargs\n```python\nimport itertools\nimport operator\n\nStream([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]).use(itertools.islice, start=3, stop=8)\nStream.of(1, 2, 3, 4, 5).use(itertools.accumulate, func=operator.mul).to_list()\nStream(range(3)).use(itertools.permutations, r=3).to_list()\n\n```\n#### Itertools 'recipes'\nInvoke the 'recipes' described [here](https://docs.python.org/3/library/itertools.html#itertools-recipes) as stream methods and pass required key-word arguments\n```python\nStream([1, 2, 3]).ncycles(count=2).to_list()\nStream.of(2, 3, 4).take_nth(10, default=66).get()\nStream([\"ABC\", \"D\", \"EF\"]).round_robin().to_list()\n```\n--------------------------------------------\n### FileStreams\n#### Querying files\n- working with <i>json</i>, <i>toml</i>, <i>yaml</i>, <i>xml</i> files\n<br>NB: FileStream reads data as series of DictItem objects from underlying dict_items view\n```python\nFileStream(\"path/to/file\").map(lambda x: f\"{x.key}=>{x.value}\").to_tuple()\n```\n```shell\n(\n \"abc=>xyz\", \n \"qwerty=>42\",\n)\n```\n```python\nfrom operator import attrgetter\nfrom pyrio import DictItem\n\n(FileStream(\"path/to/file\")\n .filter(lambda x: \"a\" in x.key)\n .map(lambda x: DictItem(x.key, sum(x.value) * 10))\n .sort(attrgetter(\"value\"), reverse=True)\n .map(lambda x: f\"{str(x.value)}::{x.key}\")\n .to_list()) \n```\n```shell\n[\"230::xza\", \"110::abba\", \"30::a\"]\n```\n- querying <i>csv</i> and <i>tsv</i> files\n<br>(each row is read as a dict with keys taken from the header)\n```python\nFileStream(\"path/to/file\").map(lambda x: f\"fizz: {x['fizz']}, buzz: {x['buzz']}\").to_tuple() \n```\n```shell\n(\n \"fizz: 42, buzz: 45\",\n \"fizz: aaa, buzz: bbb\",\n)\n```\n```python\nfrom operator import itemgetter\n\nFileStream(\"path/to/file\").map(itemgetter('fizz', 'buzz')).to_tuple()\n```\n```shell\n(('42', '45'), ('aaa', 'bbb'))\n```\nYou could query the nested dicts by creating streams out of them\n```python\n(FileStream(\"path/to/file\")\n .map(lambda x: (Stream(x).to_dict(lambda y: DictItem(y.key, y.value or \"Unknown\"))))\n .save())\n```\n- reading a file with <i>process()</i> method\n - use extra <i>f_open_options</i> (for the underlying <i>open file</i> function)\n - <i>f_read_options</i> (to be passed to the corresponding library function that is loading the file content e.g. tomllib, json)\n```python\nfrom decimal import Decimal\n\n(FileStream.process(\n file_path=\"path/to/file.json\", \n f_open_options={\"encoding\": \"utf-8\"}, \n f_read_options={\"parse_float\": Decimal})\n .map(lambda x:x.value).to_list())\n```\n```shell\n['foo', True, Decimal('1.22'), Decimal('5.456367654369698986')]\n```\nTo include the <i>root</i> tag when loading an <i>.xml</i> file pass <i>'include_root=True'</i>\n```python\nFileStream.process(\"path/to/custom_root.xml\", include_root=True).map(\n lambda x: f\"root={x.key}: inner_records={str(x.value)}\"\n).to_list()\n```\n```shell\n[\"root=custom-root: inner_records={'abc': 'xyz', 'qwerty': '42'}\"]\n```\n--------------------------------------------\n#### Saving to a file\n- write the contents of a FileStream by passing a <i>file_path</i> to the <i>save()</i> method\n```python\nin_memory_dict = Stream(json_dict).filter(lambda x: len(x.key) < 6).to_tuple()\nFileStream(\"path/to/file.json\").prepend(in_memory_dict).save(\"./tests/resources/updated.json\")\n```\nIf no path is given, the source file for the FileStream will be updated\n```python\nFileStream(\"path/to/file.json\").concat(in_memory_dict).save()\n```\nNB: if while updating the file something goes wrong, the original content will be restored/preserved\n- handle null values\n<br>(pass <i>null_handler</i> function to replace null values)\n```python\nFileStream(\"path/to/test.toml\").save(null_handler=lambda x: DictItem(x.key, x.value or \"N/A\"))\n```\nNB: useful for writing <i>.toml</i> files which don't allow None values\n- passing advanced <i>file open</i> and <i>write</i> options\n<br>similarly to the <i>process</i> method you could provide \n - <i>f_open_options</i> (for the underlying <i>open</i> function)\n - <i>f_write_options</i> (passed to the corresponding library that will 'dump' the contents of the stream e.g. tomli-w, pyyaml)\n```python\nFileStream(\"path/to/file.json\").concat(in_memory_dict).save(\n file_path=\"merged.xml\",\n f_open_options={\"encoding\": \"utf-8\"},\n f_write_options={\"indent\": 4},\n)\n```\nTo add <i>custom root</i> tag when saving an <i>.xml</i> file pass <i>'xml_root=\"my-custom-root\"'</i>\n```python\nFileStream(\"path/to/file.json\").concat(in_memory_dict).save(\n file_path=\"path/to/custom.xml\",\n f_open_options={\"encoding\": \"utf-8\"},\n f_write_options={\"indent\": 4},\n xml_root=\"my-custom-root\",\n)\n```\n--------------------------------------------\n- how far can we actually push it?\n```python\n(\n FileStream(\"path/to/file.csv\")\n .concat(\n FileStream(\"path/to/other/file.json\")\n .filter(\n lambda x: (\n Stream(x.value)\n .find_first(lambda y: y.key == \"name\" and y.value != \"Snake\")\n .or_else_get(lambda: None)\n )\n is not None\n )\n .map(lambda x: x.value)\n )\n .map(lambda x: (Stream(x).to_dict(lambda y: DictItem(y.key, y.value or \"N/A\"))))\n .save(\"path/to/third/file.tsv\")\n)\n```\nor how hideous can it get?\n<p align=\"center\">\n <img src=\"https://github.com/kaliv0/pyrio/blob/main/assets/Chubby.jpg?raw=true\" width=\"400\" alt=\"Chubby\">\n</p>",
"bugtrack_url": null,
"license": null,
"summary": "Functional-style Streams library for processing collections. Supports querying files (json, toml, yaml, xml, csv, tsv) - as well as creating and updating them. Provides easy integration with itertools",
"version": "1.3.4",
"project_urls": {
"Documentation": "https://pyrio.readthedocs.io/en/latest",
"Homepage": "https://github.com/kaliv0/pyrio",
"Repository": "https://github.com/kaliv0/pyrio"
},
"split_keywords": [
"stream",
" functional",
" processing",
" collections",
" fluent api",
" file processing",
" queries"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "a02a97cc110bdb83262ed6525365b24ea9ba0aa498737714153c40ac8a772b93",
"md5": "c8df91e68c4b173421b43744cdd0c7e5",
"sha256": "eb6e0832e99b44b1c41f44e4070f79f40f93409c2ef8898c1132f77e5a0bbd06"
},
"downloads": -1,
"filename": "pyrio-1.3.4-py3-none-any.whl",
"has_sig": false,
"md5_digest": "c8df91e68c4b173421b43744cdd0c7e5",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.12",
"size": 19023,
"upload_time": "2024-12-08T11:19:17",
"upload_time_iso_8601": "2024-12-08T11:19:17.558450Z",
"url": "https://files.pythonhosted.org/packages/a0/2a/97cc110bdb83262ed6525365b24ea9ba0aa498737714153c40ac8a772b93/pyrio-1.3.4-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "6205e090cb70bb75431f1c9c93e896a5abc447d40c9f8e2e24680c5d0f2f8502",
"md5": "c6cdf843c9c906b90aa2528fc33130d2",
"sha256": "1897d6bdcec80fa0ece2025e694cd86da4552accdc2c596a73c0c59138a15322"
},
"downloads": -1,
"filename": "pyrio-1.3.4.tar.gz",
"has_sig": false,
"md5_digest": "c6cdf843c9c906b90aa2528fc33130d2",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.12",
"size": 20197,
"upload_time": "2024-12-08T11:19:19",
"upload_time_iso_8601": "2024-12-08T11:19:19.506661Z",
"url": "https://files.pythonhosted.org/packages/62/05/e090cb70bb75431f1c9c93e896a5abc447d40c9f8e2e24680c5d0f2f8502/pyrio-1.3.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-12-08 11:19:19",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "kaliv0",
"github_project": "pyrio",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "pyrio"
}
JSON
