A progress tracker with methods for throughput, ETA and update notification;
also a compound progress meter composed from other progress meters.
*Latest release 20241122*:
* BaseProgress.bar: default for report_print now comes from not BaseCommand.Options.quiet.
* BaseProgress.bar: always print the final report if report_print.
## <a name="auto_progressbar"></a>`auto_progressbar(*da, **dkw)`
Decorator for a function accepting an optional `progress`
keyword parameter.
If `progress` is `None` and the default `Upd` is not disabled,
run the function with a progress bar.
## <a name="BaseProgress"></a>Class `BaseProgress`
The base class for `Progress` and `OverProcess`
with various common methods.
Note that durations are in seconds
and that absolute time is in seconds since the UNIX epoch
(the basis of `time.time()`).
*`BaseProgress.__init__(self, name=None, start_time=None, units_scale=None)`*:
Initialise a progress instance.
Parameters:
* `name`: optional name
* `start_time`: optional UNIX epoch start time, default from `time.time()`
* `units_scale`: a scale for use with `cs.units.transcribe`,
default `BINARY_BYTES_SCALE`
*`BaseProgress.__eq__(self, other)`*:
A Progress is equal to another object `other`
if its position equals `int(other)`.
*`BaseProgress.__ge__(self, other, NotImplemented=NotImplemented)`*:
Return a >= b. Computed by @total_ordering from (not a < b).
*`BaseProgress.__gt__(self, other, NotImplemented=NotImplemented)`*:
Return a > b. Computed by @total_ordering from (not a < b) and (a != b).
*`BaseProgress.__int__(self)`*:
`int(Progress)` returns the current position.
*`BaseProgress.__le__(self, other, NotImplemented=NotImplemented)`*:
Return a <= b. Computed by @total_ordering from (a < b) or (a == b).
*`BaseProgress.__lt__(self, other)`*:
A Progress is less then another object `other`
if its position is less than `int(other)`.
*`BaseProgress.arrow(self, width, no_padding=False)`*:
Construct a progress arrow representing completion
to fit in the specified `width`.
*`BaseProgress.bar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at 0x10a711cf0>, **kw)`*:
A context manager to create and withdraw a progress bar.
It returns the `UpdProxy` which displays the progress bar.
Parameters:
* `label`: a label for the progress bar,
default from `self.name`.
* `statusfunc`: an optional function to compute the progress bar text
accepting `(self,label,width)`.
* `width`: an optional width expressing how wide the progress bar
text may be.
The default comes from the `proxy.width` property.
* `recent_window`: optional timeframe to define "recent" in seconds;
if the default `statusfunc` (`Progress.status`) is used
this is passed to it
* `report_print`: optional `print` compatible function
with which to write a report on completion;
this may also be a `bool`, which if true will use `Upd.print`
in order to interoperate with `Upd`.
* `stalled`: optional string to replace the word `'stalled'`
in the status line; for a worked this might be betteer as `'idle'`
* `insert_pos`: where to insert the progress bar, default `1`
* `poll`: an optional callable accepting a `BaseProgress`
which can be used to update the progress state before
updating the progress bar display
Example use:
# display progress reporting during upload_filename()
# which updates the supplied Progress instance
# during its operation
P = Progress(name=label)
with P.bar(report_print=True):
upload_filename(src, progress=P)
*`BaseProgress.elapsed_time`*:
Time elapsed since `start_time`.
*`BaseProgress.eta`*:
The projected time of completion: now + `remaining_time`.
If `remaining_time` is `None`, this is also `None`.
*`BaseProgress.format_counter(self, value, scale=None, max_parts=2, sep=',', **kw)`*:
Format `value` accoridng to `scale` and `max_parts`
using `cs.units.transcribe`.
*`BaseProgress.iterbar(self, it, label=None, *, itemlenfunc=None, incfirst=False, update_period=0.3, **bar_kw)`*:
An iterable progress bar: a generator yielding values
from the iterable `it` while updating a progress bar.
Parameters:
* `it`: the iterable to consume and yield.
* `label`: a label for the progress bar,
default from `self.name`.
* `itemlenfunc`: an optional function returning the "size" of each item
from `it`, used to advance `self.position`.
The default is to assume a size of `1`.
A convenient alternative choice may be the builtin function `len`.
* `incfirst`: whether to advance `self.position` before we
`yield` an item from `it` or afterwards.
This reflects whether it is considered that progress is
made as items are obtained or only after items are processed
by whatever is consuming this generator.
The default is `False`, advancing after processing.
* `update_period`: default `DEFAULT_UPDATE_PERIOD`; if `0`
then update on every iteration, otherwise every `update_period`
seconds
Other parameters are passed to `Progress.bar`.
Example use:
from cs.units import DECIMAL_SCALE
rows = [some list of data]
P = Progress(total=len(rows), units_scale=DECIMAL_SCALE)
for row in P.iterbar(rows, incfirst=True):
... do something with each row ...
f = open(data_filename, 'rb')
datalen = os.stat(f).st_size
def readfrom(f):
while True:
bs = f.read(65536)
if not bs:
break
yield bs
P = Progress(total=datalen)
for bs in P.iterbar(readfrom(f), itemlenfunc=len):
... process the file data in bs ...
*`BaseProgress.ratio`*:
The fraction of progress completed: `(position-start)/(total-start)`.
Returns `None` if `total` is `None` or `total<=start`.
Example:
>>> P = Progress()
P.ratio
>>> P.total = 16
>>> P.ratio
0.0
>>> P.update(4)
>>> P.ratio
0.25
*`BaseProgress.remaining_time`*:
The projected time remaining to end
based on the `throughput` and `total`.
If `total` is `None`, this is `None`.
*`BaseProgress.status(self, label, width, recent_window=None, stalled=None)`*:
A progress string of the form:
*label*`: `*pos*`/`*total*` ==> ETA '*time*
Parameters:
* `label`: the label for the status line;
if `None` use `self.name`
* `width`: the available width for the status line;
if not an `int` use `width.width`
* `recent_window`: optional timeframe to define "recent" in seconds,
default : `5`
* `stalled`: the label to indicate no throughput, default `'stalled'`;
for a worker this might often b better as `'idle'`
*`BaseProgress.text_pos_of_total(self, fmt=None, fmt_pos=None, fmt_total=None, pos_first=False)`*:
Return a "total:position" or "position/total" style progress string.
Parameters:
* `fmt`: format string interpolating `pos_text` and `total_text`.
Default: `"{pos_text}/{total_text}"` if `pos_first`,
otherwise `"{total_text}:{pos_text}"`
* `fmt_pos`: formatting function for `self.position`,
default `self.format_counter`
* `fmt_total`: formatting function for `self.total`,
default from `fmt_pos`
* `pos_first`: put the position first if true (default `False`),
only consulted if `fmt` is `None`
*`BaseProgress.throughput`*:
The overall throughput: `self.throughput_overall()`.
By comparison,
the `Progress.throughput` property is `self.throughput_recent`
if the `throughput_window` is not `None`,
otherwise it falls back to `throughput_overall`.
*`BaseProgress.throughput_overall(self)`*:
The overall throughput from `start` to `position`
during `elapsed_time`.
*`BaseProgress.throughput_recent(self, time_window)`*:
The recent throughput. Implemented by subclasses.
## <a name="CheckPoint"></a>Class `CheckPoint(builtins.tuple)`
CheckPoint(time, position)
*`CheckPoint.position`*:
Alias for field number 1
*`CheckPoint.time`*:
Alias for field number 0
## <a name="OverProgress"></a>Class `OverProgress(BaseProgress)`
A `Progress`-like class computed from a set of subsidiary `Progress`es.
AN OverProgress instance has an attribute ``notify_update`` which
is a set of callables.
Whenever the position of a subsidiary `Progress` is updated,
each of these will be called with the `Progress` instance and `None`.
Example:
>>> P = OverProgress(name="over")
>>> P1 = Progress(name="progress1", position=12)
>>> P1.total = 100
>>> P1.advance(7)
>>> P2 = Progress(name="progress2", position=20)
>>> P2.total = 50
>>> P2.advance(9)
>>> P.add(P1)
>>> P.add(P2)
>>> P1.total
100
>>> P2.total
50
>>> P.total
150
>>> P1.start
12
>>> P2.start
20
>>> P.start
0
>>> P1.position
19
>>> P2.position
29
>>> P.position
16
*`OverProgress.add(self, subprogress)`*:
Add a subsidairy `Progress` to the contributing set.
*`OverProgress.eta`*:
The `eta` is the maximum of the subsidiary etas.
*`OverProgress.position`*:
The `position` is the sum off the subsidiary position offsets
from their respective starts.
*`OverProgress.remove(self, subprogress, accrue=False)`*:
Remove a subsidairy `Progress` from the contributing set.
*`OverProgress.start`*:
We always return a starting value of 0.
*`OverProgress.throughput`*:
The `throughput` is the sum of the subsidiary throughputs.
*`OverProgress.throughput_recent(self, time_window)`*:
The `throughput_recent` is the sum of the subsidiary throughput_recentss.
*`OverProgress.total`*:
The `total` is the sum of the subsidiary totals.
## <a name="Progress"></a>Class `Progress(BaseProgress)`
A progress counter to track task completion with various utility methods.
Example:
>>> P = Progress(name="example")
>>> P #doctest: +ELLIPSIS
Progress(name='example',start=0,position=0,start_time=...,throughput_window=None,total=None):[CheckPoint(time=..., position=0)]
>>> P.advance(5)
>>> P #doctest: +ELLIPSIS
Progress(name='example',start=0,position=5,start_time=...,throughput_window=None,total=None):[CheckPoint(time=..., position=0), CheckPoint(time=..., position=5)]
>>> P.total = 100
>>> P #doctest: +ELLIPSIS
Progress(name='example',start=0,position=5,start_time=...,throughput_window=None,total=100):[CheckPoint(time=..., position=0), CheckPoint(time=..., position=5)]
A Progress instance has an attribute ``notify_update`` which
is a set of callables. Whenever the position is updated, each
of these will be called with the `Progress` instance and the
latest `CheckPoint`.
`Progress` objects also make a small pretense of being an integer.
The expression `int(progress)` returns the current position,
and `+=` and `-=` adjust the position.
This is convenient for coding, but importantly it is also
useful for discretionary use of a Progress with some other
object.
If you want to make a lightweight `Progress` capable class
you can set a position attribute to an `int`
and manipulate it carefully using `+=` and `-=` entirely.
If you decide to incur the cost of maintaining a `Progress` object
you can slot it in:
# initial setup with just an int
my_thing.amount = 0
# later, or on some option, use a Progress instance
my_thing.amount = Progress(my_thing.amount)
*`Progress.__init__(self, name: Optional[str] = None, *, position: Optional[float] = None, start: Optional[float] = None, start_time: Optional[float] = None, throughput_window: Optional[int] = None, total: Optional[float] = None, units_scale=None)`*:
Initialise the Progesss object.
Parameters:
* `position`: initial position, default `0`.
* `name`: optional name for this instance.
* `start`: starting position of progress range,
default from `position`.
* `start_time`: start time of the process, default now.
* `throughput_window`: length of throughput time window in seconds,
default None.
* `total`: expected completion value, default None.
*`Progress.__iadd__(self, delta)`*:
Operator += form of advance().
>>> P = Progress()
>>> P.position
0
>>> P += 4
>>> P.position
4
>>> P += 4
>>> P.position
8
*`Progress.__isub__(self, delta)`*:
Operator -= form of advance().
>>> P = Progress()
>>> P.position
0
>>> P += 4
>>> P.position
4
>>> P -= 4
>>> P.position
0
*`Progress.advance(self, delta, update_time=None)`*:
Record more progress, return the advanced position.
>>> P = Progress()
>>> P.position
0
>>> P.advance(4)
>>> P.position
4
>>> P.advance(4)
>>> P.position
8
*`Progress.advance_total(self, delta)`*:
Function form of addition to the total.
*`Progress.latest`*:
Latest datum.
*`Progress.position`*:
Latest position.
*`Progress.throughput`*:
Current throughput per second.
If `self.throughput_window` is not `None`,
calls `self.throughput_recent(throughput_window)`.
Otherwise call `self.throughput_overall()`.
*`Progress.throughput_recent(self, time_window)`*:
Recent throughput per second within a time window in seconds.
The time span overlapping the start of the window is included
on a flat pro rata basis.
*`Progress.total`*:
Return the current total.
*`Progress.update(self, new_position, update_time=None)`*:
Record more progress.
>>> P = Progress()
>>> P.position
0
>>> P.update(12)
>>> P.position
12
## <a name="progressbar"></a>`progressbar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at 0x10a7135b0>, **kw)`
Convenience function to construct and run a `Progress.iterbar`
wrapping the iterable `it`,
issuing and withdrawning a progress bar during the iteration.
Parameters:
* `it`: the iterable to consume
* `label`: optional label, doubles as the `Progress.name`
* `position`: optional starting position
* `total`: optional value for `Progress.total`,
default from `len(it)` if supported.
* `units_scale`: optional units scale for `Progress`,
default `UNSCALED_SCALE`
If `total` is `None` and `it` supports `len()`
then the `Progress.total` is set from it.
All arguments are passed through to `Progress.iterbar`.
Example use:
for row in progressbar(rows):
... do something with row ...
## <a name="selftest"></a>`selftest(argv)`
Exercise some of the functionality.
# Release Log
*Release 20241122*:
* BaseProgress.bar: default for report_print now comes from not BaseCommand.Options.quiet.
* BaseProgress.bar: always print the final report if report_print.
*Release 20240412*:
* BaseProgress.status: fixes for the arrow_width computation.
* BaseProgress.bar: drop existing UpdProxy support, drop deferred, implement update_period using a ticker Thread.
* BaseProgress.bar: set update_period=DEFAULT_UPDATE_PERIOD by default.
* Progress.iterbar: drop preexisting UpdProxy support, update_frequency and update_min_size support.
* Progress: new advance_total(delta) method so that we have a callable for this.
* BaseProgress.bar: new optional poll parameter accepting a callable accepting a BaseProgress to update the state before updating the bar display.
* BaseProgress.bar: new stalled='stalled' parameter to specify the term for no recent throughput, workers might prefer 'idle'.
* progressbar() updated to match.
*Release 20230401*:
progressbar, BaseProgress.iterbar: use @uses_upd to provide a context Upd instance.
*Release 20230212*:
BaseProgress: new update_period=0.2 parameter to constraint updates by elapsed time.
*Release 20221207*:
* BaseProgress.format_counter: accept arbitrary keyword arguments to pass to cs.units.transcribe.
* Progress.__init__: accept floats instead of just ints.
*Release 20220918*:
Progress.iterbar: wrap the iteration in a try/finally for cleanup.
*Release 20211208*:
* Progress.__init__: make the first optional positional parameter be "name", make other parameters keyword only.
* Progress.bar: make "label" the first optional positional parameter, make others keyword only.
*Release 20210803*:
* progressbar,iterbar: accept optional RunState to cancel iteration.
* BaseProgress.iterbar: make update_min_size properly optional, was making update_frequency ineffective.
*Release 20210730*:
When there is no total just report position and no ETA.
*Release 20210717*:
Minor tweaks.
*Release 20210316*:
* Progress.iterbar: only update the status line once per iteration, either before or after the yield according to incfirst.
* Progress.iterbar: fix the meaning of update_frequency to count iterations, add update_min_size to count progress advance.
*Release 20210306*:
progressbar: accept new optional `position` parameter, used to initialise the Progress.
*Release 20201102.1*:
DISTINFO: fix module dependencies.
*Release 20201102*:
* Format/layout changes for the default status line.
* Progress.throughtput_recent: return None if no new positions beyond the starting position.
* BaseProgress.status: accept label=None (default to self.name) and width=UpdProxy (uses width.width).
* BaseProgress.status: new optional window parameter, default 5, defining the recent throughput window size in seconds.
* A few bugfixes.
*Release 20201025*:
* Some formatting improvements.
* BaseProgress.bar: new insert_pos parameter to position the progress bar, default still 1.
* BaseProgress.bar: new deferred parameter putting off the status bar until the first update.
* BaseProgress.bar: accept new optional `proxy` parameter to use (and not delete) an existing UpdProxy for display.
* Progress.text_pos_of_total: new `pos_first=False` parameter, rendering the total before the position by default (less progress bar noise).
* New @auto_progressbar decorator to provide a progress bar and initialise progress= parameter to functions which can use a Progress for reporting.
* Assorted fixes.
*Release 20200718.3*:
BaseProgress.bar, progressbar: new optional report_print parameter for reporting on completion.
*Release 20200718.2*:
Bugfix: BaseProgress.status: handle throughput=0 when total=None.
*Release 20200718.1*:
BaseProgress.bar, progressbar: new optional update_frequency parameter for less frequent updates.
*Release 20200718*:
* Readability improvement for default status line.
* progressbar: default units_scale=UNSCALED_SCALE.
*Release 20200716.1*:
BaseProgress.status: round throughput to an int if >=10.
*Release 20200716*:
* BaseProgress.status: distinguish "idle" (position >= total) from "stalled" (position < total).
* BaseProgress.status: make the status very short if the progress is idle.
*Release 20200627*:
* BaseProgress.status: handle throughput=None (before any activity).
* BaseProgress: drop count_of_total_bytes_text, superceded by format_counter (which honours the units_scale).
*Release 20200626*:
* New Progress.bar generator method iterating over an iterable while displaying a progress bar.
* New convenience function progressbar(it,...) which rolls its own Progress instance.
* Progress: always support a throughput window, default to DEFAULT_THROUGHPUT_WINDOW = 5s.
* Improve the default progress bar render returned by Progress.status().
*Release 20200613*:
* BaseProgress, Progress and OverProgress now accept an optional units_scale, such as cs.units.UNSCALED_SCALE, to use when expressing progress - the default remains BINARY_SCALE.
* New arrow(), format_counter() and text_pos_of_total() methods to produce components of the status string for tuning or external reuse.
*Release 20200520*:
OverProgress: throughput and eta implementations.
*Release 20200129.3*:
Test __version__ machinery again.
*Release 20200129.2*:
set __version__ to '20200129.2'
*Release 20200129.1*:
Dummy release to test new __version__.
*Release 20200129*:
New Progress.count_of_total_bytes_text property presenting "3kB/40MB" style text.
*Release 20190812*:
* New OverProgress class which is a composite of a set of subsidiary Progress instances.
* Assorted other small updates.
*Release 20190220*:
* Progress: be somewhat like an int.
* New status() method returning a convenient one line progress status report.
*Release 20180703.2*:
Progress: make .total into a property in order to fire the update notifications.
*Release 20180703.1*:
Progress: additions and changes to API: new .ratio, .elapsed_time, rename .projected to .remaining_time.
*Release 20180703*:
Initial release of cs.progress.
Raw data
{
"_id": null,
"home_page": null,
"name": "cs-progress",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "python2, python3",
"author": null,
"author_email": "Cameron Simpson <cs@cskk.id.au>",
"download_url": "https://files.pythonhosted.org/packages/74/b9/3e7094961ad01e0acb216ee676eac2c3c34591b1819f92b9105614e2461b/cs_progress-20241122.tar.gz",
"platform": null,
"description": "A progress tracker with methods for throughput, ETA and update notification;\nalso a compound progress meter composed from other progress meters.\n\n*Latest release 20241122*:\n* BaseProgress.bar: default for report_print now comes from not BaseCommand.Options.quiet.\n* BaseProgress.bar: always print the final report if report_print.\n\n## <a name=\"auto_progressbar\"></a>`auto_progressbar(*da, **dkw)`\n\nDecorator for a function accepting an optional `progress`\nkeyword parameter.\nIf `progress` is `None` and the default `Upd` is not disabled,\nrun the function with a progress bar.\n\n## <a name=\"BaseProgress\"></a>Class `BaseProgress`\n\nThe base class for `Progress` and `OverProcess`\nwith various common methods.\n\nNote that durations are in seconds\nand that absolute time is in seconds since the UNIX epoch\n(the basis of `time.time()`).\n\n*`BaseProgress.__init__(self, name=None, start_time=None, units_scale=None)`*:\nInitialise a progress instance.\n\nParameters:\n* `name`: optional name\n* `start_time`: optional UNIX epoch start time, default from `time.time()`\n* `units_scale`: a scale for use with `cs.units.transcribe`,\n default `BINARY_BYTES_SCALE`\n\n*`BaseProgress.__eq__(self, other)`*:\nA Progress is equal to another object `other`\nif its position equals `int(other)`.\n\n*`BaseProgress.__ge__(self, other, NotImplemented=NotImplemented)`*:\nReturn a >= b. Computed by @total_ordering from (not a < b).\n\n*`BaseProgress.__gt__(self, other, NotImplemented=NotImplemented)`*:\nReturn a > b. Computed by @total_ordering from (not a < b) and (a != b).\n\n*`BaseProgress.__int__(self)`*:\n`int(Progress)` returns the current position.\n\n*`BaseProgress.__le__(self, other, NotImplemented=NotImplemented)`*:\nReturn a <= b. Computed by @total_ordering from (a < b) or (a == b).\n\n*`BaseProgress.__lt__(self, other)`*:\nA Progress is less then another object `other`\nif its position is less than `int(other)`.\n\n*`BaseProgress.arrow(self, width, no_padding=False)`*:\nConstruct a progress arrow representing completion\nto fit in the specified `width`.\n\n*`BaseProgress.bar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at 0x10a711cf0>, **kw)`*:\nA context manager to create and withdraw a progress bar.\nIt returns the `UpdProxy` which displays the progress bar.\n\nParameters:\n* `label`: a label for the progress bar,\n default from `self.name`.\n* `statusfunc`: an optional function to compute the progress bar text\n accepting `(self,label,width)`.\n* `width`: an optional width expressing how wide the progress bar\n text may be.\n The default comes from the `proxy.width` property.\n* `recent_window`: optional timeframe to define \"recent\" in seconds;\n if the default `statusfunc` (`Progress.status`) is used\n this is passed to it\n* `report_print`: optional `print` compatible function\n with which to write a report on completion;\n this may also be a `bool`, which if true will use `Upd.print`\n in order to interoperate with `Upd`.\n* `stalled`: optional string to replace the word `'stalled'`\n in the status line; for a worked this might be betteer as `'idle'`\n* `insert_pos`: where to insert the progress bar, default `1`\n* `poll`: an optional callable accepting a `BaseProgress`\n which can be used to update the progress state before\n updating the progress bar display\n\nExample use:\n\n # display progress reporting during upload_filename()\n # which updates the supplied Progress instance\n # during its operation\n P = Progress(name=label)\n with P.bar(report_print=True):\n upload_filename(src, progress=P)\n\n*`BaseProgress.elapsed_time`*:\nTime elapsed since `start_time`.\n\n*`BaseProgress.eta`*:\nThe projected time of completion: now + `remaining_time`.\n\nIf `remaining_time` is `None`, this is also `None`.\n\n*`BaseProgress.format_counter(self, value, scale=None, max_parts=2, sep=',', **kw)`*:\nFormat `value` accoridng to `scale` and `max_parts`\nusing `cs.units.transcribe`.\n\n*`BaseProgress.iterbar(self, it, label=None, *, itemlenfunc=None, incfirst=False, update_period=0.3, **bar_kw)`*:\nAn iterable progress bar: a generator yielding values\nfrom the iterable `it` while updating a progress bar.\n\nParameters:\n* `it`: the iterable to consume and yield.\n* `label`: a label for the progress bar,\n default from `self.name`.\n* `itemlenfunc`: an optional function returning the \"size\" of each item\n from `it`, used to advance `self.position`.\n The default is to assume a size of `1`.\n A convenient alternative choice may be the builtin function `len`.\n* `incfirst`: whether to advance `self.position` before we\n `yield` an item from `it` or afterwards.\n This reflects whether it is considered that progress is\n made as items are obtained or only after items are processed\n by whatever is consuming this generator.\n The default is `False`, advancing after processing.\n* `update_period`: default `DEFAULT_UPDATE_PERIOD`; if `0`\n then update on every iteration, otherwise every `update_period`\n seconds\nOther parameters are passed to `Progress.bar`.\n\nExample use:\n\n from cs.units import DECIMAL_SCALE\n rows = [some list of data]\n P = Progress(total=len(rows), units_scale=DECIMAL_SCALE)\n for row in P.iterbar(rows, incfirst=True):\n ... do something with each row ...\n\n f = open(data_filename, 'rb')\n datalen = os.stat(f).st_size\n def readfrom(f):\n while True:\n bs = f.read(65536)\n if not bs:\n break\n yield bs\n P = Progress(total=datalen)\n for bs in P.iterbar(readfrom(f), itemlenfunc=len):\n ... process the file data in bs ...\n\n*`BaseProgress.ratio`*:\nThe fraction of progress completed: `(position-start)/(total-start)`.\nReturns `None` if `total` is `None` or `total<=start`.\n\nExample:\n\n >>> P = Progress()\n P.ratio\n >>> P.total = 16\n >>> P.ratio\n 0.0\n >>> P.update(4)\n >>> P.ratio\n 0.25\n\n*`BaseProgress.remaining_time`*:\nThe projected time remaining to end\nbased on the `throughput` and `total`.\n\nIf `total` is `None`, this is `None`.\n\n*`BaseProgress.status(self, label, width, recent_window=None, stalled=None)`*:\nA progress string of the form:\n*label*`: `*pos*`/`*total*` ==> ETA '*time*\n\nParameters:\n* `label`: the label for the status line;\n if `None` use `self.name`\n* `width`: the available width for the status line;\n if not an `int` use `width.width`\n* `recent_window`: optional timeframe to define \"recent\" in seconds,\n default : `5`\n* `stalled`: the label to indicate no throughput, default `'stalled'`;\n for a worker this might often b better as `'idle'`\n\n*`BaseProgress.text_pos_of_total(self, fmt=None, fmt_pos=None, fmt_total=None, pos_first=False)`*:\nReturn a \"total:position\" or \"position/total\" style progress string.\n\nParameters:\n* `fmt`: format string interpolating `pos_text` and `total_text`.\n Default: `\"{pos_text}/{total_text}\"` if `pos_first`,\n otherwise `\"{total_text}:{pos_text}\"`\n* `fmt_pos`: formatting function for `self.position`,\n default `self.format_counter`\n* `fmt_total`: formatting function for `self.total`,\n default from `fmt_pos`\n* `pos_first`: put the position first if true (default `False`),\n only consulted if `fmt` is `None`\n\n*`BaseProgress.throughput`*:\nThe overall throughput: `self.throughput_overall()`.\n\nBy comparison,\nthe `Progress.throughput` property is `self.throughput_recent`\nif the `throughput_window` is not `None`,\notherwise it falls back to `throughput_overall`.\n\n*`BaseProgress.throughput_overall(self)`*:\nThe overall throughput from `start` to `position`\nduring `elapsed_time`.\n\n*`BaseProgress.throughput_recent(self, time_window)`*:\nThe recent throughput. Implemented by subclasses.\n\n## <a name=\"CheckPoint\"></a>Class `CheckPoint(builtins.tuple)`\n\nCheckPoint(time, position)\n\n*`CheckPoint.position`*:\nAlias for field number 1\n\n*`CheckPoint.time`*:\nAlias for field number 0\n\n## <a name=\"OverProgress\"></a>Class `OverProgress(BaseProgress)`\n\nA `Progress`-like class computed from a set of subsidiary `Progress`es.\n\nAN OverProgress instance has an attribute ``notify_update`` which\nis a set of callables.\nWhenever the position of a subsidiary `Progress` is updated,\neach of these will be called with the `Progress` instance and `None`.\n\nExample:\n\n >>> P = OverProgress(name=\"over\")\n >>> P1 = Progress(name=\"progress1\", position=12)\n >>> P1.total = 100\n >>> P1.advance(7)\n >>> P2 = Progress(name=\"progress2\", position=20)\n >>> P2.total = 50\n >>> P2.advance(9)\n >>> P.add(P1)\n >>> P.add(P2)\n >>> P1.total\n 100\n >>> P2.total\n 50\n >>> P.total\n 150\n >>> P1.start\n 12\n >>> P2.start\n 20\n >>> P.start\n 0\n >>> P1.position\n 19\n >>> P2.position\n 29\n >>> P.position\n 16\n\n*`OverProgress.add(self, subprogress)`*:\nAdd a subsidairy `Progress` to the contributing set.\n\n*`OverProgress.eta`*:\nThe `eta` is the maximum of the subsidiary etas.\n\n*`OverProgress.position`*:\nThe `position` is the sum off the subsidiary position offsets\nfrom their respective starts.\n\n*`OverProgress.remove(self, subprogress, accrue=False)`*:\nRemove a subsidairy `Progress` from the contributing set.\n\n*`OverProgress.start`*:\nWe always return a starting value of 0.\n\n*`OverProgress.throughput`*:\nThe `throughput` is the sum of the subsidiary throughputs.\n\n*`OverProgress.throughput_recent(self, time_window)`*:\nThe `throughput_recent` is the sum of the subsidiary throughput_recentss.\n\n*`OverProgress.total`*:\nThe `total` is the sum of the subsidiary totals.\n\n## <a name=\"Progress\"></a>Class `Progress(BaseProgress)`\n\nA progress counter to track task completion with various utility methods.\n\nExample:\n\n >>> P = Progress(name=\"example\")\n >>> P #doctest: +ELLIPSIS\n Progress(name='example',start=0,position=0,start_time=...,throughput_window=None,total=None):[CheckPoint(time=..., position=0)]\n >>> P.advance(5)\n >>> P #doctest: +ELLIPSIS\n Progress(name='example',start=0,position=5,start_time=...,throughput_window=None,total=None):[CheckPoint(time=..., position=0), CheckPoint(time=..., position=5)]\n >>> P.total = 100\n >>> P #doctest: +ELLIPSIS\n Progress(name='example',start=0,position=5,start_time=...,throughput_window=None,total=100):[CheckPoint(time=..., position=0), CheckPoint(time=..., position=5)]\n\nA Progress instance has an attribute ``notify_update`` which\nis a set of callables. Whenever the position is updated, each\nof these will be called with the `Progress` instance and the\nlatest `CheckPoint`.\n\n`Progress` objects also make a small pretense of being an integer.\nThe expression `int(progress)` returns the current position,\nand `+=` and `-=` adjust the position.\n\nThis is convenient for coding, but importantly it is also\nuseful for discretionary use of a Progress with some other\nobject.\nIf you want to make a lightweight `Progress` capable class\nyou can set a position attribute to an `int`\nand manipulate it carefully using `+=` and `-=` entirely.\nIf you decide to incur the cost of maintaining a `Progress` object\nyou can slot it in:\n\n # initial setup with just an int\n my_thing.amount = 0\n\n # later, or on some option, use a Progress instance\n my_thing.amount = Progress(my_thing.amount)\n\n*`Progress.__init__(self, name: Optional[str] = None, *, position: Optional[float] = None, start: Optional[float] = None, start_time: Optional[float] = None, throughput_window: Optional[int] = None, total: Optional[float] = None, units_scale=None)`*:\nInitialise the Progesss object.\n\nParameters:\n* `position`: initial position, default `0`.\n* `name`: optional name for this instance.\n* `start`: starting position of progress range,\n default from `position`.\n* `start_time`: start time of the process, default now.\n* `throughput_window`: length of throughput time window in seconds,\n default None.\n* `total`: expected completion value, default None.\n\n*`Progress.__iadd__(self, delta)`*:\nOperator += form of advance().\n\n>>> P = Progress()\n>>> P.position\n0\n>>> P += 4\n>>> P.position\n4\n>>> P += 4\n>>> P.position\n8\n\n*`Progress.__isub__(self, delta)`*:\nOperator -= form of advance().\n\n>>> P = Progress()\n>>> P.position\n0\n>>> P += 4\n>>> P.position\n4\n>>> P -= 4\n>>> P.position\n0\n\n*`Progress.advance(self, delta, update_time=None)`*:\nRecord more progress, return the advanced position.\n\n>>> P = Progress()\n>>> P.position\n0\n>>> P.advance(4)\n>>> P.position\n4\n>>> P.advance(4)\n>>> P.position\n8\n\n*`Progress.advance_total(self, delta)`*:\nFunction form of addition to the total.\n\n*`Progress.latest`*:\nLatest datum.\n\n*`Progress.position`*:\nLatest position.\n\n*`Progress.throughput`*:\nCurrent throughput per second.\n\nIf `self.throughput_window` is not `None`,\ncalls `self.throughput_recent(throughput_window)`.\nOtherwise call `self.throughput_overall()`.\n\n*`Progress.throughput_recent(self, time_window)`*:\nRecent throughput per second within a time window in seconds.\n\nThe time span overlapping the start of the window is included\non a flat pro rata basis.\n\n*`Progress.total`*:\nReturn the current total.\n\n*`Progress.update(self, new_position, update_time=None)`*:\nRecord more progress.\n\n>>> P = Progress()\n>>> P.position\n0\n>>> P.update(12)\n>>> P.position\n12\n\n## <a name=\"progressbar\"></a>`progressbar(*a, upd: Optional[cs.upd.Upd] = <function uses_upd.<locals>.<lambda> at 0x10a7135b0>, **kw)`\n\nConvenience function to construct and run a `Progress.iterbar`\nwrapping the iterable `it`,\nissuing and withdrawning a progress bar during the iteration.\n\nParameters:\n* `it`: the iterable to consume\n* `label`: optional label, doubles as the `Progress.name`\n* `position`: optional starting position\n* `total`: optional value for `Progress.total`,\n default from `len(it)` if supported.\n* `units_scale`: optional units scale for `Progress`,\n default `UNSCALED_SCALE`\n\nIf `total` is `None` and `it` supports `len()`\nthen the `Progress.total` is set from it.\n\nAll arguments are passed through to `Progress.iterbar`.\n\nExample use:\n\n for row in progressbar(rows):\n ... do something with row ...\n\n## <a name=\"selftest\"></a>`selftest(argv)`\n\nExercise some of the functionality.\n\n# Release Log\n\n\n\n*Release 20241122*:\n* BaseProgress.bar: default for report_print now comes from not BaseCommand.Options.quiet.\n* BaseProgress.bar: always print the final report if report_print.\n\n*Release 20240412*:\n* BaseProgress.status: fixes for the arrow_width computation.\n* BaseProgress.bar: drop existing UpdProxy support, drop deferred, implement update_period using a ticker Thread.\n* BaseProgress.bar: set update_period=DEFAULT_UPDATE_PERIOD by default.\n* Progress.iterbar: drop preexisting UpdProxy support, update_frequency and update_min_size support.\n* Progress: new advance_total(delta) method so that we have a callable for this.\n* BaseProgress.bar: new optional poll parameter accepting a callable accepting a BaseProgress to update the state before updating the bar display.\n* BaseProgress.bar: new stalled='stalled' parameter to specify the term for no recent throughput, workers might prefer 'idle'.\n* progressbar() updated to match.\n\n*Release 20230401*:\nprogressbar, BaseProgress.iterbar: use @uses_upd to provide a context Upd instance.\n\n*Release 20230212*:\nBaseProgress: new update_period=0.2 parameter to constraint updates by elapsed time.\n\n*Release 20221207*:\n* BaseProgress.format_counter: accept arbitrary keyword arguments to pass to cs.units.transcribe.\n* Progress.__init__: accept floats instead of just ints.\n\n*Release 20220918*:\nProgress.iterbar: wrap the iteration in a try/finally for cleanup.\n\n*Release 20211208*:\n* Progress.__init__: make the first optional positional parameter be \"name\", make other parameters keyword only.\n* Progress.bar: make \"label\" the first optional positional parameter, make others keyword only.\n\n*Release 20210803*:\n* progressbar,iterbar: accept optional RunState to cancel iteration.\n* BaseProgress.iterbar: make update_min_size properly optional, was making update_frequency ineffective.\n\n*Release 20210730*:\nWhen there is no total just report position and no ETA.\n\n*Release 20210717*:\nMinor tweaks.\n\n*Release 20210316*:\n* Progress.iterbar: only update the status line once per iteration, either before or after the yield according to incfirst.\n* Progress.iterbar: fix the meaning of update_frequency to count iterations, add update_min_size to count progress advance.\n\n*Release 20210306*:\nprogressbar: accept new optional `position` parameter, used to initialise the Progress.\n\n*Release 20201102.1*:\nDISTINFO: fix module dependencies.\n\n*Release 20201102*:\n* Format/layout changes for the default status line.\n* Progress.throughtput_recent: return None if no new positions beyond the starting position.\n* BaseProgress.status: accept label=None (default to self.name) and width=UpdProxy (uses width.width).\n* BaseProgress.status: new optional window parameter, default 5, defining the recent throughput window size in seconds.\n* A few bugfixes.\n\n*Release 20201025*:\n* Some formatting improvements.\n* BaseProgress.bar: new insert_pos parameter to position the progress bar, default still 1.\n* BaseProgress.bar: new deferred parameter putting off the status bar until the first update.\n* BaseProgress.bar: accept new optional `proxy` parameter to use (and not delete) an existing UpdProxy for display.\n* Progress.text_pos_of_total: new `pos_first=False` parameter, rendering the total before the position by default (less progress bar noise).\n* New @auto_progressbar decorator to provide a progress bar and initialise progress= parameter to functions which can use a Progress for reporting.\n* Assorted fixes.\n\n*Release 20200718.3*:\nBaseProgress.bar, progressbar: new optional report_print parameter for reporting on completion.\n\n*Release 20200718.2*:\nBugfix: BaseProgress.status: handle throughput=0 when total=None.\n\n*Release 20200718.1*:\nBaseProgress.bar, progressbar: new optional update_frequency parameter for less frequent updates.\n\n*Release 20200718*:\n* Readability improvement for default status line.\n* progressbar: default units_scale=UNSCALED_SCALE.\n\n*Release 20200716.1*:\nBaseProgress.status: round throughput to an int if >=10.\n\n*Release 20200716*:\n* BaseProgress.status: distinguish \"idle\" (position >= total) from \"stalled\" (position < total).\n* BaseProgress.status: make the status very short if the progress is idle.\n\n*Release 20200627*:\n* BaseProgress.status: handle throughput=None (before any activity).\n* BaseProgress: drop count_of_total_bytes_text, superceded by format_counter (which honours the units_scale).\n\n*Release 20200626*:\n* New Progress.bar generator method iterating over an iterable while displaying a progress bar.\n* New convenience function progressbar(it,...) which rolls its own Progress instance.\n* Progress: always support a throughput window, default to DEFAULT_THROUGHPUT_WINDOW = 5s.\n* Improve the default progress bar render returned by Progress.status().\n\n*Release 20200613*:\n* BaseProgress, Progress and OverProgress now accept an optional units_scale, such as cs.units.UNSCALED_SCALE, to use when expressing progress - the default remains BINARY_SCALE.\n* New arrow(), format_counter() and text_pos_of_total() methods to produce components of the status string for tuning or external reuse.\n\n*Release 20200520*:\nOverProgress: throughput and eta implementations.\n\n*Release 20200129.3*:\nTest __version__ machinery again.\n\n*Release 20200129.2*:\nset __version__ to '20200129.2'\n\n*Release 20200129.1*:\nDummy release to test new __version__.\n\n*Release 20200129*:\nNew Progress.count_of_total_bytes_text property presenting \"3kB/40MB\" style text.\n\n*Release 20190812*:\n* New OverProgress class which is a composite of a set of subsidiary Progress instances.\n* Assorted other small updates.\n\n*Release 20190220*:\n* Progress: be somewhat like an int.\n* New status() method returning a convenient one line progress status report.\n\n*Release 20180703.2*:\nProgress: make .total into a property in order to fire the update notifications.\n\n*Release 20180703.1*:\nProgress: additions and changes to API: new .ratio, .elapsed_time, rename .projected to .remaining_time.\n\n*Release 20180703*:\nInitial release of cs.progress.\n",
"bugtrack_url": null,
"license": "GNU General Public License v3 or later (GPLv3+)",
"summary": "A progress tracker with methods for throughput, ETA and update notification; also a compound progress meter composed from other progress meters.",
"version": "20241122",
"project_urls": {
"MonoRepo Commits": "https://bitbucket.org/cameron_simpson/css/commits/branch/main",
"Monorepo Git Mirror": "https://github.com/cameron-simpson/css",
"Monorepo Hg/Mercurial Mirror": "https://hg.sr.ht/~cameron-simpson/css",
"Source": "https://github.com/cameron-simpson/css/blob/main/lib/python/cs/progress.py"
},
"split_keywords": [
"python2",
" python3"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "77aabfa2176fabc9a4f21fb560d1efe86b4571757aa3fbeee7e13e80b99a3ce5",
"md5": "2c8616a55f1e9549e09671005932a2e9",
"sha256": "c0a9b524f0d0a6d07e418e98b12e47daf3f536676a0e624f7e4fcff4d749a3cf"
},
"downloads": -1,
"filename": "cs_progress-20241122-py3-none-any.whl",
"has_sig": false,
"md5_digest": "2c8616a55f1e9549e09671005932a2e9",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 17095,
"upload_time": "2024-11-22T00:53:01",
"upload_time_iso_8601": "2024-11-22T00:53:01.403675Z",
"url": "https://files.pythonhosted.org/packages/77/aa/bfa2176fabc9a4f21fb560d1efe86b4571757aa3fbeee7e13e80b99a3ce5/cs_progress-20241122-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "74b93e7094961ad01e0acb216ee676eac2c3c34591b1819f92b9105614e2461b",
"md5": "af09372d5caa102ca49e059afcd30a74",
"sha256": "d86f53be3889503f39680db6828335c9e0b1b9f4499a5b26b78b7bde02ed02e3"
},
"downloads": -1,
"filename": "cs_progress-20241122.tar.gz",
"has_sig": false,
"md5_digest": "af09372d5caa102ca49e059afcd30a74",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 21431,
"upload_time": "2024-11-22T00:53:03",
"upload_time_iso_8601": "2024-11-22T00:53:03.798355Z",
"url": "https://files.pythonhosted.org/packages/74/b9/3e7094961ad01e0acb216ee676eac2c3c34591b1819f92b9105614e2461b/cs_progress-20241122.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-22 00:53:03",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "cameron-simpson",
"github_project": "css",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "cs-progress"
}
JSON
