=================
scikit-validate
=================
.. image:: https://img.shields.io/pypi/v/scikit-validate.svg
:target: https://pypi.python.org/pypi/scikit-validate
.. image:: https://readthedocs.org/projects/scikit-validate/badge/?version=latest
:target: https://scikit-validate.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status
.. image:: https://badges.gitter.im/FAST-HEP/community.svg
:target: https://gitter.im/FAST-HEP/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge
:alt: Gitter
* Free software: Apache Software License 2.0
* Documentation: https://scikit-validate.readthedocs.io.
* Issues: https://github.com/FAST-HEP/scikit-validate/issues
Overview
--------
scikit-validate is a validation package for science output developed within `F.A.S.T.`_.
This package provides commands for monitoring and comparing analysis outputs, \
computing resource usage (e.g. CPU time/RAM) as well as commands for summarising findings.
It is meant to provide analysis groups or small experiments with some of the fundamental features needed to\
validate (i.e. compare to a reference) the outcomes of their code and to provide easy access to the results.
Features
--------
* Collect metrics in JSON output
* measure file metrics (e.g. size)
* measure execution time and memory usage
* compare to previous executions
* compare ROOT files & plot discrepancies
* create validation reports
Credits
-------
This package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.
.. _`F.A.S.T.`: https://fast-hep.web.cern.ch/fast-hep/public
.. _Cookiecutter: https://github.com/audreyr/cookiecutter
.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage
Make a new RELEASE
------------------
.. code-block:: bash
export RELEASE=X.Y.Z
export CHANGELOG_GITHUB_TOKEN=<your github token>
make release
0.4.3
------------
- Merge pull request #43 from FAST-HEP/kreczko-issue-42. [Luke Kreczko]
Fixing issue #42: accessing values for failed distributions
- Fixed typo and move from np.resize to ak.pad_none + ak.fill_none.
[Luke Kreczko]
- Fixed issue #42. [Luke Kreczko]
0.4.2
------------
- Merge pull request #41 from FAST-HEP/kreczko-issue-38-part2. [Luke
Kreczko]
Handling of non-convertible ROOT objects
- Fix linting issue. [Luke Kreczko]
- Added handling of unconvertible ROOT objects. [Luke Kreczko]
- Added tests for ROOT types that cannot be numpyfied. [Luke Kreczko]
0.4.1
------------
- Merge pull request #39 from FAST-HEP/kreczko-issue-38. [Luke Kreczko]
Fixing issue #38
- Fixed issue #38: processing histograms. [Luke Kreczko]
- Added test for issue #38. [Luke Kreczko]
- Drop the obsolete History.rst from the Docs. [Luke Kreczko]
- Tagged version 0.4.0. [Luke Kreczko]
0.3.12
------------
Fix
~~~
- Added missing error info for section properties. [Luke Kreczko]
- More readable debug messages for get_pipeline_job. [Luke Kreczko]
- Wrong variable name for DEBUG messages. [Luke Kreczko]
Other
~~~~~
- Added debug output for io.download_from_gitlab. [Luke Kreczko]
- Python-gitlab 1.8 has been released, removing workarounds and
comments. [Luke Kreczko]
- Removing py 3.9 for now as not all dependencies are available yet.
[Luke Kreczko]
- Python 3.9 not yet available on travis, use 3.9-dev instead. [Luke
Kreczko]
- Updated requirements_dev.txt. [Luke Kreczko]
- CI: removed Python 2.7, 3.6. and 3.7, added python 3.8 and 3.9. [Luke
Kreczko]
- Feature: added debug output for get_pipeline_job. [Luke Kreczko]
Changelog
=========
0.3.9
------------
- Removed comment blocks in validation_detail template (would write
passing entries in there resulting in large files for large projects)
[kreczko]
- Merge pull request #28 from FAST-HEP/kreczko-issue-27. [Luke Kreczko]
Fixing issue #27: setup.py not compatible with Windows
- Setup.py: fixed Unix path for find_commands default value. [kreczko]
0.3.8
------------
- Fixed history vs releases. [kreczko]
- Make: add unreleased history at the top of HISTORY.rst (still needs
manual edit) [kreczko]
- New command: sv_version. [kreczko]
- Merge pull request #18 from kreczko/kreczko-metric-diff. [Luke
Kreczko]
New command: sv_metric_diff
- Fixed formatting. [kreczko]
- Added documentation for sv_metric_diff. [kreczko]
- New command: sv_metric_diff. [kreczko]
- Merge pull request #17 from kreczko/kreczko-issue-16. [Luke Kreczko]
Fixing issue 16: added system time to sv_execute
- Fixed usage instructions for sv_execute. [kreczko]
- Fixing issue #16: added system time to sv_execute. [kreczko]
- Updated metric names for demo report. [Luke Kreczko]
0.3.7
------------
- Merge pull request #15 from kreczko/kreczko-sanatizing-report-configs.
[Luke Kreczko]
Sanatising report configs
- Updated pytest and hypothesis dependencies. [kreczko]
- Adjust all configs to reversed-order rendering. [kreczko]
- Fixed reversed-order rendering for lists in templates. [kreczko]
- Fixed sv_root_info for more complex structures. [kreczko]
- Reversing the order of report config parsing. New order: first read
full YAML then render each value. [kreczko]
0.3.6
------------
- Fix commit description in HISTORY. [kreczko]
- Merge pull request #14 from FAST-
HEP/BK_check_nones_compare_two_root_files. [Luke Kreczko]
Add check in case we get a pair of Nones from load_values
- Catch several more places where `None`'s will matter. [Ben Krikler]
- Add check in case we get a pair of Nones from load_values. [Ben
Krikler]
0.3.5
------------
- Io.unpack: fixed behaviour for numpy arrays vs lists. [kreczko]
- Io.unpack: deal with empty flat arrays. [Luke Kreczko]
0.3.4
------------
- Added AssertionError to list of exceptions for converting ROOT
branches to numpy arrays. [Luke Kreczko]
0.3.3
------------
- Merge branch 'master' of github.com:FAST-HEP/scikit-validate.
[kreczko]
- Merge pull request #13 from kreczko/kreczko-string-branches. [Luke
Kreczko]
- Fixing issues reported by LZ
- Fixed tests for new test samples (extra branch) [kreczko]
- Compare: restored old behaviour for strings. [kreczko]
- Compare: make sure both arrays are same size and deal better with
empty arrays. [kreczko]
- Sv_root_diff: ommit string and object type arrays. [kreczko]
- Ommit flattening arrays where impossible. [kreczko]
- Added string branches to test samples. [kreczko]
0.3.2
------------
- Fix root_diff (draw_diff) crashing for string input. [kreczko]
- Added hypothesis tests for vis.adjust_axis_limits. [kreczko]
0.3.1
------------
- sv_root_info: fixed mask for non-readable. [kreczko]
0.3.0
------------
- Merge pull request #12 from kreczko/kreczko-root-info. [Luke Kreczko]
- Added sv_root_info for inspection of ROOT files with uproot
- sv_root_info: fixed lint errors. [kreczko]
- sv_root_info: added read test for non-obvious branches. [kreczko]
- Added sv_root_info for inspection of ROOT files with uproot. [kreczko]
0.2.24
------------
- When getting pipeline jobs: filter for only the last iteration of a
particular job. [kreczko]
0.2.23
------------
- Fix git.create_patch behaviour for empty diffs. [kreczko]
0.2.22
------------
- Cpp_check_format: added "--exclude" parameter. [kreczko]
- Cpp_check_format: fixed patch file (missing new line at the EOF)
[kreczko]
0.2.20
------------
- Added ability to overwrite default cpp_check_format template.
[kreczko]
- Cpp_check_format report: separated path & command variables. [kreczko]
0.2.19
------------
- Sv_cpp_check_format: fixed return code. [kreczko]
0.2.18
------------
- Sv_cpp_check_format: fixes issues with instructions. [kreczko]
0.2.17
------------
- Sv_cpp_check_format: fixes issues with detached head. [kreczko]
- Merge pull request #11 from kreczko/kreczko-cpp_format. [Luke Kreczko]
- New CI check: sv_cpp_check_format
0.2.16
------------
- Added wrappers for git commands. [kreczko]
- Sv_cpp_check_format: added report creation & publishing. [kreczko]
- Sv_cpp_check_format: added file retrival, formatting and patch
creation. [Luke Kreczko]
- Added draft for cpp_check_format. [kreczko]
0.2.15
------------
- Collapsible details for detailed validation report (issue #8)
[kreczko]
- Extending gitignore for test files. [kreczko]
- Updated history & changelog. [kreczko]
- Fixed HISTORY formatting. [kreczko]
- Fixed release example in README. [kreczko]
- CI: updated user & password for travis-ci.com. [kreczko]
0.2.14
------------
- Merge pull request #9 from kreczko/kreczko-issue-8. [Luke Kreczko]
Collapsible details for validation report (issue #8)
- CI: more verbose install. [kreczko]
- CI: updating Ubuntu distribution from 14.04 to 16.04. [kreczko]
- CI: added demo report. [kreczko]
- CI: added "make install" to script. [kreczko]
- CI: installing packages needed for development. [kreczko]
- Collapsible details for validation report (issue #8) [kreczko]
- Fix travis config. [kreczko]
- Updated travis condition for deployment. [kreczko]
- Merge pull request #7 from kreczko/kreczko-travis. [Luke Kreczko]
Updating travis to follow Gitlab CI
- Added Gitter link to README. [kreczko]
- README: added URL to issues. [kreczko]
- Updated travis to follow .gitlab-ci.yml. [kreczko]
0.2.13 (2019-05-13)
-------------------
- Merge branch 'kreczko-reduce-timeouts' into 'master' [Lukasz Kreczko]
Reduce timeouts during artifact download
See merge request fast-hep/public/scikit-validate!14
0.2.12
------------
- Added timeout to job artifact download. [kreczko]
- Merge branch 'kreczko-docs' into 'master' [Lukasz Kreczko]
Documentation "day" summary
See merge request fast-hep/public/scikit-validate!13
- Docs: expanded README and added quickstart draft. [kreczko]
- Docs: added custom CSS. [kreczko]
- Docs: switched from alabaster to sphinx_rtd_theme. [kreczko]
- Docs: added logo. [kreczko]
- Fixed flake8. [kreczko]
- Docs: added sphinxcontrib-apidoc extension. [kreczko]
- Fixed Sphinx documentation warnings. [kreczko]
- README: Fixed badges and bullet list indents. [kreczko]
- Docs: fixed typo to GitLab repo. [kreczko]
- Merge branch 'kreczko-better-error-messages' into 'master' [Lukasz
Kreczko]
Improvements to reporting: error messages & quality of life
Closes #6
See merge request fast-hep/public/scikit-validate!12
- Fixed issue #6: Bug: gitlab.get_pipeline_job returns first job only.
[kreczko]
0.2.11
------------
- Download_validation_outputs: do not append validation job name if
already part of path (e.g. user defined) [kreczko]
0.2.10
------------
- Validation report: batch size and line breaks are now configurable.
[kreczko]
- Gitlab.download_artifact will skip download if output file exists.
[kreczko]
- Report: added debug information for validation report. [kreczko]
- Report: moved parsing errors their respective subsections. [kreczko]
0.2.9
------------
- Merge branch 'kreczko-report-tuning' into 'master' [Lukasz Kreczko]
Fine-tuning reporting
Closes #5
See merge request fast-hep/public/scikit-validate!11
- Report: allow file download to fail (e.g. failed jobs in pipeline)
[kreczko]
- Validation report: increased the number of images per row from 5 to 8.
[kreczko]
- Validation report: simplified image loading. [kreczko]
- Gitlab artifact url: normalize path before use to exclude ".."
[kreczko]
- Validation report: added tests for _get_links_for_reports. [kreczko]
- Test_gitlab: added path_type to URL test. [kreczko]
- Validation report: separate report creation from link creation.
[kreczko]
- Validation report: add links to original images. [kreczko]
- Validation report: switched details from HTML to PDF output. [kreczko]
- Validation report: remove unused loop variable. [kreczko]
- Validation report: reduced image size & added overview. [kreczko]
- Sv_root_diff: fixed incorrect function name for processing. [kreczko]
- Sv_root_diff: added multi-processing support. [kreczko]
- Sv_root_diff: added reason for UNKNOWN status. [kreczko]
- Vis.draw_diff: trying to make plotting thread-safe. [kreczko]
- Sv_root_diff: added progressbar. [kreczko]
- Sv_root_diff: parallelised using threads. [kreczko]
- Added new command: sv_absolute_to_relative_path. [kreczko]
- Sv_execute: units are a separate entry --> shorten metric names.
[kreczko]
- Gitlab: always take local file path as relative to project path for
URLs. [kreczko]
0.2.8
------------
- Sv_make_report: fixed memeory_profile output file names for commands
that include paths. [kreczko]
0.2.7
------------
- Sv_root_diff: fix _reset_infinities for empty values. [kreczko]
0.2.6
------------
- Merge branch 'kreczko-root-diff-tuning' into 'master' [Lukasz Kreczko]
Bug fixes & generalisation for sv_root_diff
See merge request fast-hep/public/scikit-validate!10
- Vis.find_limits: fixed behaviour for empty arrays. [kreczko]
- Added unpack np array function to serialize JSON. [kreczko]
- Sv_root_diff: fixed issue with comparison between empty entries.
[kreczko]
- Sv_root_diff: added more information for WARNING & FAILED statuses.
[kreczko]
- Compare: added maxRelativeDifference and generalized is_ok function.
[kreczko]
- Sv_root_diff: switch WARNING color from invalid "orange" to valid
"Orange3" [kreczko]
- Sv_root_diff: improve robustness for 2D arrays and arrays of strings.
[kreczko]
0.2.5
------------
- Sv_execute: replace _thread with six.moves._thread. [kreczko]
- Sv_execute: replaced thread with six._thread. [kreczko]
- CI: increased sleep time for report stage. [kreczko]
- Sv_execute: memory profile monitoring now in separate thread.
[kreczko]
0.2.4
------------
- Added software module to setup.py. [kreczko]
0.2.3
------------
- Added gitlab.get_pipeline_url. [kreczko]
0.2.2
------------
- Merge branch 'kreczko-memory-profile' into 'master' [Lukasz Kreczko]
Adding memory profiles to validation report
See merge request fast-hep/public/scikit-validate!9
- CI: added delay of 60 seconds to report stage. [kreczko]
- CI: added PNG files to artifacts for report stage. [kreczko]
- Performance report: switched from raw link to dressed link for
profiling image. [kreczko]
- Updated example root_diff files with new plotting style and command
(root_diff -> sv_root_diff) [kreczko]
- Moved plotting style definitions from vis.profile to vis. [kreczko]
- CI report: fixed typo in report configuration. [kreczko]
- CI: fixed URL for reference memory profile. [kreczko]
- Gitlab: fix relative import for Python 2.7. [kreczko]
- Gitlab.DiskStreamer: create dowload directory if it does not exist.
[kreczko]
- Gitlab.get_jobs_for_stages: fixed typo in debug message. [kreczko]
- Report: memory profile now return full URL (local or CI) [kreczko]
- Fixed download_from_gitlab. [kreczko]
- CI: source instead of execute. [kreczko]
- Fixed lint errors. [kreczko]
- Implemented vis.draw_profile. [kreczko]
- Fix newlines when reading & writing the memory_profile. [kreczko]
- Fixed profile dictionary for profile template. [kreczko]
- Remaned example memory profile files. [kreczko]
- Added memory_profile to demo report. [kreczko]
- Fixed paths for memory profile files in CI report. [kreczko]
- Raised min. version for memory_profiler to 0.54 (first with mprof
module) [kreczko]
- Added memory profile data examples. [Lukasz Kreczko]
- Added vis.profile. [Lukasz Kreczko]
- Report: changed import of vis module, draw_profiles -->
vis.draw_profiles. [Lukasz Kreczko]
- Added profile to CI report. [Lukasz Kreczko]
- Added processing of profile files to report. [Lukasz Kreczko]
- Added processing for memory profile timestamps. [Lukasz Kreczko]
- Added function to split memory_profiler output from multiple commands.
[Lukasz Kreczko]
- Using a single profile file for memory_profile. [Lukasz Kreczko]
- Switched memory_profile from just the exe to the full command (as done
for the metrics) [Lukasz Kreczko]
- Added downloaded files to report artifacts. [Lukasz Kreczko]
- Downloading performace JSON and memory profiles for performance
report. [Lukasz Kreczko]
- Added special keyword "download" to report sections. [Lukasz Kreczko]
- Added documentation draft for report config. [Lukasz Kreczko]
- Added download capability to io package. [Lukasz Kreczko]
- Added gitlab.get_pipeline_job. [Lukasz Kreczko]
- Split performance validation across two jobs. [Lukasz Kreczko]
- Added memory profile to CI. [Lukasz Kreczko]
- Added memory profile to sv_exectute. [Lukasz Kreczko]
- Added memory_profiler as dependency. [Lukasz Kreczko]
- Merge branch 'BK_allow_multiple_argument_cmds' into 'master' [Lukasz
Kreczko]
Add support for mutiple positional arguments being used as the command to run
See merge request fast-hep/public/scikit-validate!8
- Use new command-line style in ci. [Ben Krikler]
- Remove TODO comment that I'd added. [Ben Krikler]
- Add support for mutiple positional arguments being used as the command
to run + pep8. [Ben Krikler]
0.2.1 (2019-03-12)
------------------
- Merge branch 'kreczko-rename-commands' into 'master' [Lukasz Kreczko]
More user-friendly command names
See merge request fast-hep/public/scikit-validate!7
- Switch to new command names in the CI. [Lukasz Kreczko]
- Updated command names & added TODOs. [Lukasz Kreczko]
- All commands now start with "sv\_" [Lukasz Kreczko]
0.2.0
------------
- Merge branch 'kreczko-better-validation-report' into 'master' [Lukasz
Kreczko]
Added better validation report
See merge request fast-hep/public/scikit-validate!6
- New command: submit_report_to_mr to add reports to the MR. [kreczko]
- Fix update of existing note in MR for report. [kreczko]
- Fix overwritting of values for detailed report. [Lukasz Kreczko]
- Fix validation detail template & remove debugging. [Lukasz Kreczko]
- Fix missing summary report. [Lukasz Kreczko]
- Made updating merge request with report available for GitLab < 11.6.
[Lukasz Kreczko]
- Fixed job_name. [Lukasz Kreczko]
- Added reporting to parent merge request. [Lukasz Kreczko]
- Added io.resolve_wildcard_path. [Lukasz Kreczko]
- Added job_name variable to detailed validation report. [Lukasz
Kreczko]
- Fixed report.format_software_versions. [Lukasz Kreczko]
- Added tests for report.format_software_versions. [Lukasz Kreczko]
- Reraising exception for template rendering. [Lukasz Kreczko]
- Added logging. [Lukasz Kreczko]
- Remove automatic reporting for now. [Lukasz Kreczko]
- Added debugging for validation detail template. [Lukasz Kreczko]
- Added HTML and PDF output formats for validation report. [Lukasz
Kreczko]
- Replacing pdfkit with xhtml2pdf. [Lukasz Kreczko]
- Added PDF output for validation HTML. [Lukasz Kreczko]
- Added pdfkit dependency. [Lukasz Kreczko]
- Added reporting to merge request. [Lukasz Kreczko]
- Added documentation for validation report. [Lukasz Kreczko]
- Replaced image & validation_detail URLs with RAW urls. [Lukasz
Kreczko]
- Downloading relevnt validation artifacts. [Lukasz Kreczko]
- Added download to disk option for gitlab.download_artifact. [Lukasz
Kreczko]
- Added ls for report job (debugging) [Lukasz Kreczko]
- Resolve image paths for validation jobs. [Lukasz Kreczko]
- Added validation reports to CI artifacts. [Lukasz Kreczko]
- Extract distributions from validation_json before passing them on.
[kreczko]
- Fixed prefix path for output_path in root_diff. [kreczko]
- Added missing output JSON for validate-root-diff-1_3. [kreczko]
- Fixed incorrect function calls. [kreczko]
- Added job_filter to GitLab job retrieval. [kreczko]
- Added validation report to CI. [kreczko]
- Switched Demo report to use more general values. [kreczko]
- Added more performance metrics. [kreczko]
- Added detailed validation report. [kreczko]
- Replaced demo report validation with new summary. [kreczko]
- Added validation summary. [kreczko]
- Added error reporting in validation template. [kreczko]
- Root_diff: added output_path to JSON output. [kreczko]
- Fixed tests for compare_two_root_files. [kreczko]
- Fixed lint issues. [kreczko]
- CI: fixed dependency for report. [kreczko]
- Added draft for validation detail. [Lukasz Kreczko]
- Tidied up symbols for demo report. [Lukasz Kreczko]
- Added 3rd validation example. [Lukasz Kreczko]
- Added examples for root_diff. [Lukasz Kreczko]
- Generalised gitlab download of JSON data. [Lukasz Kreczko]
- Added prefix to root_diff. [Lukasz Kreczko]
- Split root_diff validation job into two jobs (1 for each comparison)
[Lukasz Kreczko]
- Added root_diff summary. [Lukasz Kreczko]
- Making all produced JSON files human-readable. [Lukasz Kreczko]
- Added proper reporting to root_diff. [Lukasz Kreczko]
- Moved reseting infinities from draw_diff to root_diff. [Lukasz
Kreczko]
- Added short-hand option for root_diff:out-dir. [Lukasz Kreczko]
- Added first version of the logo. [kreczko]
- Merge branch 'kreczko-gitlab-access' into 'master' [Lukasz Kreczko]
First functional draft for pipeline reports
See merge request fast-hep/public/scikit-validate!5
- Fixed web_url_raw in gitlab.get_jobs_for_stages. [kreczko]
- Added skvalidate.report.get_jobs_for_stages. [kreczko]
- Moved report.demo._format_status to report.format_status. [kreczko]
- Ok -> success, fail -> failed to be more consistent with gitlab.
[kreczko]
- Link -> web_url to be more consistent with gitlab. [kreczko]
- Allow for artifact download to fail. [kreczko]
- Fixed prefix for detect_software_versions in CI. [kreczko]
- Made software_version retrieval more resilient against missing data.
[kreczko]
- Fix streamer for bytestrings. [kreczko]
- Returning to previous version but with additional error-handling.
[kreczko]
- Updated python-gitlab to latest master to avoid workaround. [kreczko]
- Made install procedure a bit more quiet. [kreczko]
- Added workaround for python-gitlab bug. [kreczko]
- Make installation of dependencies & after_script silent. [kreczko]
- Report: print section properties on error. [kreczko]
- Added robustness to performance report: only format as number if
variable is a number. [kreczko]
- Added quiet option for detect_software_versions. [kreczko]
- Enable streaming for gitlab job artifact retrieval. [kreczko]
- Replaced CI_ATUH_TOKEN with read-only API token from bot-account.
[kreczko]
- Fixed name for software_versions.json in CI. [kreczko]
- Added gitlab package. [kreczko]
- Fixed detect_software_versions after_script. [kreczko]
- Fixed unused module in get_artifact_url command. [kreczko]
- Added reporting to current CI. [kreczko]
- Added gitlab connectors. [kreczko]
- Restricting gitlab dependency to be >=1.7.0. [kreczko]
- Moved logic from get_artifact_url command to skvalidate.gitlab.
[kreczko]
- Re-enabled status symbols. [kreczko]
- Updated gitlab report config with pipelines and latest validation
section. [kreczko]
- Added prefixes to scan_software_version to allow for multiple
environments names. [kreczko]
- Removed obsolete function in report. [kreczko]
- Added validation info from JSON. [kreczko]
- Fixed table in pipeline template. [kreczko]
- Fixed lint in .software. [kreczko]
- Io: made save_metrics_to_file more general. Now have
update_data_in_json,write_data_to_json & read_data_from_json.
[kreczko]
- Added new pipelines to demo report. [kreczko]
- Added lower_is_better value to metrics. [kreczko]
- Added output file for detect_software_versions. [kreczko]
- Added detect_software_versions command. [kreczko]
- Replaced build, test and software sections in report with pipeline
section. [kreczko]
- Added markdown2 as new dependency. [kreczko]
- Fixed commands and metric names for performace and file report.
[kreczko]
- Added "make_report" command. [kreczko]
- Removed obsolete print statement. [kreczko]
- Added plumbum as new depedency. [kreczko]
- Merge branch 'kreczko-update-metrics' into 'master' [Lukasz Kreczko]
Fixed printouts for add_file_metrics & execute_with_metrics
See merge request fast-hep/public/scikit-validate!4
- Fixed tests for new-style metrics. [kreczko]
- Moved report.demo.get_metrics -> report.get_metrics. [kreczko]
- Fixed printouts for add_file_metrics & execute_with_metrics. [kreczko]
- Merge branch 'kreczko-update-metrics' into 'master' [Lukasz Kreczko]
Updated metrics for add_file_metric & execute_with_metrics to new-style metrics
See merge request fast-hep/public/scikit-validate!3
- Fixed name of performance metrics in CI. [kreczko]
- Added validation of the metrics pipeline (run & file metrics ->
report) to the CI. [kreczko]
- Updated execute_with_metrics to produce new-style metrics. [kreczko]
- Updated file metrics to new style. [kreczko]
- Merge branch 'kreczko-backwards-compatible-metrics' into 'master'
[Lukasz Kreczko]
Added backwards compatible metrics
See merge request fast-hep/public/scikit-validate!2
- Fixed linter issues. [kreczko]
- Converting metrics from old to new by default. [kreczko]
- Added method to convert from old to new metrics. [kreczko]
- Moved skvalidate.compare.compare_metrics to
skvalidate.compare.metrics.compare_metrics. [kreczko]
- Fixed typo in compare_metrics. [kreczko]
- Merge branch 'kreczko-report' into 'master' [Lukasz Kreczko]
Added report creation functionality
See merge request fast-hep/public/scikit-validate!1
- Moved data to skvalidate/data. [kreczko]
- Added report package. [kreczko]
- Trying indirect call to make_demo_report in CI. [kreczko]
0.1.8
------------
- Run demo report under python 3.7. [kreczko]
- Added Jinja2 to dependencies. [kreczko]
- Added demo_report to validation stage. [kreczko]
- Fixed potential Python2 syntax problems. [Lukasz Kreczko]
- Added PyYAML as a dependency. [Lukasz Kreczko]
- Fixed lint errors. [Lukasz Kreczko]
- Make report: output file now as command line argument instead of
config. [Lukasz Kreczko]
- Added metric comparison functionality. [Lukasz Kreczko]
- Fixed metric templates (files & performance) [Lukasz Kreczko]
- Added metric examples. [Lukasz Kreczko]
- Made tests more verbose. [Lukasz Kreczko]
- Updated gitlab report config & name. [Lukasz Kreczko]
- Updated template paths in demo report config. [Lukasz Kreczko]
- Moved config into data folder. [Lukasz Kreczko]
- Added draft for reporting. [Lukasz Kreczko]
- Added status symbols to demo. [Lukasz Kreczko]
- Moved templates to data folder. [Lukasz Kreczko]
- Made default templates more resilient. [Lukasz Kreczko]
- Added default report templates. [Lukasz Kreczko]
- Added example report configs. [Lukasz Kreczko]
- Improved development install. [Lukasz Kreczko]
0.1.7
------------
- Add_file_metrics: added fix for Python3 & test. [kreczko]
0.1.6
------------
- Execute_with_metrics: added fix for Python3 & test. [kreczko]
0.1.5
------------
- Made run-clang-tidy Python3 compatible. [kreczko]
- Fixed pep8 error in vis. [kreczko]
0.1.4
------------
- Disabled log scale for diff plot. [kreczko]
- Setting minY to non-zero for logarithmic plots. [kreczko]
- Disabled logY setting if negative values are found. [kreczko]
0.1.3
------------
- Execute_with_metrics: added soft-fail to IOException on writing
metrics file. [kreczko]
- Added tests for get_target_branch. [kreczko]
- Fix missing import. [kreczko]
0.1.2
------------
- Improved error-handling for get_target_branch and added target &
default branches. [kreczko]
- Worked through the stricter pep8 set. [Lukasz Kreczko]
- Added default target_branch to get_target_branch. [Lukasz Kreczko]
0.1.1
------------
- Visualisation adjustments for root_diff. [kreczko]
- Fixed python3 issues with io._walk. [kreczko]
0.1.0
------------
- Replaced io._walk with a more robust equivalent. [kreczko]
- Added automated logy & x-limits to drawing. [kreczko]
- Added code to generate tests/samples/objects.root. [kreczko]
- Steeled diff calculation, fixed normalisation (now w.r.t. reference)
and reporting non-comparible branches in root_diff. [kreczko]
- Made io.unpack more robust to str-arrays. [kreczko]
- Fixed pep8 error: unused include in test_io. [kreczko]
- Fixed "make test" [kreczko]
0.0.7
------------
- Added comparison of object data to tests. [kreczko]
- Updated io.walk to handle & unpack objects. [kreczko]
0.0.6
------------
- Automatically expose commands directly to command line. [kreczko]
- Made sure vector branches are flattened before comparison. [kreczko]
- Updated tests for vector branch. [kreczko]
- Added vector variable to test samples. [kreczko]
0.0.5
------------
- Added image examples for root_diff. [kreczko]
0.0.4
------------
- Added KS test to drawing. [kreczko]
- Moved compare_two_root_files to compare.compare_two_root_files.
[kreczko]
- Fixed "a" branch in tests/samples/test_3.root. [kreczko]
- Creating output/validate/test_1_3 in CI. [kreczko]
- Removed print from io test. [kreczko]
- Fixed pep8 errors. [kreczko]
- Parametrised IO tests. [kreczko]
- Switched "make test" from py.test to pytest. [kreczko]
- Fixed test samples. [kreczko]
- Added compare and vis packages to setup.py. [kreczko]
- Moved draw_diff to vis.draw_diff. [kreczko]
- _compare_mctruth --> _compare. [kreczko]
- Moved _diff and _isOK to compare.difference & compare.is_ok. [kreczko]
- Moved store_diff to io.save_array_to_file. [kreczko]
- Moved walk function into skvalidate.io. [kreczko]
- Added folder creation to validation step. [kreczko]
- Added installation to validation step in CI. [kreczko]
- Added six to substitute xrange. [kreczko]
- Fixed pep8. [kreczko]
- Added validation step to CI. [kreczko]
- Added test samples & generation script. [kreczko]
- Added auto-generated docs. [kreczko]
0.0.3
------------
- Fix package description and align versions in __init__.py and
setup.cfg. [kreczko]
- Fixed pep8 errors. [kreczko]
- Added get_artifact_url. [kreczko]
- Added folder for static docs content. [kreczko]
- Added root_diff draft. [kreczko]
- Added remove_from_env command. [kreczko]
- Updated documentation for all commands. [kreczko]
- Fixed indentation in README. [kreczko]
- Expanded on the features of scikit-validate. [Lukasz Kreczko]
- Renaming CLI class. [Lukasz Kreczko]
- Rename to scikit-validate. [Lukasz Kreczko]
- Added missing variable to setup.py. [kreczko]
0.0.2
------------
- Added version lookup in setup.py from lz_validation. [kreczko]
- CI: simplyfied twine upload to pypi. [kreczko]
- Fixed behaviour of get_target_branch for non-MR branches. [kreczko]
- Added explitcit user and pw parameters for twine (upload_to_pypi.sh)
[kreczko]
- Fixed comment in upload_to_pypi.sh. [kreczko]
- Fixed package distribution (was missing commands and io modules)
[kreczko]
- Moved pypi upload into separate script. [kreczko]
- Fixed pypi deployment procedure. [kreczko]
- Fix gitlab-ci.yml. [kreczko]
- Added pypi upload. [kreczko]
- Updated failing test. [kreczko]
- Fixed linter errors. [kreczko]
- Moved linting to quick-checks stage. [kreczko]
- Added artifacts for build stage. [kreczko]
- Added gitlab-ci.yml. [kreczko]
- Added more tox setups. [kreczko]
- Added add_file_metrics command. [kreczko]
- Added general save_metrics_to_file to lz_validation.io. [kreczko]
- Added command to merge JSON files. [kreczko]
- Added script to run clang-tidy. [kreczko]
- Added execute_with_metrics command. [kreczko]
- Added command structure and get_target_branch command. [kreczko]
- Initial commit. [kreczko]
Raw data
{
"_id": null,
"home_page": "https://gitlab.cern.ch/fast-hep/public/scikit-validate",
"name": "scikit-validate",
"maintainer": null,
"docs_url": null,
"requires_python": null,
"maintainer_email": null,
"keywords": "skvalidate",
"author": "Luke Kreczko",
"author_email": "kreczko@cern.ch",
"download_url": "https://files.pythonhosted.org/packages/71/76/7c3c4cfd159af4fb713831106820f976e987412cf0e14a59fd32b652ad3c/scikit-validate-0.4.4.tar.gz",
"platform": null,
"description": "=================\nscikit-validate\n=================\n\n\n.. image:: https://img.shields.io/pypi/v/scikit-validate.svg\n :target: https://pypi.python.org/pypi/scikit-validate\n\n.. image:: https://readthedocs.org/projects/scikit-validate/badge/?version=latest\n :target: https://scikit-validate.readthedocs.io/en/latest/?badge=latest\n :alt: Documentation Status\n\n.. image:: https://badges.gitter.im/FAST-HEP/community.svg\n :target: https://gitter.im/FAST-HEP/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge\n :alt: Gitter\n\n\n* Free software: Apache Software License 2.0\n* Documentation: https://scikit-validate.readthedocs.io.\n* Issues: https://github.com/FAST-HEP/scikit-validate/issues\n\nOverview\n--------\nscikit-validate is a validation package for science output developed within `F.A.S.T.`_.\nThis package provides commands for monitoring and comparing analysis outputs, \\\ncomputing resource usage (e.g. CPU time/RAM) as well as commands for summarising findings.\n\nIt is meant to provide analysis groups or small experiments with some of the fundamental features needed to\\\nvalidate (i.e. compare to a reference) the outcomes of their code and to provide easy access to the results.\n\nFeatures\n--------\n\n* Collect metrics in JSON output\n * measure file metrics (e.g. size)\n * measure execution time and memory usage\n * compare to previous executions\n* compare ROOT files & plot discrepancies\n* create validation reports\n\nCredits\n-------\n\nThis package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template.\n\n.. _`F.A.S.T.`: https://fast-hep.web.cern.ch/fast-hep/public\n.. _Cookiecutter: https://github.com/audreyr/cookiecutter\n.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage\n\n\nMake a new RELEASE\n------------------\n\n.. code-block:: bash\n\n export RELEASE=X.Y.Z\n export CHANGELOG_GITHUB_TOKEN=<your github token>\n make release\n\n\n0.4.3\n------------\n- Merge pull request #43 from FAST-HEP/kreczko-issue-42. [Luke Kreczko]\n\n Fixing issue #42: accessing values for failed distributions\n- Fixed typo and move from np.resize to ak.pad_none + ak.fill_none.\n [Luke Kreczko]\n- Fixed issue #42. [Luke Kreczko]\n\n\n0.4.2\n------------\n- Merge pull request #41 from FAST-HEP/kreczko-issue-38-part2. [Luke\n Kreczko]\n\n Handling of non-convertible ROOT objects\n- Fix linting issue. [Luke Kreczko]\n- Added handling of unconvertible ROOT objects. [Luke Kreczko]\n- Added tests for ROOT types that cannot be numpyfied. [Luke Kreczko]\n\n\n0.4.1\n------------\n- Merge pull request #39 from FAST-HEP/kreczko-issue-38. [Luke Kreczko]\n\n Fixing issue #38\n- Fixed issue #38: processing histograms. [Luke Kreczko]\n- Added test for issue #38. [Luke Kreczko]\n- Drop the obsolete History.rst from the Docs. [Luke Kreczko]\n- Tagged version 0.4.0. [Luke Kreczko]\n\n\n0.3.12\n------------\n\nFix\n~~~\n- Added missing error info for section properties. [Luke Kreczko]\n- More readable debug messages for get_pipeline_job. [Luke Kreczko]\n- Wrong variable name for DEBUG messages. [Luke Kreczko]\n\nOther\n~~~~~\n- Added debug output for io.download_from_gitlab. [Luke Kreczko]\n- Python-gitlab 1.8 has been released, removing workarounds and\n comments. [Luke Kreczko]\n- Removing py 3.9 for now as not all dependencies are available yet.\n [Luke Kreczko]\n- Python 3.9 not yet available on travis, use 3.9-dev instead. [Luke\n Kreczko]\n- Updated requirements_dev.txt. [Luke Kreczko]\n- CI: removed Python 2.7, 3.6. and 3.7, added python 3.8 and 3.9. [Luke\n Kreczko]\n- Feature: added debug output for get_pipeline_job. [Luke Kreczko]\n\n\nChangelog\n=========\n\n\n0.3.9\n------------\n- Removed comment blocks in validation_detail template (would write\n passing entries in there resulting in large files for large projects)\n [kreczko]\n- Merge pull request #28 from FAST-HEP/kreczko-issue-27. [Luke Kreczko]\n\n Fixing issue #27: setup.py not compatible with Windows\n- Setup.py: fixed Unix path for find_commands default value. [kreczko]\n\n\n0.3.8\n------------\n- Fixed history vs releases. [kreczko]\n- Make: add unreleased history at the top of HISTORY.rst (still needs\n manual edit) [kreczko]\n- New command: sv_version. [kreczko]\n- Merge pull request #18 from kreczko/kreczko-metric-diff. [Luke\n Kreczko]\n\n New command: sv_metric_diff\n- Fixed formatting. [kreczko]\n- Added documentation for sv_metric_diff. [kreczko]\n- New command: sv_metric_diff. [kreczko]\n- Merge pull request #17 from kreczko/kreczko-issue-16. [Luke Kreczko]\n\n Fixing issue 16: added system time to sv_execute\n- Fixed usage instructions for sv_execute. [kreczko]\n- Fixing issue #16: added system time to sv_execute. [kreczko]\n- Updated metric names for demo report. [Luke Kreczko]\n\n\n\n0.3.7\n------------\n- Merge pull request #15 from kreczko/kreczko-sanatizing-report-configs.\n [Luke Kreczko]\n\n Sanatising report configs\n- Updated pytest and hypothesis dependencies. [kreczko]\n- Adjust all configs to reversed-order rendering. [kreczko]\n- Fixed reversed-order rendering for lists in templates. [kreczko]\n- Fixed sv_root_info for more complex structures. [kreczko]\n- Reversing the order of report config parsing. New order: first read\n full YAML then render each value. [kreczko]\n\n0.3.6\n------------\n- Fix commit description in HISTORY. [kreczko]\n- Merge pull request #14 from FAST-\n HEP/BK_check_nones_compare_two_root_files. [Luke Kreczko]\n\n Add check in case we get a pair of Nones from load_values\n- Catch several more places where `None`'s will matter. [Ben Krikler]\n- Add check in case we get a pair of Nones from load_values. [Ben\n Krikler]\n\n0.3.5\n------------\n- Io.unpack: fixed behaviour for numpy arrays vs lists. [kreczko]\n- Io.unpack: deal with empty flat arrays. [Luke Kreczko]\n\n0.3.4\n------------\n- Added AssertionError to list of exceptions for converting ROOT\n branches to numpy arrays. [Luke Kreczko]\n\n0.3.3\n------------\n- Merge branch 'master' of github.com:FAST-HEP/scikit-validate.\n [kreczko]\n- Merge pull request #13 from kreczko/kreczko-string-branches. [Luke\n Kreczko]\n\n- Fixing issues reported by LZ\n- Fixed tests for new test samples (extra branch) [kreczko]\n- Compare: restored old behaviour for strings. [kreczko]\n- Compare: make sure both arrays are same size and deal better with\n empty arrays. [kreczko]\n- Sv_root_diff: ommit string and object type arrays. [kreczko]\n- Ommit flattening arrays where impossible. [kreczko]\n- Added string branches to test samples. [kreczko]\n\n0.3.2\n------------\n- Fix root_diff (draw_diff) crashing for string input. [kreczko]\n- Added hypothesis tests for vis.adjust_axis_limits. [kreczko]\n\n0.3.1\n------------\n- sv_root_info: fixed mask for non-readable. [kreczko]\n\n0.3.0\n------------\n- Merge pull request #12 from kreczko/kreczko-root-info. [Luke Kreczko]\n- Added sv_root_info for inspection of ROOT files with uproot\n- sv_root_info: fixed lint errors. [kreczko]\n- sv_root_info: added read test for non-obvious branches. [kreczko]\n- Added sv_root_info for inspection of ROOT files with uproot. [kreczko]\n\n0.2.24\n------------\n- When getting pipeline jobs: filter for only the last iteration of a\n particular job. [kreczko]\n\n0.2.23\n------------\n- Fix git.create_patch behaviour for empty diffs. [kreczko]\n\n0.2.22\n------------\n- Cpp_check_format: added \"--exclude\" parameter. [kreczko]\n- Cpp_check_format: fixed patch file (missing new line at the EOF)\n [kreczko]\n\n\n0.2.20\n------------\n- Added ability to overwrite default cpp_check_format template.\n [kreczko]\n- Cpp_check_format report: separated path & command variables. [kreczko]\n\n\n0.2.19\n------------\n- Sv_cpp_check_format: fixed return code. [kreczko]\n\n\n0.2.18\n------------\n- Sv_cpp_check_format: fixes issues with instructions. [kreczko]\n\n\n0.2.17\n------------\n- Sv_cpp_check_format: fixes issues with detached head. [kreczko]\n- Merge pull request #11 from kreczko/kreczko-cpp_format. [Luke Kreczko]\n\n- New CI check: sv_cpp_check_format\n\n\n0.2.16\n------------\n- Added wrappers for git commands. [kreczko]\n- Sv_cpp_check_format: added report creation & publishing. [kreczko]\n- Sv_cpp_check_format: added file retrival, formatting and patch\n creation. [Luke Kreczko]\n- Added draft for cpp_check_format. [kreczko]\n\n\n0.2.15\n------------\n- Collapsible details for detailed validation report (issue #8)\n [kreczko]\n- Extending gitignore for test files. [kreczko]\n- Updated history & changelog. [kreczko]\n- Fixed HISTORY formatting. [kreczko]\n- Fixed release example in README. [kreczko]\n- CI: updated user & password for travis-ci.com. [kreczko]\n\n\n0.2.14\n------------\n- Merge pull request #9 from kreczko/kreczko-issue-8. [Luke Kreczko]\n\n Collapsible details for validation report (issue #8)\n- CI: more verbose install. [kreczko]\n- CI: updating Ubuntu distribution from 14.04 to 16.04. [kreczko]\n- CI: added demo report. [kreczko]\n- CI: added \"make install\" to script. [kreczko]\n- CI: installing packages needed for development. [kreczko]\n- Collapsible details for validation report (issue #8) [kreczko]\n- Fix travis config. [kreczko]\n- Updated travis condition for deployment. [kreczko]\n- Merge pull request #7 from kreczko/kreczko-travis. [Luke Kreczko]\n\n Updating travis to follow Gitlab CI\n- Added Gitter link to README. [kreczko]\n- README: added URL to issues. [kreczko]\n- Updated travis to follow .gitlab-ci.yml. [kreczko]\n\n\n0.2.13 (2019-05-13)\n-------------------\n- Merge branch 'kreczko-reduce-timeouts' into 'master' [Lukasz Kreczko]\n Reduce timeouts during artifact download\n See merge request fast-hep/public/scikit-validate!14\n\n\n0.2.12\n------------\n- Added timeout to job artifact download. [kreczko]\n- Merge branch 'kreczko-docs' into 'master' [Lukasz Kreczko]\n\n Documentation \"day\" summary\n\n See merge request fast-hep/public/scikit-validate!13\n- Docs: expanded README and added quickstart draft. [kreczko]\n- Docs: added custom CSS. [kreczko]\n- Docs: switched from alabaster to sphinx_rtd_theme. [kreczko]\n- Docs: added logo. [kreczko]\n- Fixed flake8. [kreczko]\n- Docs: added sphinxcontrib-apidoc extension. [kreczko]\n- Fixed Sphinx documentation warnings. [kreczko]\n- README: Fixed badges and bullet list indents. [kreczko]\n- Docs: fixed typo to GitLab repo. [kreczko]\n- Merge branch 'kreczko-better-error-messages' into 'master' [Lukasz\n Kreczko]\n\n Improvements to reporting: error messages & quality of life\n\n Closes #6\n\n See merge request fast-hep/public/scikit-validate!12\n- Fixed issue #6: Bug: gitlab.get_pipeline_job returns first job only.\n [kreczko]\n\n0.2.11\n------------\n- Download_validation_outputs: do not append validation job name if\n already part of path (e.g. user defined) [kreczko]\n\n0.2.10\n------------\n- Validation report: batch size and line breaks are now configurable.\n [kreczko]\n- Gitlab.download_artifact will skip download if output file exists.\n [kreczko]\n- Report: added debug information for validation report. [kreczko]\n- Report: moved parsing errors their respective subsections. [kreczko]\n\n\n0.2.9\n------------\n- Merge branch 'kreczko-report-tuning' into 'master' [Lukasz Kreczko]\n Fine-tuning reporting\n Closes #5\n See merge request fast-hep/public/scikit-validate!11\n- Report: allow file download to fail (e.g. failed jobs in pipeline)\n [kreczko]\n- Validation report: increased the number of images per row from 5 to 8.\n [kreczko]\n- Validation report: simplified image loading. [kreczko]\n- Gitlab artifact url: normalize path before use to exclude \"..\"\n [kreczko]\n- Validation report: added tests for _get_links_for_reports. [kreczko]\n- Test_gitlab: added path_type to URL test. [kreczko]\n- Validation report: separate report creation from link creation.\n [kreczko]\n- Validation report: add links to original images. [kreczko]\n- Validation report: switched details from HTML to PDF output. [kreczko]\n- Validation report: remove unused loop variable. [kreczko]\n- Validation report: reduced image size & added overview. [kreczko]\n- Sv_root_diff: fixed incorrect function name for processing. [kreczko]\n- Sv_root_diff: added multi-processing support. [kreczko]\n- Sv_root_diff: added reason for UNKNOWN status. [kreczko]\n- Vis.draw_diff: trying to make plotting thread-safe. [kreczko]\n- Sv_root_diff: added progressbar. [kreczko]\n- Sv_root_diff: parallelised using threads. [kreczko]\n- Added new command: sv_absolute_to_relative_path. [kreczko]\n- Sv_execute: units are a separate entry --> shorten metric names.\n [kreczko]\n- Gitlab: always take local file path as relative to project path for\n URLs. [kreczko]\n\n0.2.8\n------------\n- Sv_make_report: fixed memeory_profile output file names for commands\n that include paths. [kreczko]\n\n0.2.7\n------------\n- Sv_root_diff: fix _reset_infinities for empty values. [kreczko]\n\n\n0.2.6\n------------\n- Merge branch 'kreczko-root-diff-tuning' into 'master' [Lukasz Kreczko]\n Bug fixes & generalisation for sv_root_diff\n See merge request fast-hep/public/scikit-validate!10\n- Vis.find_limits: fixed behaviour for empty arrays. [kreczko]\n- Added unpack np array function to serialize JSON. [kreczko]\n- Sv_root_diff: fixed issue with comparison between empty entries.\n [kreczko]\n- Sv_root_diff: added more information for WARNING & FAILED statuses.\n [kreczko]\n- Compare: added maxRelativeDifference and generalized is_ok function.\n [kreczko]\n- Sv_root_diff: switch WARNING color from invalid \"orange\" to valid\n \"Orange3\" [kreczko]\n- Sv_root_diff: improve robustness for 2D arrays and arrays of strings.\n [kreczko]\n\n\n0.2.5\n------------\n- Sv_execute: replace _thread with six.moves._thread. [kreczko]\n- Sv_execute: replaced thread with six._thread. [kreczko]\n- CI: increased sleep time for report stage. [kreczko]\n- Sv_execute: memory profile monitoring now in separate thread.\n [kreczko]\n\n\n0.2.4\n------------\n- Added software module to setup.py. [kreczko]\n\n0.2.3\n------------\n- Added gitlab.get_pipeline_url. [kreczko]\n\n0.2.2\n------------\n- Merge branch 'kreczko-memory-profile' into 'master' [Lukasz Kreczko]\n\n Adding memory profiles to validation report\n\n See merge request fast-hep/public/scikit-validate!9\n- CI: added delay of 60 seconds to report stage. [kreczko]\n- CI: added PNG files to artifacts for report stage. [kreczko]\n- Performance report: switched from raw link to dressed link for\n profiling image. [kreczko]\n- Updated example root_diff files with new plotting style and command\n (root_diff -> sv_root_diff) [kreczko]\n- Moved plotting style definitions from vis.profile to vis. [kreczko]\n- CI report: fixed typo in report configuration. [kreczko]\n- CI: fixed URL for reference memory profile. [kreczko]\n- Gitlab: fix relative import for Python 2.7. [kreczko]\n- Gitlab.DiskStreamer: create dowload directory if it does not exist.\n [kreczko]\n- Gitlab.get_jobs_for_stages: fixed typo in debug message. [kreczko]\n- Report: memory profile now return full URL (local or CI) [kreczko]\n- Fixed download_from_gitlab. [kreczko]\n- CI: source instead of execute. [kreczko]\n- Fixed lint errors. [kreczko]\n- Implemented vis.draw_profile. [kreczko]\n- Fix newlines when reading & writing the memory_profile. [kreczko]\n- Fixed profile dictionary for profile template. [kreczko]\n- Remaned example memory profile files. [kreczko]\n- Added memory_profile to demo report. [kreczko]\n- Fixed paths for memory profile files in CI report. [kreczko]\n- Raised min. version for memory_profiler to 0.54 (first with mprof\n module) [kreczko]\n- Added memory profile data examples. [Lukasz Kreczko]\n- Added vis.profile. [Lukasz Kreczko]\n- Report: changed import of vis module, draw_profiles -->\n vis.draw_profiles. [Lukasz Kreczko]\n- Added profile to CI report. [Lukasz Kreczko]\n- Added processing of profile files to report. [Lukasz Kreczko]\n- Added processing for memory profile timestamps. [Lukasz Kreczko]\n- Added function to split memory_profiler output from multiple commands.\n [Lukasz Kreczko]\n- Using a single profile file for memory_profile. [Lukasz Kreczko]\n- Switched memory_profile from just the exe to the full command (as done\n for the metrics) [Lukasz Kreczko]\n- Added downloaded files to report artifacts. [Lukasz Kreczko]\n- Downloading performace JSON and memory profiles for performance\n report. [Lukasz Kreczko]\n- Added special keyword \"download\" to report sections. [Lukasz Kreczko]\n- Added documentation draft for report config. [Lukasz Kreczko]\n- Added download capability to io package. [Lukasz Kreczko]\n- Added gitlab.get_pipeline_job. [Lukasz Kreczko]\n- Split performance validation across two jobs. [Lukasz Kreczko]\n- Added memory profile to CI. [Lukasz Kreczko]\n- Added memory profile to sv_exectute. [Lukasz Kreczko]\n- Added memory_profiler as dependency. [Lukasz Kreczko]\n- Merge branch 'BK_allow_multiple_argument_cmds' into 'master' [Lukasz\n Kreczko]\n\n Add support for mutiple positional arguments being used as the command to run\n\n See merge request fast-hep/public/scikit-validate!8\n- Use new command-line style in ci. [Ben Krikler]\n- Remove TODO comment that I'd added. [Ben Krikler]\n- Add support for mutiple positional arguments being used as the command\n to run + pep8. [Ben Krikler]\n\n\n0.2.1 (2019-03-12)\n------------------\n- Merge branch 'kreczko-rename-commands' into 'master' [Lukasz Kreczko]\n\n More user-friendly command names\n\n See merge request fast-hep/public/scikit-validate!7\n- Switch to new command names in the CI. [Lukasz Kreczko]\n- Updated command names & added TODOs. [Lukasz Kreczko]\n- All commands now start with \"sv\\_\" [Lukasz Kreczko]\n\n\n0.2.0\n------------\n- Merge branch 'kreczko-better-validation-report' into 'master' [Lukasz\n Kreczko]\n\n Added better validation report\n\n See merge request fast-hep/public/scikit-validate!6\n- New command: submit_report_to_mr to add reports to the MR. [kreczko]\n- Fix update of existing note in MR for report. [kreczko]\n- Fix overwritting of values for detailed report. [Lukasz Kreczko]\n- Fix validation detail template & remove debugging. [Lukasz Kreczko]\n- Fix missing summary report. [Lukasz Kreczko]\n- Made updating merge request with report available for GitLab < 11.6.\n [Lukasz Kreczko]\n- Fixed job_name. [Lukasz Kreczko]\n- Added reporting to parent merge request. [Lukasz Kreczko]\n- Added io.resolve_wildcard_path. [Lukasz Kreczko]\n- Added job_name variable to detailed validation report. [Lukasz\n Kreczko]\n- Fixed report.format_software_versions. [Lukasz Kreczko]\n- Added tests for report.format_software_versions. [Lukasz Kreczko]\n- Reraising exception for template rendering. [Lukasz Kreczko]\n- Added logging. [Lukasz Kreczko]\n- Remove automatic reporting for now. [Lukasz Kreczko]\n- Added debugging for validation detail template. [Lukasz Kreczko]\n- Added HTML and PDF output formats for validation report. [Lukasz\n Kreczko]\n- Replacing pdfkit with xhtml2pdf. [Lukasz Kreczko]\n- Added PDF output for validation HTML. [Lukasz Kreczko]\n- Added pdfkit dependency. [Lukasz Kreczko]\n- Added reporting to merge request. [Lukasz Kreczko]\n- Added documentation for validation report. [Lukasz Kreczko]\n- Replaced image & validation_detail URLs with RAW urls. [Lukasz\n Kreczko]\n- Downloading relevnt validation artifacts. [Lukasz Kreczko]\n- Added download to disk option for gitlab.download_artifact. [Lukasz\n Kreczko]\n- Added ls for report job (debugging) [Lukasz Kreczko]\n- Resolve image paths for validation jobs. [Lukasz Kreczko]\n- Added validation reports to CI artifacts. [Lukasz Kreczko]\n- Extract distributions from validation_json before passing them on.\n [kreczko]\n- Fixed prefix path for output_path in root_diff. [kreczko]\n- Added missing output JSON for validate-root-diff-1_3. [kreczko]\n- Fixed incorrect function calls. [kreczko]\n- Added job_filter to GitLab job retrieval. [kreczko]\n- Added validation report to CI. [kreczko]\n- Switched Demo report to use more general values. [kreczko]\n- Added more performance metrics. [kreczko]\n- Added detailed validation report. [kreczko]\n- Replaced demo report validation with new summary. [kreczko]\n- Added validation summary. [kreczko]\n- Added error reporting in validation template. [kreczko]\n- Root_diff: added output_path to JSON output. [kreczko]\n- Fixed tests for compare_two_root_files. [kreczko]\n- Fixed lint issues. [kreczko]\n- CI: fixed dependency for report. [kreczko]\n- Added draft for validation detail. [Lukasz Kreczko]\n- Tidied up symbols for demo report. [Lukasz Kreczko]\n- Added 3rd validation example. [Lukasz Kreczko]\n- Added examples for root_diff. [Lukasz Kreczko]\n- Generalised gitlab download of JSON data. [Lukasz Kreczko]\n- Added prefix to root_diff. [Lukasz Kreczko]\n- Split root_diff validation job into two jobs (1 for each comparison)\n [Lukasz Kreczko]\n- Added root_diff summary. [Lukasz Kreczko]\n- Making all produced JSON files human-readable. [Lukasz Kreczko]\n- Added proper reporting to root_diff. [Lukasz Kreczko]\n- Moved reseting infinities from draw_diff to root_diff. [Lukasz\n Kreczko]\n- Added short-hand option for root_diff:out-dir. [Lukasz Kreczko]\n- Added first version of the logo. [kreczko]\n- Merge branch 'kreczko-gitlab-access' into 'master' [Lukasz Kreczko]\n\n First functional draft for pipeline reports\n\n See merge request fast-hep/public/scikit-validate!5\n- Fixed web_url_raw in gitlab.get_jobs_for_stages. [kreczko]\n- Added skvalidate.report.get_jobs_for_stages. [kreczko]\n- Moved report.demo._format_status to report.format_status. [kreczko]\n- Ok -> success, fail -> failed to be more consistent with gitlab.\n [kreczko]\n- Link -> web_url to be more consistent with gitlab. [kreczko]\n- Allow for artifact download to fail. [kreczko]\n- Fixed prefix for detect_software_versions in CI. [kreczko]\n- Made software_version retrieval more resilient against missing data.\n [kreczko]\n- Fix streamer for bytestrings. [kreczko]\n- Returning to previous version but with additional error-handling.\n [kreczko]\n- Updated python-gitlab to latest master to avoid workaround. [kreczko]\n- Made install procedure a bit more quiet. [kreczko]\n- Added workaround for python-gitlab bug. [kreczko]\n- Make installation of dependencies & after_script silent. [kreczko]\n- Report: print section properties on error. [kreczko]\n- Added robustness to performance report: only format as number if\n variable is a number. [kreczko]\n- Added quiet option for detect_software_versions. [kreczko]\n- Enable streaming for gitlab job artifact retrieval. [kreczko]\n- Replaced CI_ATUH_TOKEN with read-only API token from bot-account.\n [kreczko]\n- Fixed name for software_versions.json in CI. [kreczko]\n- Added gitlab package. [kreczko]\n- Fixed detect_software_versions after_script. [kreczko]\n- Fixed unused module in get_artifact_url command. [kreczko]\n- Added reporting to current CI. [kreczko]\n- Added gitlab connectors. [kreczko]\n- Restricting gitlab dependency to be >=1.7.0. [kreczko]\n- Moved logic from get_artifact_url command to skvalidate.gitlab.\n [kreczko]\n- Re-enabled status symbols. [kreczko]\n- Updated gitlab report config with pipelines and latest validation\n section. [kreczko]\n- Added prefixes to scan_software_version to allow for multiple\n environments names. [kreczko]\n- Removed obsolete function in report. [kreczko]\n- Added validation info from JSON. [kreczko]\n- Fixed table in pipeline template. [kreczko]\n- Fixed lint in .software. [kreczko]\n- Io: made save_metrics_to_file more general. Now have\n update_data_in_json,write_data_to_json & read_data_from_json.\n [kreczko]\n- Added new pipelines to demo report. [kreczko]\n- Added lower_is_better value to metrics. [kreczko]\n- Added output file for detect_software_versions. [kreczko]\n- Added detect_software_versions command. [kreczko]\n- Replaced build, test and software sections in report with pipeline\n section. [kreczko]\n- Added markdown2 as new dependency. [kreczko]\n- Fixed commands and metric names for performace and file report.\n [kreczko]\n- Added \"make_report\" command. [kreczko]\n- Removed obsolete print statement. [kreczko]\n- Added plumbum as new depedency. [kreczko]\n- Merge branch 'kreczko-update-metrics' into 'master' [Lukasz Kreczko]\n\n Fixed printouts for add_file_metrics & execute_with_metrics\n\n See merge request fast-hep/public/scikit-validate!4\n- Fixed tests for new-style metrics. [kreczko]\n- Moved report.demo.get_metrics -> report.get_metrics. [kreczko]\n- Fixed printouts for add_file_metrics & execute_with_metrics. [kreczko]\n- Merge branch 'kreczko-update-metrics' into 'master' [Lukasz Kreczko]\n\n Updated metrics for add_file_metric & execute_with_metrics to new-style metrics\n\n See merge request fast-hep/public/scikit-validate!3\n- Fixed name of performance metrics in CI. [kreczko]\n- Added validation of the metrics pipeline (run & file metrics ->\n report) to the CI. [kreczko]\n- Updated execute_with_metrics to produce new-style metrics. [kreczko]\n- Updated file metrics to new style. [kreczko]\n- Merge branch 'kreczko-backwards-compatible-metrics' into 'master'\n [Lukasz Kreczko]\n\n Added backwards compatible metrics\n\n See merge request fast-hep/public/scikit-validate!2\n- Fixed linter issues. [kreczko]\n- Converting metrics from old to new by default. [kreczko]\n- Added method to convert from old to new metrics. [kreczko]\n- Moved skvalidate.compare.compare_metrics to\n skvalidate.compare.metrics.compare_metrics. [kreczko]\n- Fixed typo in compare_metrics. [kreczko]\n- Merge branch 'kreczko-report' into 'master' [Lukasz Kreczko]\n\n Added report creation functionality\n\n See merge request fast-hep/public/scikit-validate!1\n- Moved data to skvalidate/data. [kreczko]\n- Added report package. [kreczko]\n- Trying indirect call to make_demo_report in CI. [kreczko]\n\n0.1.8\n------------\n- Run demo report under python 3.7. [kreczko]\n- Added Jinja2 to dependencies. [kreczko]\n- Added demo_report to validation stage. [kreczko]\n- Fixed potential Python2 syntax problems. [Lukasz Kreczko]\n- Added PyYAML as a dependency. [Lukasz Kreczko]\n- Fixed lint errors. [Lukasz Kreczko]\n- Make report: output file now as command line argument instead of\n config. [Lukasz Kreczko]\n- Added metric comparison functionality. [Lukasz Kreczko]\n- Fixed metric templates (files & performance) [Lukasz Kreczko]\n- Added metric examples. [Lukasz Kreczko]\n- Made tests more verbose. [Lukasz Kreczko]\n- Updated gitlab report config & name. [Lukasz Kreczko]\n- Updated template paths in demo report config. [Lukasz Kreczko]\n- Moved config into data folder. [Lukasz Kreczko]\n- Added draft for reporting. [Lukasz Kreczko]\n- Added status symbols to demo. [Lukasz Kreczko]\n- Moved templates to data folder. [Lukasz Kreczko]\n- Made default templates more resilient. [Lukasz Kreczko]\n- Added default report templates. [Lukasz Kreczko]\n- Added example report configs. [Lukasz Kreczko]\n- Improved development install. [Lukasz Kreczko]\n\n0.1.7\n------------\n- Add_file_metrics: added fix for Python3 & test. [kreczko]\n\n0.1.6\n------------\n- Execute_with_metrics: added fix for Python3 & test. [kreczko]\n\n0.1.5\n------------\n- Made run-clang-tidy Python3 compatible. [kreczko]\n- Fixed pep8 error in vis. [kreczko]\n\n0.1.4\n------------\n- Disabled log scale for diff plot. [kreczko]\n- Setting minY to non-zero for logarithmic plots. [kreczko]\n- Disabled logY setting if negative values are found. [kreczko]\n\n0.1.3\n------------\n- Execute_with_metrics: added soft-fail to IOException on writing\n metrics file. [kreczko]\n- Added tests for get_target_branch. [kreczko]\n- Fix missing import. [kreczko]\n\n0.1.2\n------------\n- Improved error-handling for get_target_branch and added target &\n default branches. [kreczko]\n- Worked through the stricter pep8 set. [Lukasz Kreczko]\n- Added default target_branch to get_target_branch. [Lukasz Kreczko]\n\n0.1.1\n------------\n- Visualisation adjustments for root_diff. [kreczko]\n- Fixed python3 issues with io._walk. [kreczko]\n\n0.1.0\n------------\n- Replaced io._walk with a more robust equivalent. [kreczko]\n- Added automated logy & x-limits to drawing. [kreczko]\n- Added code to generate tests/samples/objects.root. [kreczko]\n- Steeled diff calculation, fixed normalisation (now w.r.t. reference)\n and reporting non-comparible branches in root_diff. [kreczko]\n- Made io.unpack more robust to str-arrays. [kreczko]\n- Fixed pep8 error: unused include in test_io. [kreczko]\n- Fixed \"make test\" [kreczko]\n\n0.0.7\n------------\n- Added comparison of object data to tests. [kreczko]\n- Updated io.walk to handle & unpack objects. [kreczko]\n\n\n0.0.6\n------------\n- Automatically expose commands directly to command line. [kreczko]\n- Made sure vector branches are flattened before comparison. [kreczko]\n- Updated tests for vector branch. [kreczko]\n- Added vector variable to test samples. [kreczko]\n\n0.0.5\n------------\n- Added image examples for root_diff. [kreczko]\n\n0.0.4\n------------\n- Added KS test to drawing. [kreczko]\n- Moved compare_two_root_files to compare.compare_two_root_files.\n [kreczko]\n- Fixed \"a\" branch in tests/samples/test_3.root. [kreczko]\n- Creating output/validate/test_1_3 in CI. [kreczko]\n- Removed print from io test. [kreczko]\n- Fixed pep8 errors. [kreczko]\n- Parametrised IO tests. [kreczko]\n- Switched \"make test\" from py.test to pytest. [kreczko]\n- Fixed test samples. [kreczko]\n- Added compare and vis packages to setup.py. [kreczko]\n- Moved draw_diff to vis.draw_diff. [kreczko]\n- _compare_mctruth --> _compare. [kreczko]\n- Moved _diff and _isOK to compare.difference & compare.is_ok. [kreczko]\n- Moved store_diff to io.save_array_to_file. [kreczko]\n- Moved walk function into skvalidate.io. [kreczko]\n- Added folder creation to validation step. [kreczko]\n- Added installation to validation step in CI. [kreczko]\n- Added six to substitute xrange. [kreczko]\n- Fixed pep8. [kreczko]\n- Added validation step to CI. [kreczko]\n- Added test samples & generation script. [kreczko]\n- Added auto-generated docs. [kreczko]\n\n0.0.3\n------------\n- Fix package description and align versions in __init__.py and\n setup.cfg. [kreczko]\n- Fixed pep8 errors. [kreczko]\n- Added get_artifact_url. [kreczko]\n- Added folder for static docs content. [kreczko]\n- Added root_diff draft. [kreczko]\n- Added remove_from_env command. [kreczko]\n- Updated documentation for all commands. [kreczko]\n- Fixed indentation in README. [kreczko]\n- Expanded on the features of scikit-validate. [Lukasz Kreczko]\n- Renaming CLI class. [Lukasz Kreczko]\n- Rename to scikit-validate. [Lukasz Kreczko]\n- Added missing variable to setup.py. [kreczko]\n\n0.0.2\n------------\n- Added version lookup in setup.py from lz_validation. [kreczko]\n- CI: simplyfied twine upload to pypi. [kreczko]\n- Fixed behaviour of get_target_branch for non-MR branches. [kreczko]\n- Added explitcit user and pw parameters for twine (upload_to_pypi.sh)\n [kreczko]\n- Fixed comment in upload_to_pypi.sh. [kreczko]\n- Fixed package distribution (was missing commands and io modules)\n [kreczko]\n- Moved pypi upload into separate script. [kreczko]\n- Fixed pypi deployment procedure. [kreczko]\n- Fix gitlab-ci.yml. [kreczko]\n- Added pypi upload. [kreczko]\n- Updated failing test. [kreczko]\n- Fixed linter errors. [kreczko]\n- Moved linting to quick-checks stage. [kreczko]\n- Added artifacts for build stage. [kreczko]\n- Added gitlab-ci.yml. [kreczko]\n- Added more tox setups. [kreczko]\n- Added add_file_metrics command. [kreczko]\n- Added general save_metrics_to_file to lz_validation.io. [kreczko]\n- Added command to merge JSON files. [kreczko]\n- Added script to run clang-tidy. [kreczko]\n- Added execute_with_metrics command. [kreczko]\n- Added command structure and get_target_branch command. [kreczko]\n- Initial commit. [kreczko]\n",
"bugtrack_url": null,
"license": "Apache Software License 2.0",
"summary": "Science validation toolkit",
"version": "0.4.4",
"project_urls": {
"Homepage": "https://gitlab.cern.ch/fast-hep/public/scikit-validate"
},
"split_keywords": [
"skvalidate"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "71767c3c4cfd159af4fb713831106820f976e987412cf0e14a59fd32b652ad3c",
"md5": "63c0d5915dc71554a10b620a7597628b",
"sha256": "de7d24533d0e996d9d1c2230f0db8af985dc8e167246645c3f776491cf4c51e7"
},
"downloads": -1,
"filename": "scikit-validate-0.4.4.tar.gz",
"has_sig": false,
"md5_digest": "63c0d5915dc71554a10b620a7597628b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 37088741,
"upload_time": "2024-04-17T16:16:52",
"upload_time_iso_8601": "2024-04-17T16:16:52.858782Z",
"url": "https://files.pythonhosted.org/packages/71/76/7c3c4cfd159af4fb713831106820f976e987412cf0e14a59fd32b652ad3c/scikit-validate-0.4.4.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-17 16:16:52",
"github": false,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"lcname": "scikit-validate"
}
JSON