VerityPy


NameVerityPy JSON
Version 2024.1.21 PyPI version JSON
download
home_pageNone
SummaryVerityPy is a library by Technik Interlytics for Better Data = Better Decisions. It is a Data Inspector, Remediator, Normalizer on large structured data sets. It combines curated human expert knowledge gained from in-depth forensic analysis and remediating data in many fields, with Machine Learning pattern extraction results to know where to look, what to look for, and what to do about problems found lurking deep within huge data sets.
upload_time2024-08-12 16:46:34
maintainerNone
docs_urlNone
authorNone
requires_python>=3.8
licenseLicense +++++++ copyright (c) 2024 Technik Interlytics (TM) all rights reserved Last update: 20240521 Technik Interlytics provides its software development libraries VerityPy and VerityDotNet under a license model to meet development and distribution needs of several types of projects. All types of subscriptions, purchases, and use whether free or paid must adhere to the terms presented in this license agreement. Terms and Conditions ==================== This License Agreement (“Agreement”) is by and between Technik Interlytics (“Technik”) and the person or entity licensing the Technik Software (“Company”). This Agreement consists of each Order and these Terms and Conditions (“Terms”). Overview ========== Technik is the owner and licensor of several software libraries and packages, as described by Technik on Technik’s website and in Technik’s marketing materials (the “Technik Software”). The Technik Software consists of multiple products (each, a “Product”), which may be licensed individually or as a suite of Products (the “TechnikSuite”), which are described in greater detail in an Order. Products and the TechnikSuite are referred to herein as Technik Software. Technik offers Company one or more options to use the Technik Software as documented by Technik subject to the restrictions stated in this Agreement. Technik may also provide support services (“Support and Updates”) and other services, including consulting services regarding Company’s use of the Technik Software (collectively with Support and Updates, the “Services”). Technik makes the Technik Software and Services available to Company directly or through a third party (such as an authorized reseller) using a written online or offline order process (each, an “Order”). Pursuant to the terms of this Agreement, Technik will provide to Company the Technik Software or Services described in each Order. 1. Technik Software and Services -------------------------------- During the term of this Agreement, Technik grants to Company and Company accepts a non-exclusive, nontransferable, revocable and limited license to use each item of Technik Software listed on an Order by Company for Company’s own internal use (and not for resale in a materially unaltered form), subject to the terms, obligations, and restrictions set forth in the Order and in this Agreement (“Technik Software License”). Each Technik Software License and each Service is subject to permissions, limitations and responsibilities stated in the applicable Order (“Metrics”), which may state whether Company is permitted to: * use the Technik Software for a limited trial only, for a stated time period, or perpetually; * evaluate the Technik Software or use it in a production environment; * use the Technik Software at no charge or for a stated fee; * permit access to the Technik Software to an unlimited number of authorized users or to a limited number of authorized users; * install and use the Technik Software in one computing device (“Machine”) or multiple Machines; * use a single copy or multiple copies of the Technik Software (“Copies”); * use the Technik Software solely for internal development use (“Internal Use”); * combine the Technik Software with Company’s own software (“Company Software”); * distribute to one or more of Company’s customers (each, a “Customer”); * install the Technik Software in Company’s or Customer’s systems (“Installed”) or made available by Company on a website owned or controlled by Company as software as a service (“SaaS”); * use the Technik Software in one or more city (each, a “Location”); * receive Support and Updates for a limited period of time or not at all; or * receive new versions of the Technik Software made available by Company to the general public (“New Versions”) while Support and Updates is current and active, or receive New Versions only when Company pays additional fees. Currently, Technik offers Technik Software Licenses in the following combination of Metrics (each license type expressly excludes any rights not specifically granted): * Free Non-Commercial License. Grants the use of the Technik Software for private purposes only when the Company does not receive payment for its software or services using the software (Non-Commercial License”). The Technik Software may be deployed to any number of web applications, intranet applications, or desktop software applications. This license type does not allow distribution of the Technik Software to third parties, clients of the Company, or SaaS project usage scenarios. * Free Trial License. Grants the use of the Technik Software for private non-production evaluation purposes only (“Trial License”). The default period for this license is thirty (30) days but the specific order may include a different binding period. The Technik Software may be deployed to any number of web applications, intranet applications, or desktop software applications. This license type does not allow distribution of the Technik Software to third parties, clients of the Company, or SaaS project usage scenarios. * Developer License Annual. Grants the use of either (i) a specific Technik Software Product or (ii) the TechnikSuite by a single software developer in an organization at a single Location for a period of one year. The Technik Software may be deployed to any number of web applications, intranet applications, or desktop software applications. This license type does not allow distribution of the Technik Software to third parties, clients of the Company, or SaaS project usage scenarios. * Developer License Perpetual. Grants the use of either (i) a specific Technik Software Product or (ii) the TechnikSuite by a single software developer in an organization at a single Location without an expiration. The Technik Software may be deployed within to any number of web applications, intranet applications, or desktop software applications. This license type does not allow distribution of the Technik Software to third parties, clients of the Company, or SaaS project usage scenarios. * Royalty-Free Redistribution Coverage. This add-on, which may be purchased under an Order, grants Company the right to distribute the Technik Software (without any duty to pay royalties) as part of a number of distinct packaged commercial or open-source products to third parties, in accordance with the number of projects covered in the base license (“Royalty-Free Redistribution Coverage”). Grants the use of the Technik Software to deploy within SaaS software services, in accordance with the number of projects covered in the base license. * Company may subscribe separately to Support and Updates, which Technik may modify in it its sole discretion, on either a 1-year or multi-year basis. Then-current pricing for Support and Updates is set forth in an Order. * Unlimited Enterprise License. This license includes Royalty-Free Redistribution Coverage. The Unlimited Enterprise License is perpetual, subject to Company’s compliance with the License Agreement (“Unlimited Enterprise License”). The Unlimited Enterprise License permits an unlimited number of developers in an organization in an unlimited number of Locations to deploy the TechnikSuite within an unlimited number of web applications, intranet applications, or desktop software applications. * Unlimited Monthly License. This license includes Royalty-free Redistribution Coverage. The Unlimited Monthly License permits the use on a subscription basis by an unlimited number of developers in an organization in an unlimited number of Locations to deploy the TechnikSuite within an unlimited number of web applications, intranet applications, or desktop software applications (“Unlimited Monthly License”). * All Licenses are non-transferable and sharing of licenses outside an organization or agency/client relationship is prohibited. Licenses expressly exclude any rights not specifically granted under the Agreement, and expressly exclude, without limitation, OEM redistribution, and using the Technik Software as SaaS, without additional purchase of Royalty-Free Distribution Coverage. Regardless of the type of Technik Software License, Technik and Company expressly agree that Company will not directly compete with the functionality of any current Technik Software or Service. * Company may not redistribute, republish, or otherwise make available the Technik Software or Services to any third party without the prior written consent of Technik as stated in an Order or this Agreement. * Company will monitor its own use of the Technik Software and report any use in excess of the Metrics and volume. Technik Software may monitor use to verify with Technik servers compliance with Metrics, volume, and the Agreement. Notwithstanding the foregoing, no Company or end-user data is ever transmitted to Technik servers. * Inclusion of Technik Software in a package must include this license in its entirety or a statement of this license and a link to its full text on the Technik website * Inclusion of Technik Software in a package must include a statement of Technik's copyright for the components of the package from Technik * Inclusion of Technik Software with open source software or made into a new work distributed as open source software will not amend the non-open source license of Technik Software which must be stated explicitly in the new work's license or notice. 2. Data and Content --------------------- Company retains any and all rights in any information, content, and data provided by Company, including all rights in new versions and derivative works of Company’s information, content, and data. Technik retains any and all rights in any information, content, and data provided by Technik, including all rights in new versions and derivative works of Technik’s information, content, and data. Technik may collect, store, use, aggregate, and share information about Company and its customers, including about how Company and its customers use the Technik Software and any Services. Details about how Technik currently processes Company data are stated in Technik’s Privacy Statement, which Technik may modify from time to time to reflect Technik’s current practices and in response to evolving laws and regulations around the world. 3. Financial --------------- Company payment shall be as outlined in each Order. Unless otherwise stated in the Order, payment in full is due to be received by Technik on or prior to delivery of the Technik Software or Services. All fees and payments are nonrefundable. Technik reserves the right to increase fees for Unlimited Monthly Licenses, and Support and Updates, and will provide notice to Company of any increase within 30 days’ of the effectiveness of such increase. All of the prices hereunder are exclusive of any taxes. Company shall be responsible for any federal, state, or local sales, use, Canadian Goods and Services, property, VAT, or similar taxes that are or may be imposed on transactions. In addition to all of its other rights and remedies, if payment is not made by the due date, then Technik may suspend or terminate performance of any or all Services, and suspend or terminate the Technik Software License on all Technik Software. Any amounts not paid by the due date will accrue late payment charges in the amount of one percent (1.0%) per month of the overdue amount. In addition, Technik will have the right to recover all costs of collection, including attorneys’ fees and expenses, accrued or incurred by Technik in any court or other tribunal action to collect unpaid amounts due. Non-payment of recurring monthly subscription fees for more than thirty (30) days after the due date shall result in suspension or termination of Technik Software and Services. 4. Intellectual Property -------------------------- The Technik Software and the results of any Services are owned solely by Technik and are protected by copyright, trademark, and trade secret law, among other types of law. All rights, titles and interests to the Technik Software and Services reside in and shall remain with Technik. Company receives only the limited and non-exclusive use rights expressly stated in the Order and in this Agreement. Company may not sell, license, assign, pledge, or otherwise transfer the Technik Software or Services, or any copies thereof, to any third party, or permit or allow any third party to use the Technik Software or Services, except as expressly permitted by this Agreement. Any unauthorized sale, sublicense, assignment, pledge, other transfer, or impermissible use of the Technik Software or Services shall be void and will be a violation of Technik’s rights in the Technik Software and Services. Company acknowledges that without Technik’s written permission in an Order: (i) Company may not duplicate the Technik Software or Services for resale, publication, transfer, conveyance, licensing or sublicensing, or redistribution to any third party; (ii) Company will neither duplicate the Technik Software or Services for redistribution to any third party; (iii) Company will not duplicate or otherwise dispose of the Technik Software of Services in any manner violative of the United States Copyright Act (Title 17, U.S. Code); (iv) the Technik Software and Services may be used only by the aggregate number of authorized users at any given time; (v) the Technik Software and Services may be installed, accessed and used only on as many processors as there are authorized users; (vi) if the Company uses networked equipment, Company will never allow more than the maximum number of authorized users to use the Technik Software or Services at any given time; and (vii) Company may not modify, disassemble, decode, or decompile the Technik Software or Services, in whole or in part. Regardless as to the number of authorized users or uses, Company may make one (1) copy of the Technik Software as an archival or backup copy, which shall bear Technik’s copyright notice and other proprietary markings. Company shall make no other copies of the Technik Software for any purpose. Company shall submit to Technik for approval, prior to use, distribution, or disclosure, any advertising, promotion, or publicity that refers to Technik or that uses the trade names, trademarks, or service marks of Technik (“Technik Marks”). Technik shall have the right to require at its discretion the correction or deletion of any incorrect or misleading material regarding Technik or the Technik Marks in any advertising, promotion, or publicity. Technik may use the name and trademark of Company (“Company Marks”) to list or highlight Company as a licensee of Technik. The use of each Mark, and the goodwill generated thereby, shall inure to the benefit of the owner of the Mark. 5. Confidentiality -------------------- For purposes of this Agreement, “Confidential Information” shall mean non-public information and data received by either party (“receiving party”) from the other party (“disclosing party”). Notwithstanding the foregoing, information and data shall not be deemed to be Confidential Information if such information or data: (i) was known by the receiving party at the time of such disclosure; (ii) was known to the general public at the time of such disclosure or becomes known to the general public (other than by act of the receiving party) subsequent to such disclosure; (iii) is disclosed lawfully to the receiving party by a third party without restriction; (iv) is developed independently by the receiving party without reference to the disclosing party’s Confidential Information; or (v) is approved in writing by the disclosing party for disclosure by the receiving party. The receiving party may disclose the disclosing party’s Confidential Information when the receiving party is required by law (e.g., by subpoena), provided however the receiving party will give the disclosing party prior written notice of such required disclosure (unless notifying the disclosing party is prohibited by law), and will work with the disclosing party to limit the required disclosure or have the requirement revoked (e.g., a court revokes a subpoena for the Confidential Information). The receiving hereunder shall not disclose to any third-party firm, corporation, individual, or other entity, any Confidential Information which it receives from the disclosing party. For clarity, Technik may disclose the Company’s Confidential Information to Technik’s employees and independent contractors in connection with providing the Technik Software and performing the Services hereunder. The receiving party shall use the same degree of care in safeguarding the Confidential Information as the receiving party uses for its own confidential and proprietary information, but in no event less than a standard of reasonable care. 6. Term and Termination -------------------------- For each item of Technik Software and each Service, the initial term will be stated in the Order, and may renew for additional periods stated as Renewal Terms on the Order. If no Initial Term is specified on the Order, then the Initial Term will be one year from the Effective Date. If no Renewal Term is specified on the Order, then each Renewal Term will be one year beginning with the expiration of the preceding term, but only if Technik receives any renewal payment at least thirty (30) days prior to the first day of the applicable Renewal Term. This Agreement may be terminated by either party for material breach by the other party provided that the terminating party gives thirty (30) days prior written notice specifying the breach, and the breaching party fails to cure or correct the breach within the thirty-day notice period. In the event of any such termination, Company shall pay Technik for all Technik Software provided and for all Services performed by Technik and for all work-in-progress up to the date of termination. Company may terminate an Unlimited Monthly License upon thirty (30) days written notice prior to the start of the next month of the subscription term. Upon Technik’s receipt of a termination notice by Company, the Unlimited Monthly License subscription shall terminate either when the maximum API calls permitted under the Order have been made or the end of the month in which such termination was effective. This Agreement shall be deemed to be automatically terminated upon any material breach of Company’s obligations under Sections 1 (Technik Software and Services), 3 (Financial), 4 (Intellectual Property), or 5 (Confidentiality). Upon termination of this Agreement for any reason, Company shall immediately return to Technik any Technik Software, related materials, and all copies thereof or, with Technik’s prior written permission, Company shall destroy all such materials and certify in writing as to their destruction. Sections: Financial, Intellectual Property, Confidentiality, Term and Termination, and Representations and Warranties, Disclaimers and Limitations of Liability, Indemnification, and General Provisions, of this Agreement shall survive any expiration or termination of this Agreement. Company recognizes that Technik has expended considerable amounts of time, effort, and money to develop the Technik Software and Services, and that Company’s unauthorized copying, use, transfer, or disclosure of the Technik Software or Services, or their contents, may cause Technik to sustain substantial, irreparable harm and damage. Similarly, each party has expended considerable amounts of time, effort, and money to develop and protect their respective Confidential Information. In addition to all other legal and equitable remedies available to a party, each party may seek from an arbitrator (pursuant to the arbitration provisions of this Agreement) temporary and permanent injunctive relief to remedy any breach of the other party’s obligations under Sections Intellectual Property, or Confidentiality of this Agreement. 7. Representations and Warranties ---------------------------------- The parties hereby covenant, represent, and warrant to each other that they are duly authorized and empowered to enter into this Agreement, and that this Agreement constitutes a valid and binding, enforceable Agreement. Company further represents covenants and warrants that this Agreement is not inconsistent with any Agreement or obligation binding Company or its property or assets. 8. Disclaimers and Limitations of Liability -------------------------------------------- THE Technik SOFTWARE AND SERVICES ARE PROVIDED “AS IS” AND “AS AVAILABLE” WITH ALL FAULTS. Technik SHALL NOT BE LIABLE FOR ANY DAMAGES, LOSSES, OR EXPENSES, OF ANY TYPE, KIND OR DESCRIPTION, INCURRED OR SUSTAINED BY COMPANY OR BY ANY OTHER PERSON OR ENTITY FOR ANY REASON. THE WARRANTIES STATED IN THIS AGREEMENT ARE EXCLUSIVE AND IN LIEU OF ANY OTHER WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING WITHOUT LIMITATION ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FOR FITNESS FOR A PARTICULAR PURPOSE. NEITHER Technik NOR ITS AFFILIATES, NOR ANY OF THEIR RESPECTIVE THIRD PARTY LICENSORS MAKE ANY WARRANTIES, EXPRESS OR IMPLIED, AS TO RESULTS TO BE OBTAINED FROM THE USE OF ANY Technik SOFTWARE OR SERVICES OR THAT THE Technik SOFTWARE OR SERVICES WILL BE ERROR-FREE. IN NO EVENT SHALL Technik NOR ITS AFFILIATES, NOR ANY OF THEIR RESPECTIVE THIRD-PARTY LICENSORS HAVE ANY LIABILITY FOR LOST PROFITS OR FOR INDIRECT, SPECIAL, PUNITIVE OR CONSEQUENTIAL DAMAGES, OR ANY LIABILITY TO ANY PERSON ARISING OUT OF THIS AGREEMENT, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES OR LIABILITY. IN NO EVENT SHALL THE CUMULATIVE LIABILITY OF Technik, ITS AFFILIATES, AND THEIR RESPECTIVE THIRDPARTY LICENSORS HEREUNDER EXCEED THE GREATER OF (I) THE AMOUNTS RECEIVED BY Technik FROM COMPANY DURING THE TWELVE (12) MONTHS PRIOR TO THE MAKING OF THE CLAIM, AND (II) $100, WHICH SHALL BE COMPANY’S EXCLUSIVE REMEDY AGAINST Technik, ITS AFFILIATES AND THEIR RESPECTIVE THIRDPARTY LICENSORS UNDER THIS AGREEMENT. All disclaimers herein shall not be applicable to liabilities that cannot be waived under the applicable laws or regulations of the United States (including US State and Federal law) or any other applicable country or jurisdiction. 9. Indemnification --------------------- Should any portion of the Technik Software become, or in Technik’s opinion be likely to become the subject of a claim of infringement, Technik may, as Company’s sole and exclusive remedy, elect to (1) obtain for Company at Technik’s expense the right to use such portion, or (2) replace or modify the Technik Software so that it becomes non-infringing, or (3) remove the infringing portion and grant to Company a credit prorated to reflect the portion of the overall fees paid by Company attributable to such portion of the Technik Software. The foregoing states the entire liability of Technik with respect to allegation or claim of infringement of any intellectual property rights. Company will defend, indemnify, and hold Technik harmless at Company’s expense for any claim that alleges that the Company Software, Company’s data, or Company’s materials infringes upon a copyright, patent or other right, and Company will pay all damages and costs awarded by a court in connection with such claim. In the event Company redistributes, republishes, or otherwise permits use of the Technik Software or Services, as provided by Technik or as modified by Company, in violation of this Agreement, Company shall indemnify, defend, and hold Technik harmless from any loss, damage or claim arising therefrom. Each indemnification claim requires that Technik: (i) gives prompt written notice of the claim to the Company; (ii) gives Company sole control of the defense or settlement of the claim; and (iii) provides to Company all necessary information, assistance, and authority to defend. Company shall promptly defend or settle the claim at Company’s sole cost and with Company’s counsel, but Company shall not have the right to admit liability on behalf of Technik, or assess any blame, responsibility, costs, or fees to Technik. 10. Entire Agreement. --------------------- The parties acknowledge and agree that this Agreement (including each Order) embodies the complete and exclusive understanding and Agreement of the parties with respect to the Technik Software and Services, and supersedes any prior or contemporaneous proposal, agreement, or license whether oral or written, and any other communication between the parties. Changes to this Agreement. This Agreement shall not be modified or amended except by a written instrument, signed by both parties. The version of this Agreement posted to Technik’s website is the current set of Terms that Technik may modify from time to time. For each Order placed by Company, the current version of this Agreement will apply to that Order, unless the parties have otherwise signed a hard copy or digital version of these Terms independent of the web-based Terms, in which case the signed version of the Terms will control. 11. Export Control. ------------------- Company will comply with all export control laws and regulations of the United States and all other countries and jurisdictions. Company will not remove or export from the United States or allow the export or re-export of any part of the Technik Software or any direct product thereof: (a) into (or to a national or resident of) any embargoed or terrorist-supporting country; (b) to anyone on the U.S. Commerce Department's Table of Denial Orders or U.S. Treasury Department's list of Specially Designated Nationals (or any list that replaces such list); (c) to any country to which such export or re-export is restricted or prohibited, or as to which the United States government or any agency thereof requires an export license or other governmental approval at the time of export or re-export without first obtaining such license or approval; or (d) otherwise in violation of any export or import restrictions, laws or regulations of any United States or foreign agency or authority that has jurisdiction. Company agrees to the foregoing and warrants that it is not located in, under the control of, or a national or resident of any such prohibited country or on any such prohibited party list. The Technik Software is further restricted from being used for terrorist activity, or for the design or development of nuclear, chemical, or biological weapons or missile technology without the prior permission of the U.S. government. 12. Governing Law. ------------------ This Agreement shall be governed by and construed in accordance with the laws of the State of Texas and the federal laws of the United States, without reference to conflicts of laws provisions and policies. Any dispute between the parties shall be determined by one arbitrator in binding arbitration administered by the American Arbitration Association (adr.org) if all parties are based in the United States; otherwise, the arbitration shall be administered by the International Centre for Dispute Resolution (icdr.org). The arbitrator shall decide all questions of arbitrability, and shall be empowered to issue preliminary, temporary, and injunctive relief by issuing an initial award. All hearings will be conducted by audio conference or video conference. The language of the arbitration shall be English. The arbitrator’s awards will be binding on the parties and may be entered and enforced in any courts or tribunals of competent jurisdiction. 13. Severability. -------------------- If any provision of this Agreement shall be determined to be void or unenforceable by a court of competent jurisdiction, the remaining provisions shall remain in full force and effect. 14. Waiver. ------------ Any waiver of a breach or of performance of a term or condition of this Agreement shall be in writing, signed by both parties hereto. The failure of either party to insist on strict adherence to any term of this Agreement on any occasion shall not be deemed a waiver of its right thereafter to insist on such adherence. 15. Headings. ------------- Headings are for convenience only and are not part of this Agreement. They shall not be used to modify or construe the terms of the sections they introduce. 16. Notice. ------------ All notices required to be given by this Agreement shall be sent by certified mail, return receipt requested, or by courier, to the party to be notified at the address set forth in the Order, or to such other or new address as to which notice has been given. All notices made under this Agreement shall be effective twenty calendar days after mailing. 17. Successors and Assigns. ----------------------------- This Agreement is not assignable by either party without the other party’s written permission and is binding upon the permitted successors of each party to this Agreement. However, either party may assign no less than all of its rights under this Agreement to an affiliate or successor as a result of a merger, acquisition, or reorganization, and the assignment will be effective on receipt of written notice by the nonassigning party. 18. Independent Contractor. ---------------------------- The relationship of the parties is that of independent contractors. This Agreement does not create any actual or apparent agency, partnership, or relationship of employer and employee between the parties hereto. 19. Force Majeure. ------------------- Except for a party’s payment obligations, neither party shall have any liability for any defaults or delays resulting from circumstances reasonably beyond its reasonable control. 20. Company Documents. ---------------------- The provisions and terms of any document issued by Company in conjunction with this Agreement shall be of no effect and shall not in any way extend, affect, or amend the terms and conditions set forth in this Agreement (including any Order) unless expressly accepted in writing by Technik. **Technik Interlytics** https://technikinterlytics.com support@technikinterlytics.com
keywords data quality data profiling data enrichment data normalization data remediation human data expert data intelligence embedded delimiter data cleaning data veracity data prep data wrangling data forensics data remediation data science data blending structured data metadata broken records parsing error
VCS
bugtrack_url
requirements No requirements were recorded.
Travis-CI No Travis.
coveralls test coverage No coveralls. 
            Overview
========

**VerityPy** is a Python library by Technik Interlytics for Better Data = Better AI. 
It is a Data Inspector, Remediator, Normalizer on large structured data sets. 
It combines curated human expert knowledge gained from in-depth forensic analysis and remediating data in many fields, 
with Machine Learning pattern extraction results to know where to look, what to look for, and what to do about problems. 
It automates labor-intensive, difficult tasks to improve data quality, remediate errors, 
and enrich data sets ML, Data Science, analytics, forensics, auditing, and warehousing.

|
|

Structured Data Functions
==========================

There are two main types of functions provided on structured data sets:

1. Analyzing: inspects data records for structural and value consistency, 
   anomalies, errors especially deeply buried problems that may only occur in a few records within millions 
   but enough to cause significant bias, discrepancies, and failed validations. Our human experts have discovered many problems common 
   in data systems across business and technical fields that are either not detected or cannot be remediated with standard DataOps. Some examples:

      * data variations (types, formats, encoding) from import/export in different systems especially spreadsheets and legacy mainframes
      * special characters not visible to users causing downstream problems
      * small number of anomalies buried within very large data sets overwhelming tools
      * mismatched joint field values such as geographic location codes and names
      * long strings of digits used as codes (e.g. accounting, ERP) cast into number format stripping digits thereby corrupting codes
      * records broken into multiple lines causing fields to be misaligned, partial records, and incorrect values
      * open source data with embedded information-only records (e.g. IRS USA Migration demographics, Covid disease census) unknown to users

|

2. Normalizing & Enriching: this encompasses everything necessary to get data records 
   accurate for structure (field positions, data types, formats), 
   values (numeric ranges, codes, lists of allowed values), and enrichment 
   (added metadata fields for forensics, training models, analytics). 
   Our human experts have worked extensively in 
   the entire lifecycle of data in multiple fields allowing them to see and understand how and where data can become infected with 
   problems and both the indicators of these problems and how to correct and annotate them to pass scrutiny by human and 
   computer QC. Some examples are:

      * rebuilding records broken into multiple lines
      * adding enrichment fields to annotate pedigree, trust, privacy, increased granularity with conditional logic and controlled vocabulary
      * several levels of conditional testing of multiple fields within single records to correctly encode/decode, transform field values
      * allowing multiple versions of lookup decoding per field based on other field indicators (e.g. time varying encoding schemes)
      * identifying when long values of numeric digits are strings or numbers and handling accordingly
      * lookup dictionary replacements using 1 or multiple fields as well as wildcard tokens and both boolean AND and NOT conditions

|
|

Why Use **VerityPy**
=======================

It was created to solve the need to not just make data accurate, but also provide high visibility into how and why it 
was handled with transforms and codings such that it can be managed like other key business assets with oversight and collaborative review. 
All too often, the actual manipulation of data is handled by proficient engineers but the people who most need to review, understand, and adjust 
what is done are unable to decipher the complicated code, scripts, and technical documentation. Our human experts witnessed this situation in 
many clients and had to solve this challenge before the technical results would be accepted. Doing so led us to develop new data processing and 
reporting approaches that jointly handled complicated data engineering requirements along with visible and easy to understand business reporting. 
**VerityPy** was created to provide this capability to a wide community with the following key concepts:

   * easily reuse and adjust processing parameters for multiple iterations of transforms, codings, rules with reviews of results in end-use applications
   * review data processing steps and intermediate results throughout the entire process (i.e. no black boxes)
   * use processing commands that can be reviewed by business and technical people at both staff and manager levels
   * enable drop-in reporting with summary and detailed charts and tables of data actions, discoveries, and results
   * provide multiple views of data before and after to maximize understanding and discovery among all user types


|
|

What **VerityPy** Does
=======================


Analysis
-----------

**VerityPy** analyzes structured source data and generates a thorough assessment of each field's 
actual value space for data types, formats, ranges, special characters, unique values and even coValues 
which are joint field (2 or 3) value distributions. This is a quick way to both profile source 
data, extract its schema, and discover anomalies that can be overlooked by other tools or 
missed by manual Quality Control reviews. A comprehensive report is returned in a Python object 
that can be used in further processing to make tables and charts such as in Jupyter Notebook.



Goal
+++++++++

The goal of analysis is to characterize data to know its structure, range of values, 
and presence of anomalies in relation to what it should be as defined by its architecture. 
Ideally, the documentation would describe the details of how it was collected, stored, and 
the meaning of the data in the context of its intended use. In this ideal case, unit tests could be 
made to automatically measure quality metrics both as the data is received and processed, and 
as it is distributed and used. Unfortunately, this ideal situation rarely exists and we are forced 
to manage data with uncertain quality, pedigree, and trustworthiness. When the use can tolerate 
imperfect data then this is not much of a problem. However, we now have increasingly stringent needs for 
better data to feed Artificial Intelligence (AI), Data Science (DataSci), and more sophisticated 
forecasting models in financial markets, global supply chain, consumer activity, and many others.




Objectives
++++++++++++++

We have learned from in-depth analysis and thorough reconstruction of data sets across many fields 
and types of data systems that there are several specific types of anomalies that frequenetly exist and 
go undetected by even the most modern tools. Part of this expert assessment included following and measuring 
the impact of the imperfect data on the end-use business activities for how outcome errors impacted 
decision making, audit, compliance, analytics, forecast accuracy, etc. From this we created a combined 
human expert and big data Machine Learning (ML) technology to filter through all data to find several types 
of problems and reliable approaches to correcting them automatically. This led to the following key objectives 
of the VerityX (X denotes both the Python and DotNet libraries) analysis process:

    * capture details of field datatype and format for all records and visibly showcase even infrequent variations.
         - many tools limit the depth and breadth of records analyzed and variations captured due to processing, memory, and storage limitations.
    * capture complete range of values for each field and emphasize low frequency instances since this is how anomalies can be quickly discovered.
    * capture complete range of values for combinations of several fields as another quick way of detecting 
      anomalies and overall quality of fields that are linked by what they mean in end use cases.
    * track number of field values parsed per record as key indicator of presence of extra delimiters and line feeds 
      that are not apparent during human review but which cause parsing code to break what should be a single record 
      into multiple partial records or generate more field values than there are fields. This is surprisingly common 
      in many mid and large size data systems. 
    * provide automated correction algorithms that repair all of the above problems with minimal required data 
      architecture and engineering which tends to be so complicated and labor intensive that it often lags 
      actual data cuasing serious 'technical debt'.
    * provide multiple views and types of results into data quality and problems since real world data teams are typically 
      too constrained in time and personnel to probe every data set, system, and operation in detail.





Results
+++++++++

Results are coordinated in a Python class 'QualityAnalysis' allowing concise handling 
of the setup parameters and the breadth and depth of discovered characteristics and 
known/suspected errors. These results include:

   * field unique values: per field unique values with count of instances.
   * field datatype distributions: each field has counts for detected datatypes (int, real, bool, date, string, empty).
   * field quality: each field is assigned a quality factor 0-100 based on discovered characteristics and knowledge-based algorithms.
   * record size distribution: record sizes (byte lengths) to count of instances.
   * record parsing errors: parsing errors (number fields after parsing relative to defined fields) 
     by small1 (1 too few fields), small2 (2 or more missing fields), big (1 or more too many fields). Also, has example records.
   * record parsing distribution: number of parsed fields to count of instances.
   * special character distribution: special characters and their count of instances, as well as example records.
   * coValues: field combinations (2 or 3) unique value information. 
   * error statistics: values such as number records with any kind of error, number records 
     with datatype error, number records with format error and more



Normalize & Enrich
-------------------

**VerityPy's** transforms allow Normalizing and Enriching source data with 
a higher level of quality, accuracy, and meaning to support demanding use cases. There are five 
kinds of transforms (see transforms page for details):

   1. Assignment: assigns values to field as a fixed value, reference to another field in record, random number, list of categories via frequencies, lookup dictionaries
   2. Conditional: conditional tests of equality and inequality for numeric, string, and date values
   3. Numeric: numeric calculation functions including using other fields in record by reference
   4. Text: manipulate with slicing, adding, padding, replacing
   5. Date: Change date format to ISO 8601 including from special Excel format 

This is an example of a transform to populate an enrichment field 'useAGI' that denotes whether the record should be used 
in analytics based on the value of a numeric source field 'AGI'.

   1. setToRef("AGI")
   2. ifEq("-1")
   3. setToValue("true")
   4. setToValue("false")

In order to allow chaining of conditional functions, the flow is condition -> [false action] else [true action]. Thus, if step 2 above is False 
then step 3 is done and the chain stops, whereas if step 2 is True then step 3 is skipped and step 4 is done (and any steps after it if they existed). 
The net result is this simple transform fills an enrichment field with boolean value enabling easy filtering downstream in a spreadsheet, database, 
or analytics dashboard.

A slightly more complicated logic flow that includes fixing formatting is the following transform that uses a source field 'y2_statefips' containing a 2 character 
code to lookup the corresponding title in an external lookup dictionary and then assigns that to an enrichment field 'DestStateName' since the 
original source data only had the code making it non-intuitive for users to understand the data records. 

   1. setToRef("y2_statefips")
   2. setLength("2","left","0")
   3. lookup("StateName")

Step 1 gets the value of the field 'y2_statefips' from the current record. Step 2 fixes the string length to 2 characters with changes made 
to the left side of the string if it is too long (characters cut from left) or too short (characters added to left) with the character to 
add set to be '0' (zero). This is critical for code lookups since a very common problem when data is moved among systems is for leading 
zeros to be removed thereby changing a code like '01' into '1' which would not be found in the lookup. This ensures that such an error 
is fixed prior to doing the lookup which occurs in step 3 to a dictionary name 'StateName' (loaded during the setup phase of the job).

|
|

License
===========

This is not open source software and cannot be included in an open source project as its license will break the open source license. 
However, there is a license allowing free use for non-commercial, personal applications. 
Read the license file for full details about allowed scope of free use. 
Paid licenses are required for commercial products either distributed or web hosted (e.g. SaaS), as well as enterprise applications with multiple users. 
There are licenses for developers, royalty inclusion in other products, and support.

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "VerityPy",
    "maintainer": null,
    "docs_url": null,
    "requires_python": ">=3.8",
    "maintainer_email": null,
    "keywords": "data quality, data profiling, data enrichment, data normalization, data remediation, human data expert, data intelligence, embedded delimiter, data cleaning, data veracity, data prep, data wrangling, data forensics, data remediation, data science, data blending, structured data, metadata, broken records, parsing error",
    "author": null,
    "author_email": "Geoffrey Malafsky <gmalafsky@technikinterlytics.com>",
    "download_url": "https://files.pythonhosted.org/packages/02/ec/bc570af4a69fbe43e979c9b2ca1f6b7e108dce49b7557fab8a15eb118677/veritypy-2024.1.21.tar.gz",
    "platform": null,
    "description": "Overview\r\n========\r\n\r\n**VerityPy** is a Python library by Technik Interlytics for Better Data = Better AI. \r\nIt is a Data Inspector, Remediator, Normalizer on large structured data sets. \r\nIt combines curated human expert knowledge gained from in-depth forensic analysis and remediating data in many fields, \r\nwith Machine Learning pattern extraction results to know where to look, what to look for, and what to do about problems. \r\nIt automates labor-intensive, difficult tasks to improve data quality, remediate errors, \r\nand enrich data sets ML, Data Science, analytics, forensics, auditing, and warehousing.\r\n\r\n|\r\n|\r\n\r\nStructured Data Functions\r\n==========================\r\n\r\nThere are two main types of functions provided on structured data sets:\r\n\r\n1. Analyzing: inspects data records for structural and value consistency, \r\n   anomalies, errors especially deeply buried problems that may only occur in a few records within millions \r\n   but enough to cause significant bias, discrepancies, and failed validations. Our human experts have discovered many problems common \r\n   in data systems across business and technical fields that are either not detected or cannot be remediated with standard DataOps. Some examples:\r\n\r\n      * data variations (types, formats, encoding) from import/export in different systems especially spreadsheets and legacy mainframes\r\n      * special characters not visible to users causing downstream problems\r\n      * small number of anomalies buried within very large data sets overwhelming tools\r\n      * mismatched joint field values such as geographic location codes and names\r\n      * long strings of digits used as codes (e.g. accounting, ERP) cast into number format stripping digits thereby corrupting codes\r\n      * records broken into multiple lines causing fields to be misaligned, partial records, and incorrect values\r\n      * open source data with embedded information-only records (e.g. IRS USA Migration demographics, Covid disease census) unknown to users\r\n\r\n|\r\n\r\n2. Normalizing & Enriching: this encompasses everything necessary to get data records \r\n   accurate for structure (field positions, data types, formats), \r\n   values (numeric ranges, codes, lists of allowed values), and enrichment \r\n   (added metadata fields for forensics, training models, analytics). \r\n   Our human experts have worked extensively in \r\n   the entire lifecycle of data in multiple fields allowing them to see and understand how and where data can become infected with \r\n   problems and both the indicators of these problems and how to correct and annotate them to pass scrutiny by human and \r\n   computer QC. Some examples are:\r\n\r\n      * rebuilding records broken into multiple lines\r\n      * adding enrichment fields to annotate pedigree, trust, privacy, increased granularity with conditional logic and controlled vocabulary\r\n      * several levels of conditional testing of multiple fields within single records to correctly encode/decode, transform field values\r\n      * allowing multiple versions of lookup decoding per field based on other field indicators (e.g. time varying encoding schemes)\r\n      * identifying when long values of numeric digits are strings or numbers and handling accordingly\r\n      * lookup dictionary replacements using 1 or multiple fields as well as wildcard tokens and both boolean AND and NOT conditions\r\n\r\n|\r\n|\r\n\r\nWhy Use **VerityPy**\r\n=======================\r\n\r\nIt was created to solve the need to not just make data accurate, but also provide high visibility into how and why it \r\nwas handled with transforms and codings such that it can be managed like other key business assets with oversight and collaborative review. \r\nAll too often, the actual manipulation of data is handled by proficient engineers but the people who most need to review, understand, and adjust \r\nwhat is done are unable to decipher the complicated code, scripts, and technical documentation. Our human experts witnessed this situation in \r\nmany clients and had to solve this challenge before the technical results would be accepted. Doing so led us to develop new data processing and \r\nreporting approaches that jointly handled complicated data engineering requirements along with visible and easy to understand business reporting. \r\n**VerityPy** was created to provide this capability to a wide community with the following key concepts:\r\n\r\n   * easily reuse and adjust processing parameters for multiple iterations of transforms, codings, rules with reviews of results in end-use applications\r\n   * review data processing steps and intermediate results throughout the entire process (i.e. no black boxes)\r\n   * use processing commands that can be reviewed by business and technical people at both staff and manager levels\r\n   * enable drop-in reporting with summary and detailed charts and tables of data actions, discoveries, and results\r\n   * provide multiple views of data before and after to maximize understanding and discovery among all user types\r\n\r\n\r\n|\r\n|\r\n\r\nWhat **VerityPy** Does\r\n=======================\r\n\r\n\r\nAnalysis\r\n-----------\r\n\r\n**VerityPy** analyzes structured source data and generates a thorough assessment of each field's \r\nactual value space for data types, formats, ranges, special characters, unique values and even coValues \r\nwhich are joint field (2 or 3) value distributions. This is a quick way to both profile source \r\ndata, extract its schema, and discover anomalies that can be overlooked by other tools or \r\nmissed by manual Quality Control reviews. A comprehensive report is returned in a Python object \r\nthat can be used in further processing to make tables and charts such as in Jupyter Notebook.\r\n\r\n\r\n\r\nGoal\r\n+++++++++\r\n\r\nThe goal of analysis is to characterize data to know its structure, range of values, \r\nand presence of anomalies in relation to what it should be as defined by its architecture. \r\nIdeally, the documentation would describe the details of how it was collected, stored, and \r\nthe meaning of the data in the context of its intended use. In this ideal case, unit tests could be \r\nmade to automatically measure quality metrics both as the data is received and processed, and \r\nas it is distributed and used. Unfortunately, this ideal situation rarely exists and we are forced \r\nto manage data with uncertain quality, pedigree, and trustworthiness. When the use can tolerate \r\nimperfect data then this is not much of a problem. However, we now have increasingly stringent needs for \r\nbetter data to feed Artificial Intelligence (AI), Data Science (DataSci), and more sophisticated \r\nforecasting models in financial markets, global supply chain, consumer activity, and many others.\r\n\r\n\r\n\r\n\r\nObjectives\r\n++++++++++++++\r\n\r\nWe have learned from in-depth analysis and thorough reconstruction of data sets across many fields \r\nand types of data systems that there are several specific types of anomalies that frequenetly exist and \r\ngo undetected by even the most modern tools. Part of this expert assessment included following and measuring \r\nthe impact of the imperfect data on the end-use business activities for how outcome errors impacted \r\ndecision making, audit, compliance, analytics, forecast accuracy, etc. From this we created a combined \r\nhuman expert and big data Machine Learning (ML) technology to filter through all data to find several types \r\nof problems and reliable approaches to correcting them automatically. This led to the following key objectives \r\nof the VerityX (X denotes both the Python and DotNet libraries) analysis process:\r\n\r\n    * capture details of field datatype and format for all records and visibly showcase even infrequent variations.\r\n         - many tools limit the depth and breadth of records analyzed and variations captured due to processing, memory, and storage limitations.\r\n    * capture complete range of values for each field and emphasize low frequency instances since this is how anomalies can be quickly discovered.\r\n    * capture complete range of values for combinations of several fields as another quick way of detecting \r\n      anomalies and overall quality of fields that are linked by what they mean in end use cases.\r\n    * track number of field values parsed per record as key indicator of presence of extra delimiters and line feeds \r\n      that are not apparent during human review but which cause parsing code to break what should be a single record \r\n      into multiple partial records or generate more field values than there are fields. This is surprisingly common \r\n      in many mid and large size data systems. \r\n    * provide automated correction algorithms that repair all of the above problems with minimal required data \r\n      architecture and engineering which tends to be so complicated and labor intensive that it often lags \r\n      actual data cuasing serious 'technical debt'.\r\n    * provide multiple views and types of results into data quality and problems since real world data teams are typically \r\n      too constrained in time and personnel to probe every data set, system, and operation in detail.\r\n\r\n\r\n\r\n\r\n\r\nResults\r\n+++++++++\r\n\r\nResults are coordinated in a Python class 'QualityAnalysis' allowing concise handling \r\nof the setup parameters and the breadth and depth of discovered characteristics and \r\nknown/suspected errors. These results include:\r\n\r\n   * field unique values: per field unique values with count of instances.\r\n   * field datatype distributions: each field has counts for detected datatypes (int, real, bool, date, string, empty).\r\n   * field quality: each field is assigned a quality factor 0-100 based on discovered characteristics and knowledge-based algorithms.\r\n   * record size distribution: record sizes (byte lengths) to count of instances.\r\n   * record parsing errors: parsing errors (number fields after parsing relative to defined fields) \r\n     by small1 (1 too few fields), small2 (2 or more missing fields), big (1 or more too many fields). Also, has example records.\r\n   * record parsing distribution: number of parsed fields to count of instances.\r\n   * special character distribution: special characters and their count of instances, as well as example records.\r\n   * coValues: field combinations (2 or 3) unique value information. \r\n   * error statistics: values such as number records with any kind of error, number records \r\n     with datatype error, number records with format error and more\r\n\r\n\r\n\r\nNormalize & Enrich\r\n-------------------\r\n\r\n**VerityPy's** transforms allow Normalizing and Enriching source data with \r\na higher level of quality, accuracy, and meaning to support demanding use cases. There are five \r\nkinds of transforms (see transforms page for details):\r\n\r\n   1. Assignment: assigns values to field as a fixed value, reference to another field in record, random number, list of categories via frequencies, lookup dictionaries\r\n   2. Conditional: conditional tests of equality and inequality for numeric, string, and date values\r\n   3. Numeric: numeric calculation functions including using other fields in record by reference\r\n   4. Text: manipulate with slicing, adding, padding, replacing\r\n   5. Date: Change date format to ISO 8601 including from special Excel format \r\n\r\nThis is an example of a transform to populate an enrichment field 'useAGI' that denotes whether the record should be used \r\nin analytics based on the value of a numeric source field 'AGI'.\r\n\r\n   1. setToRef(\"AGI\")\r\n   2. ifEq(\"-1\")\r\n   3. setToValue(\"true\")\r\n   4. setToValue(\"false\")\r\n\r\nIn order to allow chaining of conditional functions, the flow is condition -> [false action] else [true action]. Thus, if step 2 above is False \r\nthen step 3 is done and the chain stops, whereas if step 2 is True then step 3 is skipped and step 4 is done (and any steps after it if they existed). \r\nThe net result is this simple transform fills an enrichment field with boolean value enabling easy filtering downstream in a spreadsheet, database, \r\nor analytics dashboard.\r\n\r\nA slightly more complicated logic flow that includes fixing formatting is the following transform that uses a source field 'y2_statefips' containing a 2 character \r\ncode to lookup the corresponding title in an external lookup dictionary and then assigns that to an enrichment field 'DestStateName' since the \r\noriginal source data only had the code making it non-intuitive for users to understand the data records. \r\n\r\n   1. setToRef(\"y2_statefips\")\r\n   2. setLength(\"2\",\"left\",\"0\")\r\n   3. lookup(\"StateName\")\r\n\r\nStep 1 gets the value of the field 'y2_statefips' from the current record. Step 2 fixes the string length to 2 characters with changes made \r\nto the left side of the string if it is too long (characters cut from left) or too short (characters added to left) with the character to \r\nadd set to be '0' (zero). This is critical for code lookups since a very common problem when data is moved among systems is for leading \r\nzeros to be removed thereby changing a code like '01' into '1' which would not be found in the lookup. This ensures that such an error \r\nis fixed prior to doing the lookup which occurs in step 3 to a dictionary name 'StateName' (loaded during the setup phase of the job).\r\n\r\n|\r\n|\r\n\r\nLicense\r\n===========\r\n\r\nThis is not open source software and cannot be included in an open source project as its license will break the open source license. \r\nHowever, there is a license allowing free use for non-commercial, personal applications. \r\nRead the license file for full details about allowed scope of free use. \r\nPaid licenses are required for commercial products either distributed or web hosted (e.g. SaaS), as well as enterprise applications with multiple users. \r\nThere are licenses for developers, royalty inclusion in other products, and support.\r\n",
    "bugtrack_url": null,
    "license": "License +++++++  copyright (c) 2024 Technik Interlytics (TM) all rights reserved  Last update: 20240521  Technik Interlytics provides its software development libraries VerityPy and VerityDotNet under a license model to meet development and distribution needs of several types of projects. All types of subscriptions, purchases, and use whether free or paid must adhere to the terms presented in this license agreement.  Terms and Conditions ====================  This License Agreement (\u201cAgreement\u201d) is by and between Technik Interlytics (\u201cTechnik\u201d) and the person or entity licensing the Technik Software (\u201cCompany\u201d). This Agreement consists of each Order and these Terms and Conditions (\u201cTerms\u201d).  Overview ==========  Technik is the owner and licensor of several software libraries and packages, as described by Technik on Technik\u2019s website and in Technik\u2019s marketing materials (the \u201cTechnik Software\u201d). The Technik Software consists of multiple products (each, a \u201cProduct\u201d), which may be licensed individually or as a suite of Products (the \u201cTechnikSuite\u201d), which are described in greater detail in an Order. Products and the TechnikSuite are referred to herein as Technik Software. Technik offers Company one or more options to use the Technik Software as documented by Technik subject to the restrictions stated in this Agreement. Technik may also provide support services (\u201cSupport and Updates\u201d) and other services, including consulting services regarding Company\u2019s use of the Technik Software (collectively with Support and Updates, the \u201cServices\u201d).  Technik makes the Technik Software and Services available to Company directly or through a third party (such as an authorized reseller) using a written online or offline order process (each, an \u201cOrder\u201d). Pursuant to the terms of this Agreement, Technik will provide to Company the Technik Software or Services described in each Order.  1. Technik Software and Services --------------------------------  During the term of this Agreement, Technik grants to Company and Company accepts a non-exclusive, nontransferable, revocable and limited license to use each item of Technik Software listed on an Order by Company for Company\u2019s own internal use (and not for resale in a materially unaltered form), subject to the terms, obligations, and restrictions set forth in the Order and in this Agreement (\u201cTechnik Software License\u201d).  Each Technik Software License and each Service is subject to permissions, limitations and responsibilities stated in the applicable Order (\u201cMetrics\u201d), which may state whether Company is permitted to:  * use the Technik Software for a limited trial only, for a stated time period, or perpetually; * evaluate the Technik Software or use it in a production environment; * use the Technik Software at no charge or for a stated fee; * permit access to the Technik Software to an unlimited number of authorized users or to a limited number of authorized users; * install and use the Technik Software in one computing device (\u201cMachine\u201d) or multiple Machines; * use a single copy or multiple copies of the Technik Software (\u201cCopies\u201d); * use the Technik Software solely for internal development use (\u201cInternal Use\u201d); * combine the Technik Software with Company\u2019s own software (\u201cCompany Software\u201d); * distribute to one or more of Company\u2019s customers (each, a \u201cCustomer\u201d); * install the Technik Software in Company\u2019s or Customer\u2019s systems (\u201cInstalled\u201d) or made available by Company on a website owned or controlled by Company as software as a service (\u201cSaaS\u201d); * use the Technik Software in one or more city (each, a \u201cLocation\u201d); * receive Support and Updates for a limited period of time or not at all; or * receive new versions of the Technik Software made available by Company to the general public (\u201cNew Versions\u201d) while Support and Updates is current and active, or receive New Versions only when Company pays additional fees.   Currently, Technik offers Technik Software Licenses in the following combination of Metrics (each license type expressly excludes any rights not specifically granted):  * Free Non-Commercial License. Grants the use of the Technik Software for private purposes only when the Company does not receive payment for its software or services using the software (Non-Commercial License\u201d). The Technik Software may be deployed to any number of web applications, intranet applications, or desktop software applications. This license type does not allow distribution of the Technik Software to third parties, clients of the Company, or SaaS project usage scenarios. * Free Trial License. Grants the use of the Technik Software for private non-production evaluation purposes only (\u201cTrial License\u201d). The default period for this license is thirty (30) days but the specific order may include a different binding period. The Technik Software may be deployed to any number of web applications, intranet applications, or desktop software applications. This license type does not allow distribution of the Technik Software to third parties, clients of the Company, or SaaS project usage scenarios. * Developer License Annual. Grants the use of either (i) a specific Technik Software Product or (ii) the TechnikSuite by a single software developer in an organization at a single Location for a period of one year. The Technik Software may be deployed to any number of web applications, intranet applications, or desktop software applications. This license type does not allow distribution of the Technik Software to third parties, clients of the Company, or SaaS project usage scenarios. * Developer License Perpetual. Grants the use of either (i) a specific Technik Software Product or (ii) the TechnikSuite by a single software developer in an organization at a single Location without an expiration. The Technik Software may be deployed within to any number of web applications, intranet applications, or desktop software applications. This license type does not allow distribution of the Technik Software to third parties, clients of the Company, or SaaS project usage scenarios. * Royalty-Free Redistribution Coverage. This add-on, which may be purchased under an Order, grants Company the right to distribute the Technik Software (without any duty to pay royalties) as part of a number of distinct packaged commercial or open-source products to third parties, in accordance with the number of projects covered in the base license (\u201cRoyalty-Free Redistribution Coverage\u201d). Grants the use of the Technik Software to deploy within SaaS software services, in accordance with the number of projects covered in the base license. * Company may subscribe separately to Support and Updates, which Technik may modify in it its sole discretion, on either a 1-year or multi-year basis. Then-current pricing for Support and Updates is set forth in an Order. * Unlimited Enterprise License. This license includes Royalty-Free Redistribution Coverage. The Unlimited Enterprise License is perpetual, subject to Company\u2019s compliance with the License Agreement (\u201cUnlimited Enterprise License\u201d). The Unlimited Enterprise License permits an unlimited number of developers in an organization in an unlimited number of Locations to deploy the TechnikSuite within an unlimited number of web applications, intranet applications, or desktop software applications. * Unlimited Monthly License. This license includes Royalty-free Redistribution Coverage. The Unlimited Monthly License permits the use on a subscription basis by an unlimited number of developers in an organization in an unlimited number of Locations to deploy the TechnikSuite within an unlimited number of web applications, intranet applications, or desktop software applications (\u201cUnlimited Monthly License\u201d). * All Licenses are non-transferable and sharing of licenses outside an organization or agency/client relationship is prohibited. Licenses expressly exclude any rights not specifically granted under the Agreement, and expressly exclude, without limitation, OEM redistribution, and using the Technik Software as SaaS, without additional purchase of Royalty-Free Distribution Coverage.  Regardless of the type of Technik Software License, Technik and Company expressly agree that Company will not directly compete with the functionality of any current Technik Software or Service.  * Company may not redistribute, republish, or otherwise make available the Technik Software or Services to any third party without the prior written consent of Technik as stated in an Order or this Agreement. * Company will monitor its own use of the Technik Software and report any use in excess of the Metrics and volume. Technik Software may monitor use to verify with Technik servers compliance with Metrics, volume, and the Agreement. Notwithstanding the foregoing, no Company or end-user data is ever transmitted to Technik servers. * Inclusion of Technik Software in a package must include this license in its entirety or a statement of this license and a link to its full text on the Technik website * Inclusion of Technik Software in a package must include a statement of Technik's copyright for the components of the package from Technik * Inclusion of Technik Software with open source software or made into a new work distributed as open source software will not amend the non-open source license of Technik Software which must be stated explicitly in the new work's license or notice.   2. Data and Content ---------------------  Company retains any and all rights in any information, content, and data provided by Company, including all rights in new versions and derivative works of Company\u2019s information, content, and data. Technik retains any and all rights in any information, content, and data provided by Technik, including all rights in new versions and derivative works of Technik\u2019s information, content, and data.  Technik may collect, store, use, aggregate, and share information about Company and its customers, including about how Company and its customers use the Technik Software and any Services. Details about how Technik currently processes Company data are stated in Technik\u2019s Privacy Statement, which Technik may modify from time to time to reflect Technik\u2019s current practices and in response to evolving laws and regulations around the world.  3. Financial ---------------  Company payment shall be as outlined in each Order. Unless otherwise stated in the Order, payment in full is due to be received by Technik on or prior to delivery of the Technik Software or Services. All fees and payments are nonrefundable. Technik reserves the right to increase fees for Unlimited Monthly Licenses, and Support and Updates, and will provide notice to Company of any increase within 30 days\u2019 of the effectiveness of such increase.  All of the prices hereunder are exclusive of any taxes. Company shall be responsible for any federal, state, or local sales, use, Canadian Goods and Services, property, VAT, or similar taxes that are or may be imposed on transactions. In addition to all of its other rights and remedies, if payment is not made by the due date, then Technik may suspend or terminate performance of any or all Services, and suspend or terminate the Technik Software License on all Technik Software.  Any amounts not paid by the due date will accrue late payment charges in the amount of one percent (1.0%) per month of the overdue amount. In addition, Technik will have the right to recover all costs of collection, including attorneys\u2019 fees and expenses, accrued or incurred by Technik in any court or other tribunal action to collect unpaid amounts due. Non-payment of recurring monthly subscription fees for more than thirty (30) days after the due date shall result in suspension or termination of Technik Software and Services.  4. Intellectual Property --------------------------  The Technik Software and the results of any Services are owned solely by Technik and are protected by copyright, trademark, and trade secret law, among other types of law. All rights, titles and interests to the Technik Software and Services reside in and shall remain with Technik. Company receives only the limited and non-exclusive use rights expressly stated in the Order and in this Agreement. Company may not sell, license, assign, pledge, or otherwise transfer the Technik Software or Services, or any copies thereof, to any third party, or permit or allow any third party to use the Technik Software or Services, except as expressly permitted by this Agreement. Any unauthorized sale, sublicense, assignment, pledge, other transfer, or impermissible use of the Technik Software or Services shall be void and will be a violation of Technik\u2019s rights in the Technik Software and Services.  Company acknowledges that without Technik\u2019s written permission in an Order: (i) Company may not duplicate the Technik Software or Services for resale, publication, transfer, conveyance, licensing or sublicensing, or redistribution to any third party; (ii) Company will neither duplicate the Technik Software or Services for redistribution to any third party; (iii) Company will not duplicate or otherwise dispose of the Technik Software of Services in any manner violative of the United States Copyright Act (Title 17, U.S. Code); (iv) the Technik Software and Services may be used only by the aggregate number of authorized users at any given time; (v) the Technik Software and Services may be installed, accessed and used only on as many processors as there are authorized users; (vi) if the Company uses networked equipment, Company will never allow more than the maximum number of authorized users to use the Technik Software or Services at any given time; and (vii) Company may not modify, disassemble, decode, or decompile the Technik Software or Services, in whole or in part.  Regardless as to the number of authorized users or uses, Company may make one (1) copy of the Technik Software as an archival or backup copy, which shall bear Technik\u2019s copyright notice and other proprietary markings. Company shall make no other copies of the Technik Software for any purpose.  Company shall submit to Technik for approval, prior to use, distribution, or disclosure, any advertising, promotion, or publicity that refers to Technik or that uses the trade names, trademarks, or service marks of Technik (\u201cTechnik Marks\u201d). Technik shall have the right to require at its discretion the correction or deletion of any incorrect or misleading material regarding Technik or the Technik Marks in any advertising, promotion, or publicity. Technik may use the name and trademark of Company (\u201cCompany Marks\u201d) to list or highlight Company as a licensee of Technik. The use of each Mark, and the goodwill generated thereby, shall inure to the benefit of the owner of the Mark.  5. Confidentiality --------------------  For purposes of this Agreement, \u201cConfidential Information\u201d shall mean non-public information and data received by either party (\u201creceiving party\u201d) from the other party (\u201cdisclosing party\u201d).  Notwithstanding the foregoing, information and data shall not be deemed to be Confidential Information if such information or data: (i) was known by the receiving party at the time of such disclosure; (ii) was known to the general public at the time of such disclosure or becomes known to the general public (other than by act of the receiving party) subsequent to such disclosure; (iii) is disclosed lawfully to the receiving party by a third party without restriction; (iv) is developed independently by the receiving party without reference to the disclosing party\u2019s Confidential Information; or (v) is approved in writing by the disclosing party for disclosure by the receiving party.  The receiving party may disclose the disclosing party\u2019s Confidential Information when the receiving party is required by law (e.g., by subpoena), provided however the receiving party will give the disclosing party prior written notice of such required disclosure (unless notifying the disclosing party is prohibited by law), and will work with the disclosing party to limit the required disclosure or have the requirement revoked (e.g., a court revokes a subpoena for the Confidential Information).  The receiving hereunder shall not disclose to any third-party firm, corporation, individual, or other entity, any Confidential Information which it receives from the disclosing party. For clarity, Technik may disclose the Company\u2019s Confidential Information to Technik\u2019s employees and independent contractors in connection with providing the Technik Software and performing the Services hereunder. The receiving party shall use the same degree of care in safeguarding the Confidential Information as the receiving party uses for its own confidential and proprietary information, but in no event less than a standard of reasonable care.  6. Term and Termination --------------------------  For each item of Technik Software and each Service, the initial term will be stated in the Order, and may renew for additional periods stated as Renewal Terms on the Order. If no Initial Term is specified on the Order, then the Initial Term will be one year from the Effective Date. If no Renewal Term is specified on the Order, then each Renewal Term will be one year beginning with the expiration of the preceding term, but only if Technik receives any renewal payment at least thirty (30) days prior to the first day of the applicable Renewal Term.  This Agreement may be terminated by either party for material breach by the other party provided that the terminating party gives thirty (30) days prior written notice specifying the breach, and the breaching party fails to cure or correct the breach within the thirty-day notice period. In the event of any such termination, Company shall pay Technik for all Technik Software provided and for all Services performed by Technik and for all  work-in-progress up to the date of termination.  Company may terminate an Unlimited Monthly License upon thirty (30) days written notice prior to the start of the next month of the subscription term. Upon Technik\u2019s receipt of a termination notice by Company, the Unlimited Monthly License subscription shall terminate either when the maximum API calls permitted under the Order have been made or the end of the month in which such termination was effective.  This Agreement shall be deemed to be automatically terminated upon any material breach of Company\u2019s obligations under Sections 1 (Technik Software and Services), 3 (Financial), 4 (Intellectual Property), or 5 (Confidentiality).  Upon termination of this Agreement for any reason, Company shall immediately return to Technik any Technik Software, related materials, and all copies thereof or, with Technik\u2019s prior written permission, Company shall destroy all such materials and certify in writing as to their destruction.  Sections: Financial, Intellectual Property, Confidentiality, Term and Termination, and Representations and Warranties, Disclaimers and Limitations of Liability, Indemnification, and General Provisions, of this Agreement shall survive any expiration or termination of this Agreement.  Company recognizes that Technik has expended considerable amounts of time, effort, and money to develop the Technik Software and Services, and that Company\u2019s unauthorized copying, use, transfer, or disclosure of the Technik Software or Services, or their contents, may cause Technik to sustain substantial, irreparable harm and damage. Similarly, each party has expended considerable amounts of time, effort, and money to develop and protect their respective Confidential Information. In addition to all other legal and equitable remedies available to a party, each party may seek from an arbitrator (pursuant to the arbitration provisions of this Agreement) temporary and permanent injunctive relief to remedy any breach of the other party\u2019s obligations under Sections Intellectual Property, or Confidentiality of this Agreement.  7. Representations and Warranties ----------------------------------  The parties hereby covenant, represent, and warrant to each other that they are duly authorized and empowered to enter into this Agreement, and that this Agreement constitutes a valid and binding, enforceable Agreement. Company further represents covenants and warrants that this Agreement is not inconsistent with any Agreement or obligation binding Company or its property or assets.  8. Disclaimers and Limitations of Liability --------------------------------------------  THE Technik SOFTWARE AND SERVICES ARE PROVIDED \u201cAS IS\u201d AND \u201cAS AVAILABLE\u201d WITH ALL FAULTS. Technik SHALL NOT BE LIABLE FOR ANY DAMAGES, LOSSES, OR EXPENSES, OF ANY TYPE, KIND OR DESCRIPTION, INCURRED OR SUSTAINED BY COMPANY OR BY ANY OTHER PERSON OR ENTITY FOR ANY REASON.  THE WARRANTIES STATED IN THIS AGREEMENT ARE EXCLUSIVE AND IN LIEU OF ANY OTHER WARRANTIES, EXPRESS, IMPLIED, OR STATUTORY, INCLUDING WITHOUT LIMITATION ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FOR FITNESS FOR A PARTICULAR PURPOSE.  NEITHER Technik NOR ITS AFFILIATES, NOR ANY OF THEIR RESPECTIVE THIRD PARTY LICENSORS MAKE ANY WARRANTIES, EXPRESS OR IMPLIED, AS TO RESULTS TO BE OBTAINED FROM THE USE OF ANY Technik SOFTWARE OR SERVICES OR THAT THE Technik SOFTWARE OR SERVICES WILL BE ERROR-FREE.  IN NO EVENT SHALL Technik NOR ITS AFFILIATES, NOR ANY OF THEIR RESPECTIVE THIRD-PARTY LICENSORS HAVE ANY LIABILITY FOR LOST PROFITS OR FOR INDIRECT, SPECIAL, PUNITIVE OR CONSEQUENTIAL DAMAGES, OR ANY LIABILITY TO ANY PERSON ARISING OUT OF THIS AGREEMENT, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES OR LIABILITY.  IN NO EVENT SHALL THE CUMULATIVE LIABILITY OF Technik, ITS AFFILIATES, AND THEIR RESPECTIVE THIRDPARTY LICENSORS HEREUNDER EXCEED THE GREATER OF (I) THE AMOUNTS RECEIVED BY Technik FROM COMPANY DURING THE TWELVE (12) MONTHS PRIOR TO THE MAKING OF THE CLAIM, AND (II) $100, WHICH SHALL BE COMPANY\u2019S EXCLUSIVE REMEDY AGAINST Technik, ITS AFFILIATES AND THEIR RESPECTIVE THIRDPARTY LICENSORS UNDER THIS AGREEMENT.  All disclaimers herein shall not be applicable to liabilities that cannot be waived under the applicable laws or regulations of the United States (including US State and Federal law) or any other applicable country or jurisdiction.  9.  Indemnification ---------------------  Should any portion of the Technik Software become, or in Technik\u2019s opinion be likely to become the subject of a claim of infringement, Technik may, as Company\u2019s sole and exclusive remedy, elect to (1) obtain for Company at Technik\u2019s expense the right to use such portion, or (2) replace or modify the Technik Software so that it becomes non-infringing, or (3) remove the infringing portion and grant to Company a credit prorated to reflect the portion of the overall fees paid by Company attributable to such portion of the Technik Software. The foregoing states the entire liability of Technik with respect to allegation or claim of infringement of any intellectual property rights.  Company will defend, indemnify, and hold Technik harmless at Company\u2019s expense for any claim that alleges that the Company Software, Company\u2019s data, or Company\u2019s materials infringes upon a copyright, patent or other right, and Company will pay all damages and costs awarded by a court in connection with such claim. In the event Company redistributes, republishes, or otherwise permits use of the Technik Software or Services, as provided by Technik or as modified by Company, in violation of this Agreement, Company shall indemnify, defend, and hold Technik harmless from any loss, damage or claim arising therefrom. Each indemnification claim requires that Technik: (i) gives prompt written notice of the claim to the Company; (ii) gives Company sole control of the defense or settlement of the claim; and (iii) provides to Company all necessary information, assistance, and authority to defend. Company shall promptly defend or settle the claim at Company\u2019s sole cost and with Company\u2019s counsel, but Company shall not have the right to admit liability on behalf of Technik, or assess any blame, responsibility, costs, or fees to Technik.  10. Entire Agreement. ---------------------  The parties acknowledge and agree that this Agreement (including each Order) embodies the complete and exclusive understanding and Agreement of the parties with respect to the Technik Software and Services, and supersedes any prior or contemporaneous proposal, agreement, or license whether oral or written, and any other communication between the parties.  Changes to this Agreement. This Agreement shall not be modified or amended except by a written instrument, signed by both parties. The version of this Agreement posted to Technik\u2019s website is the current set of Terms that Technik may modify from time to time. For each Order placed by Company, the current version of this Agreement will apply to that Order, unless the parties have otherwise signed a hard copy or digital version of these Terms independent of the web-based Terms, in which case the signed version of the Terms will control.  11. Export Control. -------------------  Company will comply with all export control laws and regulations of the United States and all other countries and jurisdictions. Company will not remove or export from the United States or allow the export or re-export of any part of the Technik Software or any direct product thereof: (a) into (or to a national or resident of) any embargoed or terrorist-supporting country; (b) to anyone on the U.S. Commerce Department's Table of Denial Orders or U.S. Treasury Department's list of Specially Designated Nationals (or any list that replaces such list); (c) to any country to which such export or re-export is restricted or prohibited, or as to which the United States government or any agency thereof requires an export license or other governmental approval at the time of export or re-export without first obtaining such license or approval; or (d) otherwise in violation of any export or import restrictions, laws or regulations of any United States or foreign agency or authority that has jurisdiction.  Company agrees to the foregoing and warrants that it is not located in, under the control of, or a national or resident of any such prohibited country or on any such prohibited party list. The Technik Software is further restricted from being used for terrorist activity, or for the design or development of nuclear, chemical, or biological weapons or missile technology without the prior permission of the U.S. government.  12. Governing Law. ------------------  This Agreement shall be governed by and construed in accordance with the laws of the State of Texas and the federal laws of the United States, without reference to conflicts of laws provisions and policies. Any dispute between the parties shall be determined by one arbitrator in binding arbitration administered by the American Arbitration Association (adr.org) if all parties are based in the United States; otherwise, the arbitration shall be administered by the International Centre for Dispute Resolution (icdr.org). The arbitrator shall decide all questions of arbitrability, and shall be empowered to issue preliminary, temporary, and injunctive relief by issuing an initial award. All hearings will be conducted by audio conference or video conference. The language of the arbitration shall be English. The arbitrator\u2019s awards will be binding on the parties and may be entered and enforced in any courts or tribunals of competent jurisdiction.  13. Severability. --------------------  If any provision of this Agreement shall be determined to be void or unenforceable by a court of competent jurisdiction, the remaining provisions shall remain in full force and effect.  14. Waiver. ------------  Any waiver of a breach or of performance of a term or condition of this Agreement shall be in writing, signed by both parties hereto. The failure of either party to insist on strict adherence to any term of this Agreement on any occasion shall not be deemed a waiver of its right thereafter to insist on such adherence.  15. Headings. -------------  Headings are for convenience only and are not part of this Agreement. They shall not be used to modify or construe the terms of the sections they introduce.  16. Notice. ------------  All notices required to be given by this Agreement shall be sent by certified mail, return receipt requested, or by courier, to the party to be notified at the address set forth in the Order, or to such other or new address as to which notice has been given. All notices made under this Agreement shall be effective twenty calendar days after mailing.  17. Successors and Assigns. -----------------------------  This Agreement is not assignable by either party without the other party\u2019s written permission and is binding upon the permitted successors of each party to this Agreement. However, either party may assign no less than all of its rights under this Agreement to an affiliate or successor as a result of a merger, acquisition, or reorganization, and the assignment will be effective on receipt of written notice by the nonassigning party.  18. Independent Contractor. ----------------------------  The relationship of the parties is that of independent contractors. This Agreement does not create any actual or apparent agency, partnership, or relationship of employer and employee between the parties hereto.  19. Force Majeure. -------------------  Except for a party\u2019s payment obligations, neither party shall have any liability for any defaults or delays resulting from circumstances reasonably beyond its reasonable control.  20. Company Documents. ----------------------  The provisions and terms of any document issued by Company in conjunction with this Agreement shall be of no effect and shall not in any way extend, affect, or amend the terms and conditions set forth in this Agreement (including any Order) unless expressly accepted in writing by Technik.  **Technik Interlytics**  https://technikinterlytics.com  support@technikinterlytics.com ",
    "summary": "VerityPy is a library by Technik Interlytics for Better Data = Better Decisions. It is a Data Inspector, Remediator, Normalizer on large structured data sets. It combines curated human expert knowledge gained from in-depth forensic analysis and remediating data in many fields, with Machine Learning pattern extraction results to know where to look, what to look for, and what to do about problems found lurking deep within huge data sets.",
    "version": "2024.1.21",
    "project_urls": {
        "Documentation": "https://technikinterlytics.com/VerityPy/",
        "Homepage": "https://technikinterlytics.com"
    },
    "split_keywords": [
        "data quality",
        " data profiling",
        " data enrichment",
        " data normalization",
        " data remediation",
        " human data expert",
        " data intelligence",
        " embedded delimiter",
        " data cleaning",
        " data veracity",
        " data prep",
        " data wrangling",
        " data forensics",
        " data remediation",
        " data science",
        " data blending",
        " structured data",
        " metadata",
        " broken records",
        " parsing error"
    ],
    "urls": [
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "9adb226309f790e67cd6020605cf3965fecb2bb676eb0b18edb375b2c4714803",
                "md5": "9fb8f8d65048a82433f85bf82acecec7",
                "sha256": "a640985d21cc41c1d91f9194cd28319c3e1e01e0ddbe8143adc3784c1abcb27f"
            },
            "downloads": -1,
            "filename": "VerityPy-2024.1.21-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "9fb8f8d65048a82433f85bf82acecec7",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": ">=3.8",
            "size": 93170,
            "upload_time": "2024-08-12T16:46:32",
            "upload_time_iso_8601": "2024-08-12T16:46:32.445569Z",
            "url": "https://files.pythonhosted.org/packages/9a/db/226309f790e67cd6020605cf3965fecb2bb676eb0b18edb375b2c4714803/VerityPy-2024.1.21-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": "",
            "digests": {
                "blake2b_256": "02ecbc570af4a69fbe43e979c9b2ca1f6b7e108dce49b7557fab8a15eb118677",
                "md5": "e097f8451f96e857f936644664388786",
                "sha256": "5eedb8fb05e4773670eb2c1efcd8a1bbb90f7c1a1dfe49619de1f3f6fabf5970"
            },
            "downloads": -1,
            "filename": "veritypy-2024.1.21.tar.gz",
            "has_sig": false,
            "md5_digest": "e097f8451f96e857f936644664388786",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": ">=3.8",
            "size": 101064,
            "upload_time": "2024-08-12T16:46:34",
            "upload_time_iso_8601": "2024-08-12T16:46:34.857940Z",
            "url": "https://files.pythonhosted.org/packages/02/ec/bc570af4a69fbe43e979c9b2ca1f6b7e108dce49b7557fab8a15eb118677/veritypy-2024.1.21.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2024-08-12 16:46:34",
    "github": false,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "lcname": "veritypy"
}
None
Elapsed time: 4.74096s