<div align="center">
<h1>CodeSurvey</h1>
<a href="https://pypi.org/project/codesurvey">
<img alt="PyPI" src="https://img.shields.io/pypi/v/codesurvey">
</a>
<p>
<a href="https://github.com/lean-python-org/codesurvey">GitHub</a> - <a href="https://lean-python-org.github.io/codesurvey">Documentation</a>
</p>
</div>
CodeSurvey is a framework and tool to survey code repositories for
language feature usage, library usage, and more:
* Survey a specific set of repositories, or randomly sample
repositories from services like GitHub
* Built-in support for analyzing Python code; extensible to support
any language
* Write simple Python functions to define the code features you want
to survey; record arbitrary details of feature occurrences
* Supports parallelizization of repository downloading and analysis
across multiple processes
* Logging and progress tracking to monitor your survey as it runs
* Inspect the results as Python objects, or in an sqlite database
## Installation
```
pip install codesurvey
```
## Usage
The `CodeSurvey` class can easily be configured to run a survey, such
as to measure how often the `math` module is used in a random set of
recently updated Python repositories from GitHub:
```python
from codesurvey import CodeSurvey
from codesurvey.sources import GithubSampleSource
from codesurvey.analyzers.python import PythonAstAnalyzer
from codesurvey.analyzers.python.features import py_module_feature_finder
# Define a FeatureFinder to look for the `math` module in Python code
has_math = py_module_feature_finder('math', modules=['math'])
# Configure the survey
survey = CodeSurvey(
db_filepath='math_survey.sqlite3',
sources=[
GithubSampleSource(language='python'),
],
analyzers=[
PythonAstAnalyzer(
feature_finders=[
has_math,
],
),
],
max_workers=5,
)
# Run the survey on 10 repositories
survey.run(max_repos=10)
# Report on the results
repo_features = survey.get_repo_features(feature_names=['math'])
repo_count_with_math = sum([
1 for repo_feature in repo_features if
repo_feature.occurrence_count > 0
])
print(f'{repo_count_with_math} out of {len(repo_features)} repos use math')
```
* For more Sources of repositories, see [Source
docs](https://lean-python-org.github.io/codesurvey/sources/core)
* For more Analyzers and FeatureFinders, see [Analyzer
docs](https://lean-python-org.github.io/codesurvey/analyzers/core)
* For more options and methods for inspecting results, see
[`CodeSurvey` docs](https://lean-python-org.github.io/codesurvey/core)
* For details on directly inspecting the sqlite database of survey
results see [Database docs](https://lean-python-org.github.io/codesurvey/database)
* More examples can be found in
[examples](https://github.com/lean-python-org/codesurvey/tree/main/examples)
## Contributing
* Install Poetry dependencies with `make deps`
* Documentation:
* Run local server: `make docs-serve`
* Build docs: `make docs-build`
* Deploy docs to GitHub Pages: `make docs-github`
* Docstring style follows the [Google style guide](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings)
## TODO
* Add unit tests
Raw data
{
"_id": null,
"home_page": "https://github.com/lean-python-org/codesurvey",
"name": "codesurvey",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Ben Denham",
"author_email": "ben@denham.nz",
"download_url": "https://files.pythonhosted.org/packages/3a/5d/9cc8ecde2873cfd1d7ce65c918683163f032f32ac19e8b538d6a43f548c7/codesurvey-0.1.5.tar.gz",
"platform": null,
"description": "<div align=\"center\">\n\n<h1>CodeSurvey</h1>\n\n<a href=\"https://pypi.org/project/codesurvey\">\n <img alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/codesurvey\">\n</a>\n\n<p>\n <a href=\"https://github.com/lean-python-org/codesurvey\">GitHub</a> - <a href=\"https://lean-python-org.github.io/codesurvey\">Documentation</a>\n</p>\n\n</div>\n\nCodeSurvey is a framework and tool to survey code repositories for\nlanguage feature usage, library usage, and more:\n\n* Survey a specific set of repositories, or randomly sample\n repositories from services like GitHub\n* Built-in support for analyzing Python code; extensible to support\n any language\n* Write simple Python functions to define the code features you want\n to survey; record arbitrary details of feature occurrences\n* Supports parallelizization of repository downloading and analysis\n across multiple processes\n* Logging and progress tracking to monitor your survey as it runs\n* Inspect the results as Python objects, or in an sqlite database\n\n\n## Installation\n\n```\npip install codesurvey\n```\n\n\n## Usage\n\nThe `CodeSurvey` class can easily be configured to run a survey, such\nas to measure how often the `math` module is used in a random set of\nrecently updated Python repositories from GitHub:\n\n```python\nfrom codesurvey import CodeSurvey\nfrom codesurvey.sources import GithubSampleSource\nfrom codesurvey.analyzers.python import PythonAstAnalyzer\nfrom codesurvey.analyzers.python.features import py_module_feature_finder\n\n# Define a FeatureFinder to look for the `math` module in Python code\nhas_math = py_module_feature_finder('math', modules=['math'])\n\n# Configure the survey\nsurvey = CodeSurvey(\n db_filepath='math_survey.sqlite3',\n sources=[\n GithubSampleSource(language='python'),\n ],\n analyzers=[\n PythonAstAnalyzer(\n feature_finders=[\n has_math,\n ],\n ),\n ],\n max_workers=5,\n)\n\n# Run the survey on 10 repositories\nsurvey.run(max_repos=10)\n\n# Report on the results\nrepo_features = survey.get_repo_features(feature_names=['math'])\nrepo_count_with_math = sum([\n 1 for repo_feature in repo_features if\n repo_feature.occurrence_count > 0\n])\nprint(f'{repo_count_with_math} out of {len(repo_features)} repos use math')\n```\n\n\n\n* For more Sources of repositories, see [Source\n docs](https://lean-python-org.github.io/codesurvey/sources/core)\n* For more Analyzers and FeatureFinders, see [Analyzer\n docs](https://lean-python-org.github.io/codesurvey/analyzers/core)\n* For more options and methods for inspecting results, see\n [`CodeSurvey` docs](https://lean-python-org.github.io/codesurvey/core)\n* For details on directly inspecting the sqlite database of survey\n results see [Database docs](https://lean-python-org.github.io/codesurvey/database)\n* More examples can be found in\n [examples](https://github.com/lean-python-org/codesurvey/tree/main/examples)\n\n\n## Contributing\n\n* Install Poetry dependencies with `make deps`\n* Documentation:\n * Run local server: `make docs-serve`\n * Build docs: `make docs-build`\n * Deploy docs to GitHub Pages: `make docs-github`\n * Docstring style follows the [Google style guide](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings)\n\n\n## TODO\n\n* Add unit tests\n",
"bugtrack_url": null,
"license": "GPL-3.0-only",
"summary": "Analyse source code repositories for language feature and library usage.",
"version": "0.1.5",
"project_urls": {
"Homepage": "https://github.com/lean-python-org/codesurvey",
"Repository": "https://github.com/lean-python-org/codesurvey"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "1fb1887de9870659c81e0c0c09283858d213bf780b422d96addd4683f5532244",
"md5": "3314333c85604561559693560f46231b",
"sha256": "01d7bcce489ee07fe298a0a0bc18a0f3ad90b518e94ef5053711a387d3f5094d"
},
"downloads": -1,
"filename": "codesurvey-0.1.5-py3-none-any.whl",
"has_sig": false,
"md5_digest": "3314333c85604561559693560f46231b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 40129,
"upload_time": "2024-08-10T22:46:22",
"upload_time_iso_8601": "2024-08-10T22:46:22.123250Z",
"url": "https://files.pythonhosted.org/packages/1f/b1/887de9870659c81e0c0c09283858d213bf780b422d96addd4683f5532244/codesurvey-0.1.5-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3a5d9cc8ecde2873cfd1d7ce65c918683163f032f32ac19e8b538d6a43f548c7",
"md5": "380ee7a8d9641537dc95f5cea70dfa2e",
"sha256": "de307090684361f11c48541c13aafbac4a02039a568741966e1a893c8e5e6aa3"
},
"downloads": -1,
"filename": "codesurvey-0.1.5.tar.gz",
"has_sig": false,
"md5_digest": "380ee7a8d9641537dc95f5cea70dfa2e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8",
"size": 37119,
"upload_time": "2024-08-10T22:46:24",
"upload_time_iso_8601": "2024-08-10T22:46:24.371500Z",
"url": "https://files.pythonhosted.org/packages/3a/5d/9cc8ecde2873cfd1d7ce65c918683163f032f32ac19e8b538d6a43f548c7/codesurvey-0.1.5.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-08-10 22:46:24",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "lean-python-org",
"github_project": "codesurvey",
"travis_ci": false,
"coveralls": false,
"github_actions": false,
"lcname": "codesurvey"
}
JSON
