# GrimoireLab
[](https://chaoss.biterg.io/app/kibana#/dashboard/Overview)
GrimoireLab is a [CHAOSS](https://chaoss.community) toolset for software development analytics. It includes a coordinated set of tools
to retrieve data from systems used to support software development (repositories), store it in databases,
enrich it by computing relevant metrics, and make it easy to run analytics and visualizations on it.
You can learn more about GrimoireLab in the [GrimoireLab tutorial](https://chaoss.github.io/grimoirelab-tutorial/),
or visit the [GrimoireLab website](https://chaoss.github.io/grimoirelab).
Metrics available in GrimoireLab are, in part, developed in the CHAOSS project. For more information regarding CHAOSS metrics, see the latest release at: https://chaoss.community/metrics/
# Getting started
To ease the newcomer experience we are providing a [default setup](default-grimoirelab-settings)
to analyze git activity for this repository. For this set up, there are several options to run GrimoireLab:
## Using `docker-compose`
Requirements:
* **Software**: [git](https://git-scm.com/), [docker client](https://docs.docker.com/get-docker/) and [docker compose](https://docs.docker.com/compose/install/). An example of working configuration:
```console
root@test-68b8628f:~# git --version
git version 2.17.1
root@test-68b8628f:~# docker --version
Docker version 19.03.1, build 74b1e89
root@test-68b8628f:~# docker-compose --version
docker-compose version 1.22.0, build f46880fe
```
* **Hardware**: 2 CPU cores, 8GB memory RAM and [enough virtual memory for Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html)
Steps:
1. Clone this project:
```console
foo@bar:~$ git clone https://github.com/chaoss/grimoirelab
```
2. Go to `docker-compose` folder and run the following command:
```console
foo@bar:~$ cd grimoirelab/docker-compose
foo@bar:~/grimoirelab/docker-compose$ docker-compose up -d
```
Your dashboard will be ready after a while at `http://localhost:8000`. The waiting time depends on the amount of data to fetch from a repo, for small repositories you can expect your data to be visible in the dashboard after 10-15 minutes.
More details in the [docker-compose folder](./docker-compose/README.md).
## Using `docker run`
Requirements:
* **Software**: [git](https://git-scm.com/) and [docker client](https://docs.docker.com/get-docker/). An example of working configuration:
```console
root@test-68b8628f:~# git --version
git version 2.17.1
root@test-68b8628f:~# docker --version
Docker version 19.03.1, build 74b1e89
```
* **Hardware**: 2 CPU cores, 8GB memory RAM and set
* ElasticSearch and Kibana up and running.
* SortingHat up and running
Steps:
1. Clone this project:
```console
$ git clone https://github.com/chaoss/grimoirelab
```
2. Go to the project folder and run the following command:
```console
foo@bar:~$ cd grimoirelab
foo@bar:~/grimoirelab $ docker run --net=host \
-v $(pwd)/default-grimoirelab-settings/projects.json:/home/grimoire/conf/projects.json \
-v $(pwd)/default-grimoirelab-settings/setup-docker.cfg:/home/grimoire/conf/setup.cfg \
-t grimoirelab/grimoirelab
```
Your dashboard will be ready after a while at `http://localhost:8000`. The waiting time depends on the amount of data to fetch from a repo, for small repositories you can expect your data to be visible in the dashboard after 10-15 minutes.
More details in the [docker folder](./docker/README.md).
# GrimoireLab components
Currently, GrimoireLab toolkit is organized in the following repositories:
* Data retrieval related components:
* [Perceval](https://github.com/chaoss/grimoirelab-perceval): retrieval of data from data sources
* [Perceval (bundle for OPNFV)](https://github.com/chaoss/grimoirelab-perceval-opnfv)
* [Perceval (bundle for Mozilla)](https://github.com/chaoss/grimoirelab-perceval-mozilla)
* [Perceval (bundle for Puppet)](https://github.com/chaoss/grimoirelab-perceval-puppet)
* [Perceval (bundle for Weblate)](https://github.com/chaoss/grimoirelab-perceval-weblate)
* [Graal](https://github.com/chaoss/grimoirelab-graal): source data analysis with external tools
* [KingArthur](https://github.com/chaoss/grimoirelab-kingarthur): batch processing for massive retrieval
* Data enrichment related components:
* [GrimoireElk](https://github.com/chaoss/grimoirelab-elk): storage and enrichment of data
* [Cereslib](https://github.com/chaoss/grimoirelab-cereslib): generic data processor
* [SortingHat](https://github.com/chaoss/grimoirelab-sortinghat): identity management
* Data consumption related components:
* [Kibiter](https://github.com/chaoss/grimoirelab-kibiter): dashboard, downstream version of Kibana
* [Sigils](https://github.com/chaoss/grimoirelab-sigils): visualizations and dashboards
* [Kidash](https://github.com/chaoss/grimoirelab-kidash): visualizations and dashboards manager
* [Manuscripts](https://github.com/chaoss/grimoirelab-manuscripts): reporting
* Platform management, orchestration, and common utils:
* [Mordred](https://github.com/chaoss/grimoirelab-mordred): orchestration
* [GrimoireLab Toolkit](https://github.com/chaoss/grimoirelab-toolkit): common utilities
* [Bestiary](https://github.com/chaoss/grimoirelab-bestiary): web-based user interface to manage repositories and projects for Mordred
There are also some [components built by the GrimoreLab community](community_components.md),
which can be useful for you. Other related repositories are:
* [GrimoireLab Tutorial](https://github.com/chaoss/grimoirelab-tutorial)
* [GrimoireLab as a whole](https://github.com/chaoss/grimoirelab) (this repository)
## Contents of this repository
This repository is for content relevant to GrimoireLab as a whole. For example:
* Issues for new features or bug reports that affect more than one GrimoireLab module. In this case, let's open an issue here, and when implementing the fix or the feature, let´s comment about the specific tickets in the specific modules that are used. For example, when supporting a new datasource, we will need patches (at least) in `Perceval`, `GrimoireELK` and panels. In this case, we would open a feature request (or the user story) for the whole case, an issue (and later a pull request) in `Perceval` for the data retriever, same for `GrimoireELK` for the enriching code, and same for `panels` for the Kibiter panels.
* Release notes for most GrimoireLab components (directory [releases](releases)).
* Docker container for showcasing GrimoireLab (directory [docker](docker)).
Includes a Dockerfile and configuration files for the GrimoireLab containers
that can be used to demo the technology, and can be the basis for real
deployments. See more information in the [docker README.md file](docker/README.md).
* If you feel more comfortable with `docker-compose`, the [docker-compose](docker-compose)
folder includes instructions and configuration files to deploy GrimoireLab using
`docker-compose` command.
* Source code of the GrimoireLab components is available in `src`. Each directory is a
Git submodule, so its contents will not be available after cloning the repository. To
fetch all the data, and get the latest version, you can run the following command:
```console
$ git submodule update --init --remote
```
* How releases of GrimoireLab are built and tested: [Building](BUILDING.md)
## Citation
If you use GrimoireLab in your research papers, please refer to [GrimoireLab: A toolset for software development analytics](https://doi.org/10.7717/peerj-cs.601):
APA style:
```
Dueñas S, Cosentino V, Gonzalez-Barahona JM, del Castillo San Felix A, Izquierdo-Cortazar D, Cañas-Díaz L, Pérez García-Plaza A. 2021. GrimoireLab: A toolset for software development analytics. PeerJ Computer Science 7:e601 https://doi.org/10.7717/peerj-cs.601
```
BibTeX / BibLaTeX:
```
@Article{duenas2021:grimoirelab,
author = {Dueñas, Santiago and Cosentino, Valerio and Gonzalez-Barahona, Jesus M. and del Castillo San Felix, Alvaro and Izquierdo-Cortazar, Daniel and Cañas-Díaz, Luis and Pérez García-Plaza, Alberto},
title = {GrimoireLab: A toolset for software development analytics},
journaltitle = {PeerJ Computer Science},
date = {2021-07-09},
volume = 7,
number = {e601},
doi = {10.7717/peerj-cs.601},
url = {https://doi.org/10.7717/peerj-cs.601}}
```
# Contributing
Contributions are welcome, please check the [Contributing Guidelines](CONTRIBUTING.md).
Raw data
{
"_id": null,
"home_page": "https://chaoss.github.io/grimoirelab/",
"name": "grimoirelab",
"maintainer": null,
"docs_url": null,
"requires_python": "<4.0,>=3.8",
"maintainer_email": null,
"keywords": "development, grimoirelab",
"author": "GrimoireLab Developers",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/ed/46/4430d8ae3c38c00d02edd5c649b5488b85fb829a3f89e500bc0df1135a8d/grimoirelab-1.0.0.tar.gz",
"platform": null,
"description": "# GrimoireLab\n\n[![grimoirelab-showcase](https://user-images.githubusercontent.com/25265451/84442403-30dcce80-ac5b-11ea-9f5b-60266d875ebd.png \"GrimoireLab | CHAOSS Bitergia Analytics\")](https://chaoss.biterg.io/app/kibana#/dashboard/Overview)\n\nGrimoireLab is a [CHAOSS](https://chaoss.community) toolset for software development analytics. It includes a coordinated set of tools\nto retrieve data from systems used to support software development (repositories), store it in databases,\nenrich it by computing relevant metrics, and make it easy to run analytics and visualizations on it.\n\nYou can learn more about GrimoireLab in the [GrimoireLab tutorial](https://chaoss.github.io/grimoirelab-tutorial/),\nor visit the [GrimoireLab website](https://chaoss.github.io/grimoirelab).\n\nMetrics available in GrimoireLab are, in part, developed in the CHAOSS project. For more information regarding CHAOSS metrics, see the latest release at: https://chaoss.community/metrics/\n\n# Getting started\n\nTo ease the newcomer experience we are providing a [default setup](default-grimoirelab-settings)\nto analyze git activity for this repository. For this set up, there are several options to run GrimoireLab:\n\n## Using `docker-compose`\n\nRequirements:\n* **Software**: [git](https://git-scm.com/), [docker client](https://docs.docker.com/get-docker/) and [docker compose](https://docs.docker.com/compose/install/). An example of working configuration:\n```console\nroot@test-68b8628f:~# git --version\ngit version 2.17.1\nroot@test-68b8628f:~# docker --version\nDocker version 19.03.1, build 74b1e89\nroot@test-68b8628f:~# docker-compose --version\ndocker-compose version 1.22.0, build f46880fe\n```\n* **Hardware**: 2 CPU cores, 8GB memory RAM and [enough virtual memory for Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/vm-max-map-count.html) \n\nSteps:\n1. Clone this project:\n```console\nfoo@bar:~$ git clone https://github.com/chaoss/grimoirelab\n```\n2. Go to `docker-compose` folder and run the following command:\n```console\nfoo@bar:~$ cd grimoirelab/docker-compose\nfoo@bar:~/grimoirelab/docker-compose$ docker-compose up -d\n```\n\nYour dashboard will be ready after a while at `http://localhost:8000`. The waiting time depends on the amount of data to fetch from a repo, for small repositories you can expect your data to be visible in the dashboard after 10-15 minutes.\n\nMore details in the [docker-compose folder](./docker-compose/README.md).\n\n## Using `docker run`\n\nRequirements: \n* **Software**: [git](https://git-scm.com/) and [docker client](https://docs.docker.com/get-docker/). An example of working configuration:\n```console\nroot@test-68b8628f:~# git --version\ngit version 2.17.1\nroot@test-68b8628f:~# docker --version\nDocker version 19.03.1, build 74b1e89\n```\n* **Hardware**: 2 CPU cores, 8GB memory RAM and set\n* ElasticSearch and Kibana up and running.\n* SortingHat up and running\n\nSteps:\n1. Clone this project:\n```console\n$ git clone https://github.com/chaoss/grimoirelab\n```\n2. Go to the project folder and run the following command:\n```console\nfoo@bar:~$ cd grimoirelab\nfoo@bar:~/grimoirelab $ docker run --net=host \\ \n -v $(pwd)/default-grimoirelab-settings/projects.json:/home/grimoire/conf/projects.json \\\n -v $(pwd)/default-grimoirelab-settings/setup-docker.cfg:/home/grimoire/conf/setup.cfg \\\n -t grimoirelab/grimoirelab\n```\n\nYour dashboard will be ready after a while at `http://localhost:8000`. The waiting time depends on the amount of data to fetch from a repo, for small repositories you can expect your data to be visible in the dashboard after 10-15 minutes.\n\nMore details in the [docker folder](./docker/README.md).\n\n# GrimoireLab components\n\nCurrently, GrimoireLab toolkit is organized in the following repositories:\n\n* Data retrieval related components:\n * [Perceval](https://github.com/chaoss/grimoirelab-perceval): retrieval of data from data sources\n * [Perceval (bundle for OPNFV)](https://github.com/chaoss/grimoirelab-perceval-opnfv)\n * [Perceval (bundle for Mozilla)](https://github.com/chaoss/grimoirelab-perceval-mozilla)\n * [Perceval (bundle for Puppet)](https://github.com/chaoss/grimoirelab-perceval-puppet)\n * [Perceval (bundle for Weblate)](https://github.com/chaoss/grimoirelab-perceval-weblate)\n * [Graal](https://github.com/chaoss/grimoirelab-graal): source data analysis with external tools\n * [KingArthur](https://github.com/chaoss/grimoirelab-kingarthur): batch processing for massive retrieval\n* Data enrichment related components:\n * [GrimoireElk](https://github.com/chaoss/grimoirelab-elk): storage and enrichment of data\n * [Cereslib](https://github.com/chaoss/grimoirelab-cereslib): generic data processor\n * [SortingHat](https://github.com/chaoss/grimoirelab-sortinghat): identity management\n* Data consumption related components:\n * [Kibiter](https://github.com/chaoss/grimoirelab-kibiter): dashboard, downstream version of Kibana\n * [Sigils](https://github.com/chaoss/grimoirelab-sigils): visualizations and dashboards\n * [Kidash](https://github.com/chaoss/grimoirelab-kidash): visualizations and dashboards manager\n * [Manuscripts](https://github.com/chaoss/grimoirelab-manuscripts): reporting\n* Platform management, orchestration, and common utils:\n * [Mordred](https://github.com/chaoss/grimoirelab-mordred): orchestration\n * [GrimoireLab Toolkit](https://github.com/chaoss/grimoirelab-toolkit): common utilities\n * [Bestiary](https://github.com/chaoss/grimoirelab-bestiary): web-based user interface to manage repositories and projects for Mordred\n\nThere are also some [components built by the GrimoreLab community](community_components.md),\nwhich can be useful for you. Other related repositories are:\n* [GrimoireLab Tutorial](https://github.com/chaoss/grimoirelab-tutorial)\n* [GrimoireLab as a whole](https://github.com/chaoss/grimoirelab) (this repository)\n\n## Contents of this repository\n\nThis repository is for content relevant to GrimoireLab as a whole. For example:\n\n* Issues for new features or bug reports that affect more than one GrimoireLab module. In this case, let's open an issue here, and when implementing the fix or the feature, let\u00b4s comment about the specific tickets in the specific modules that are used. For example, when supporting a new datasource, we will need patches (at least) in `Perceval`, `GrimoireELK` and panels. In this case, we would open a feature request (or the user story) for the whole case, an issue (and later a pull request) in `Perceval` for the data retriever, same for `GrimoireELK` for the enriching code, and same for `panels` for the Kibiter panels.\n\n* Release notes for most GrimoireLab components (directory [releases](releases)).\n\n* Docker container for showcasing GrimoireLab (directory [docker](docker)).\nIncludes a Dockerfile and configuration files for the GrimoireLab containers\nthat can be used to demo the technology, and can be the basis for real\ndeployments. See more information in the [docker README.md file](docker/README.md).\n\n* If you feel more comfortable with `docker-compose`, the [docker-compose](docker-compose)\nfolder includes instructions and configuration files to deploy GrimoireLab using\n`docker-compose` command.\n\n* Source code of the GrimoireLab components is available in `src`. Each directory is a\nGit submodule, so its contents will not be available after cloning the repository. To\nfetch all the data, and get the latest version, you can run the following command:\n```console\n$ git submodule update --init --remote\n```\n\n* How releases of GrimoireLab are built and tested: [Building](BUILDING.md)\n\n## Citation\n\nIf you use GrimoireLab in your research papers, please refer to [GrimoireLab: A toolset for software development analytics](https://doi.org/10.7717/peerj-cs.601):\n\nAPA style:\n\n```\nDue\u00f1as S, Cosentino V, Gonzalez-Barahona JM, del Castillo San Felix A, Izquierdo-Cortazar D, Ca\u00f1as-D\u00edaz L, P\u00e9rez Garc\u00eda-Plaza A. 2021. GrimoireLab: A toolset for software development analytics. PeerJ Computer Science 7:e601 https://doi.org/10.7717/peerj-cs.601\n```\n\nBibTeX / BibLaTeX:\n\n```\n@Article{duenas2021:grimoirelab,\n author = \t {Due\u00f1as, Santiago and Cosentino, Valerio and Gonzalez-Barahona, Jesus M. and del Castillo San Felix, Alvaro and Izquierdo-Cortazar, Daniel and Ca\u00f1as-D\u00edaz, Luis and P\u00e9rez Garc\u00eda-Plaza, Alberto},\n title = \t {GrimoireLab: A toolset for software development analytics},\n journaltitle = {PeerJ Computer Science},\n date = \t {2021-07-09},\n volume = \t 7,\n number = \t {e601},\n doi = \t {10.7717/peerj-cs.601},\n url = \t {https://doi.org/10.7717/peerj-cs.601}}\n```\n\n# Contributing\n\nContributions are welcome, please check the [Contributing Guidelines](CONTRIBUTING.md).\n",
"bugtrack_url": null,
"license": "GPL-3.0+",
"summary": "Tool set for software development analytics",
"version": "1.0.0",
"project_urls": {
"Bug Tracker": "https://github.com/chaoss/grimoirelab/issues",
"Homepage": "https://chaoss.github.io/grimoirelab/",
"Repository": "https://github.com/chaoss/grimoirelab"
},
"split_keywords": [
"development",
" grimoirelab"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "642cc625b18ff5f4f35d4d4c84ab96cc9878cdbb824970d599b7640cbf4c1479",
"md5": "7691e94476b70c2cd521f70a38c954c4",
"sha256": "2acdaf9b434b46925e9ec3d8144ebc9b64d0dcafdde5772e1ee9104fa1e7119c"
},
"downloads": -1,
"filename": "grimoirelab-1.0.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "7691e94476b70c2cd521f70a38c954c4",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "<4.0,>=3.8",
"size": 19327,
"upload_time": "2024-04-13T08:27:38",
"upload_time_iso_8601": "2024-04-13T08:27:38.507928Z",
"url": "https://files.pythonhosted.org/packages/64/2c/c625b18ff5f4f35d4d4c84ab96cc9878cdbb824970d599b7640cbf4c1479/grimoirelab-1.0.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "ed464430d8ae3c38c00d02edd5c649b5488b85fb829a3f89e500bc0df1135a8d",
"md5": "59e75d3e5e869e44fc224c92761b866b",
"sha256": "37a28bf4d22344cc55894e49be6e2875521f01d596c595fb9a7db129ccd85ba4"
},
"downloads": -1,
"filename": "grimoirelab-1.0.0.tar.gz",
"has_sig": false,
"md5_digest": "59e75d3e5e869e44fc224c92761b866b",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "<4.0,>=3.8",
"size": 17133,
"upload_time": "2024-04-13T08:27:40",
"upload_time_iso_8601": "2024-04-13T08:27:40.723076Z",
"url": "https://files.pythonhosted.org/packages/ed/46/4430d8ae3c38c00d02edd5c649b5488b85fb829a3f89e500bc0df1135a8d/grimoirelab-1.0.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-13 08:27:40",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "chaoss",
"github_project": "grimoirelab",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"requirements": [],
"lcname": "grimoirelab"
}
JSON
