# Machine Stats for Unix-like systems
[](https://pypi.org/project/machine-stats/)
A simple and effective way to gather machine statistics (RAM, Storage, CPU)
from a server environment as a first layer of a [Tidal discovery process](https://guides.tidal.cloud/).
Machine Stats for Linux/Unix leverages [Ansible](https://www.ansible.com/) to
gather facts in a cross-platform way.
## Interactive tutorial
Get familiar with Machine Stats, Tidal Tools and Tidal Accelerator!
[](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2Ftidalmigrations%2Fmachine-stats-workshop&cloudshell_image=gcr.io%2Ftidal-1529434400027%2Fmachine-stats-workshop&cloudshell_tutorial=machine-stats.md&shellonly=true)
## Installation
Install locally in a Python 3 environment:
```sh
python3 -m pip install machine-stats
```
_Need to install in an environment without internet access?_ [Checkout how to do that below](#offline-installation).
<details>
<summary>Distribution-specific installation instructions</summary>
### Ubuntu 16.04
1. Make sure `pip` is installed and is one of the latest version available:
```sh
sudo apt update && \
sudo apt install -y python3-pip && \
python3 -m pip install --user pip==18.1 && \
python3 -m pip install --upgrade --user pip
```
**Note:** Direct upgrade to the latest available `pip` version results with an unusable `pip` installation. That's why we perform the upgrade through the intermediate version (`18.1`).
2. Install `machine-stats`:
```sh
python3 -m pip install machine-stats
```
### Debian 9/Ubuntu 18.04
1. Make sure `pip` is installed and is one of the latest version:
```sh
sudo apt update && \
sudo apt install -y python3-pip && \
python3 -m pip install --upgrade pip
```
2. Install `machine-stats`:
```sh
python3 -m pip install machine-stats
```
### Debian 10/Debian 11/Ubuntu 20.04/Ubuntu 21.04
1. Make sure `pip` is installed:
```sh
sudo apt update && \
sudo apt install -y python3-pip
```
2. Install `machine-stats`:
```sh
python3 -m pip install machine-stats
```
### CentOS 7/CentOS 8/CentOS Stream/Red Hat Enterprise Linux 7/Red Hat Enterprise Linux 8/Rocky Linux 8
1. Install Python 3:
```sh
sudo yum install -y python3
```
2. Upgrade `pip` to the latest available version:
```sh
python3 -m pip install --upgrade --user pip
```
3. Install `machine-stats`:
```sh
python3 -m pip install machine-stats
```
### SUSE Linux Enterprise Server 12
1. Install Python 3.6:
```sh
sudo zypper install -y python36-base
```
2. Install `machine-stats`:
```sh
pip install machine-stats
```
### SUSE Linux Enterprise Server 15
1. Install `pip`:
```sh
sudo zypper install -y python3-pip
```
2. Install `machine-stats`:
```sh
pip install machine-stats
```
</details>
## Data captured
For Linux/Unix based systems, by default, the following metrics are captured
from the resources and sent and stored in Tidal Accelerator:
- Host Name
- FQDN
- IP Addresses
- RAM Allocated (GB)
- RAM Used (GB)
- Storage Allocated (GB)
- Storage Used (GB)
- CPU Count
- Operating System
- Operating System Version
- CPU name
You also can optionally capture metrics about processes running on the server:
- User
- Process Name
- Process Path
- Memory Used (MB)
- Max Memory Used (MB)
- Total Alive Time
To enable capturing process metrics add the command-line flag `--process-stats`:
```sh
machine-stats --process-stats
```
## Minimal example
1. Create a `hosts` file in the current directory. See [below](#Generating-a-hosts-file-from-Tidal-Migrations) on a couple ways
you can easily create this.
2. Add connection strings in the form of `ssh-user@ip-address` or
`ssh-user@domain` to the `hosts` file one per line If the `ssh-user@` part
is omitted, then the current user name is used.
3. If you need to use a custom SSH identity file for some particular host,
provide it as the following:
```sh
my-user@example.com ansible_ssh_private_key_file=path/to/key-file.pem
```
4. Make sure that Python 2.6+ is installed on the machines from `hosts` file.
5. If `python` executable was installed into non-default location (**not** in
`/usr/bin/python`), add the `ansible_python_interpreter` parameter to the
`hosts` file after the host IP/domain, for example:
```text
freebsd.example.com ansible_python_interpreter=/usr/local/bin/python
```
6. Execute `machine-stats` and pipe its output to Tidal Tools:
```sh
machine-stats | tidal sync servers
```
### Additional notes
By default Machine Stats looks for the `hosts` file in current working
directory. If your inventory file has another name or is located on another
path, then you should specify it explicitly:
```sh
machine-stats /path/to/myhosts | tidal sync servers
```
You can specify multiple inventory files as the following:
```sh
machine-stats hosts myhosts /path/to/myhosts
```
### Configuration
Machine Stats uses Ansible under the hood. Most of the [Ansible configuration
options](https://docs.ansible.com/ansible/2.9/reference_appendices/config.html#common-options)
can be used with Machine Stats too. By default, Machine Stats will look for
configuration files in the following locations:
- `$PWD/machine_stats.cfg`
- `$PWD/machine-stats.cfg`
- `$PWD/machinestats.cfg`
- `$PWD/ansible.cfg`
- `$HOME/.machine_stats.cfg`
- `$HOME/.machine-stats.cfg`
- `$HOME/.machinestats.cfg`
- `$HOME/.ansible.cfg`
- `/etc/ansible/ansible.cfg`
Also, it is possible to specify the custom configuration file location by
setting the `ANSIBLE_CONFIG` environment variable, for example:
```sh
ANSIBLE_CONFIG=/path/to/my/machine_stats.cfg machine_stats /path/to/my/hosts
```
**Note:** if `ANSIBLE_CONFIG` value points to a directory, then Machine Stats
will look for `ansible.cfg` in that directory.
### Getting information about RHEL 5 hosts
Red Hat Enterprise Linux 5 is shipped with Python 2.4 but `machine_stats`
requires at least Python 2.6. To install Python 2.6 on your RHEL 5 machine
follow these steps. **NOTE:** this doesn't update the existing Python packages,
but installs Python 2.6 alongside with system Python packages.
1. Download Python 2.6 package and its dependencies from EPEL repository:
```console
sudo curl -L -OOO -k \
http://download.fedoraproject.org/pub/archive/epel/5/x86_64/{python26-libs-2.6.8-2.el5.x86_64.rpm,libffi-3.0.5-1.el5.x86_64.rpm,python26-2.6.8-2.el5.x86_64.rpm}
```
2. Install the packages:
```console
sudo rpm -ivh python26*.rpm libffi*.rpm
```
3. Use non-standard Python location in your `hosts` file:
```text
my-user@rhel5.example.com ansible_python_interpreter=/usr/bin/python2.6
```
### Offline installation
**NOTE:** Creating the packages archive for offline installation and the actual
offline installation process must be performed on machines with the same OS and
Python versions.
1. On the machine with internet connection create the packages archive using
the following commands:
```sh
python3 -m pip download -d machine-stats-offline machine-stats
tar czf machine-stats-offline.tar.gz machine-stats-offline
```
2. Transfer the archive to the machine where you need to perform the offline
installation (replace `<remote-host>` and `<remote-dir>` with the
appropriate values):
```sh
scp machine-stats-offline.tar.gz <remote-host>:/<remote-dir>/
```
3. On the remote host, extract the archive and switch to extracted directory:
```sh
tar xf machine-stats-offline.tar.gz
cd machine-stats-offline
```
4. Install Machine Stats and its dependencies:
```sh
python3 -m pip install --no-index --find-links . machine_stats-*.whl
```
## Generating a `hosts` file from Tidal Accelerator
You can easily generate a hosts file directly from your server inventory in
Tidal Accelerator. For example you can use this command:
```sh
tidal export servers | jq '.[].host_name' > hosts
```
This will create a file (`hosts`), in your current directory, that you can
use above in Step 1.
Alternatively, if you use Tidal Accelerator [Ansible Tower integration
script](https://github.com/tidalmigrations/ansible-tower-integration) you can
use its output to generate the `hosts` file for `machine_stats`.
### Requirements
- [`jq`](https://stedolan.github.io/jq/)
### Usage
```sh
cd ansible-tower-integration
./tidal_inventory.py | jq -r '.servers.hosts[]' > path/to/hosts
```
## Troubleshooting
### machine-stats: command not found
If running Machine Stats as a CLI failed, try running it as the following:
```sh
python3 -m machine_stats
```
### How to permanently enable the Python 3.8 software collection on RHEL 7
You should always enable the Python software collection before using `pipenv`
with the following command:
```sh
scl enable rh-python38 bash
```
To permanently add Python 3 to your `$PATH`, you can add an `scl_source`
command to the “dot files” for your specific user. The benefit of this approach
is that the collection is already enabled at every login.
Using your preferred text editor, add the following line to your `~/.bashrc`:
```sh
# Add RHSCL Python 3 to my login environment
source scl_source enable rh-python38
```
Raw data
{
"_id": null,
"home_page": "https://github.com/tidalmigrations/machine_stats",
"name": "machine-stats",
"maintainer": "",
"docs_url": null,
"requires_python": "",
"maintainer_email": "",
"keywords": "",
"author": "Tidal SW",
"author_email": "support@tidalcloud.com",
"download_url": "https://files.pythonhosted.org/packages/4d/9b/272a4a240ef42a825e6b8d77798a5cd5e9ef28715ef26d55706998c79b07/machine_stats-2.6.0.tar.gz",
"platform": null,
"description": "# Machine Stats for Unix-like systems\n\n[](https://pypi.org/project/machine-stats/)\n\nA simple and effective way to gather machine statistics (RAM, Storage, CPU)\nfrom a server environment as a first layer of a [Tidal discovery process](https://guides.tidal.cloud/).\n\nMachine Stats for Linux/Unix leverages [Ansible](https://www.ansible.com/) to\ngather facts in a cross-platform way.\n\n## Interactive tutorial\n\nGet familiar with Machine Stats, Tidal Tools and Tidal Accelerator!\n\n[](https://ssh.cloud.google.com/cloudshell/editor?cloudshell_git_repo=https%3A%2F%2Fgithub.com%2Ftidalmigrations%2Fmachine-stats-workshop&cloudshell_image=gcr.io%2Ftidal-1529434400027%2Fmachine-stats-workshop&cloudshell_tutorial=machine-stats.md&shellonly=true)\n\n## Installation\n\nInstall locally in a Python 3 environment:\n\n```sh\npython3 -m pip install machine-stats\n```\n\n_Need to install in an environment without internet access?_ [Checkout how to do that below](#offline-installation).\n\n<details>\n <summary>Distribution-specific installation instructions</summary>\n\n### Ubuntu 16.04\n\n1. Make sure `pip` is installed and is one of the latest version available:\n ```sh\n sudo apt update && \\\n sudo apt install -y python3-pip && \\\n python3 -m pip install --user pip==18.1 && \\\n python3 -m pip install --upgrade --user pip\n ```\n **Note:** Direct upgrade to the latest available `pip` version results with an unusable `pip` installation. That's why we perform the upgrade through the intermediate version (`18.1`).\n2. Install `machine-stats`:\n ```sh\n python3 -m pip install machine-stats\n ```\n\n### Debian 9/Ubuntu 18.04\n\n1. Make sure `pip` is installed and is one of the latest version:\n ```sh\n sudo apt update && \\\n sudo apt install -y python3-pip && \\\n python3 -m pip install --upgrade pip\n ```\n2. Install `machine-stats`:\n ```sh\n python3 -m pip install machine-stats\n ```\n\n### Debian 10/Debian 11/Ubuntu 20.04/Ubuntu 21.04\n\n1. Make sure `pip` is installed:\n ```sh\n sudo apt update && \\\n sudo apt install -y python3-pip\n ```\n2. Install `machine-stats`:\n ```sh\n python3 -m pip install machine-stats\n ```\n\n### CentOS 7/CentOS 8/CentOS Stream/Red Hat Enterprise Linux 7/Red Hat Enterprise Linux 8/Rocky Linux 8\n\n1. Install Python 3:\n ```sh\n sudo yum install -y python3\n ```\n2. Upgrade `pip` to the latest available version:\n ```sh\n python3 -m pip install --upgrade --user pip\n ```\n3. Install `machine-stats`:\n ```sh\n python3 -m pip install machine-stats\n ```\n\n### SUSE Linux Enterprise Server 12\n\n1. Install Python 3.6:\n ```sh\n sudo zypper install -y python36-base\n ```\n2. Install `machine-stats`:\n ```sh\n pip install machine-stats\n ```\n\n### SUSE Linux Enterprise Server 15\n\n1. Install `pip`:\n ```sh\n sudo zypper install -y python3-pip\n ```\n2. Install `machine-stats`:\n ```sh\n pip install machine-stats\n ``` \n</details>\n\n## Data captured\n\nFor Linux/Unix based systems, by default, the following metrics are captured\nfrom the resources and sent and stored in Tidal Accelerator:\n\n- Host Name\n- FQDN\n- IP Addresses\n- RAM Allocated (GB)\n- RAM Used (GB)\n- Storage Allocated (GB)\n- Storage Used (GB)\n- CPU Count\n- Operating System\n- Operating System Version\n- CPU name\n\nYou also can optionally capture metrics about processes running on the server:\n\n- User\n- Process Name\n- Process Path\n- Memory Used (MB)\n- Max Memory Used (MB)\n- Total Alive Time\n\nTo enable capturing process metrics add the command-line flag `--process-stats`:\n```sh\nmachine-stats --process-stats\n```\n\n## Minimal example\n\n1. Create a `hosts` file in the current directory. See [below](#Generating-a-hosts-file-from-Tidal-Migrations) on a couple ways\n you can easily create this.\n\n2. Add connection strings in the form of `ssh-user@ip-address` or\n `ssh-user@domain` to the `hosts` file one per line If the `ssh-user@` part\n is omitted, then the current user name is used.\n3. If you need to use a custom SSH identity file for some particular host,\n provide it as the following:\n\n ```sh\n my-user@example.com ansible_ssh_private_key_file=path/to/key-file.pem\n ```\n\n4. Make sure that Python 2.6+ is installed on the machines from `hosts` file.\n5. If `python` executable was installed into non-default location (**not** in\n `/usr/bin/python`), add the `ansible_python_interpreter` parameter to the\n `hosts` file after the host IP/domain, for example:\n\n ```text\n freebsd.example.com ansible_python_interpreter=/usr/local/bin/python\n ```\n\n6. Execute `machine-stats` and pipe its output to Tidal Tools:\n\n ```sh\n machine-stats | tidal sync servers\n ```\n\n### Additional notes\n\nBy default Machine Stats looks for the `hosts` file in current working\ndirectory. If your inventory file has another name or is located on another\npath, then you should specify it explicitly:\n\n```sh\nmachine-stats /path/to/myhosts | tidal sync servers\n```\n\nYou can specify multiple inventory files as the following:\n\n```sh\nmachine-stats hosts myhosts /path/to/myhosts\n```\n\n### Configuration\n\nMachine Stats uses Ansible under the hood. Most of the [Ansible configuration\noptions](https://docs.ansible.com/ansible/2.9/reference_appendices/config.html#common-options)\ncan be used with Machine Stats too. By default, Machine Stats will look for\nconfiguration files in the following locations:\n\n- `$PWD/machine_stats.cfg`\n- `$PWD/machine-stats.cfg`\n- `$PWD/machinestats.cfg`\n- `$PWD/ansible.cfg`\n- `$HOME/.machine_stats.cfg`\n- `$HOME/.machine-stats.cfg`\n- `$HOME/.machinestats.cfg`\n- `$HOME/.ansible.cfg`\n- `/etc/ansible/ansible.cfg`\n\nAlso, it is possible to specify the custom configuration file location by\nsetting the `ANSIBLE_CONFIG` environment variable, for example:\n\n```sh\nANSIBLE_CONFIG=/path/to/my/machine_stats.cfg machine_stats /path/to/my/hosts\n```\n\n**Note:** if `ANSIBLE_CONFIG` value points to a directory, then Machine Stats\nwill look for `ansible.cfg` in that directory.\n\n### Getting information about RHEL 5 hosts\n\nRed Hat Enterprise Linux 5 is shipped with Python 2.4 but `machine_stats`\nrequires at least Python 2.6. To install Python 2.6 on your RHEL 5 machine\nfollow these steps. **NOTE:** this doesn't update the existing Python packages,\nbut installs Python 2.6 alongside with system Python packages.\n\n1. Download Python 2.6 package and its dependencies from EPEL repository:\n\n ```console\n sudo curl -L -OOO -k \\\n http://download.fedoraproject.org/pub/archive/epel/5/x86_64/{python26-libs-2.6.8-2.el5.x86_64.rpm,libffi-3.0.5-1.el5.x86_64.rpm,python26-2.6.8-2.el5.x86_64.rpm}\n ```\n\n2. Install the packages:\n\n ```console\n sudo rpm -ivh python26*.rpm libffi*.rpm\n ```\n\n3. Use non-standard Python location in your `hosts` file:\n\n ```text\n my-user@rhel5.example.com ansible_python_interpreter=/usr/bin/python2.6\n ```\n\n### Offline installation\n\n**NOTE:** Creating the packages archive for offline installation and the actual\noffline installation process must be performed on machines with the same OS and\nPython versions.\n\n1. On the machine with internet connection create the packages archive using\n the following commands:\n\n ```sh\n python3 -m pip download -d machine-stats-offline machine-stats\n tar czf machine-stats-offline.tar.gz machine-stats-offline\n ```\n\n2. Transfer the archive to the machine where you need to perform the offline\n installation (replace `<remote-host>` and `<remote-dir>` with the\n appropriate values):\n\n ```sh\n scp machine-stats-offline.tar.gz <remote-host>:/<remote-dir>/\n ```\n\n3. On the remote host, extract the archive and switch to extracted directory:\n\n ```sh\n tar xf machine-stats-offline.tar.gz\n cd machine-stats-offline\n ```\n\n4. Install Machine Stats and its dependencies:\n\n ```sh\n python3 -m pip install --no-index --find-links . machine_stats-*.whl\n ```\n\n## Generating a `hosts` file from Tidal Accelerator\n\nYou can easily generate a hosts file directly from your server inventory in\nTidal Accelerator. For example you can use this command:\n\n```sh\ntidal export servers | jq '.[].host_name' > hosts\n```\n\nThis will create a file (`hosts`), in your current directory, that you can\nuse above in Step 1.\n\nAlternatively, if you use Tidal Accelerator [Ansible Tower integration\nscript](https://github.com/tidalmigrations/ansible-tower-integration) you can\nuse its output to generate the `hosts` file for `machine_stats`.\n\n### Requirements\n\n- [`jq`](https://stedolan.github.io/jq/)\n\n### Usage\n\n```sh\ncd ansible-tower-integration\n./tidal_inventory.py | jq -r '.servers.hosts[]' > path/to/hosts\n```\n\n## Troubleshooting\n\n### machine-stats: command not found\n\nIf running Machine Stats as a CLI failed, try running it as the following:\n\n```sh\npython3 -m machine_stats\n```\n\n### How to permanently enable the Python 3.8 software collection on RHEL 7\n\nYou should always enable the Python software collection before using `pipenv`\nwith the following command:\n\n```sh\nscl enable rh-python38 bash\n```\n\nTo permanently add Python 3 to your `$PATH`, you can add an `scl_source`\ncommand to the \u201cdot files\u201d for your specific user. The benefit of this approach\nis that the collection is already enabled at every login.\n\nUsing your preferred text editor, add the following line to your `~/.bashrc`:\n\n```sh\n# Add RHSCL Python 3 to my login environment\nsource scl_source enable rh-python38\n```\n",
"bugtrack_url": null,
"license": "",
"summary": "A simple and effective way to gather machine statistics (RAM, Storage, CPU, etc.) from server environment",
"version": "2.6.0",
"project_urls": {
"Homepage": "https://github.com/tidalmigrations/machine_stats"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "26752ef92f8ee575a501f50584eb63fc6df01a0e7347f7c3817c2267702e2ef7",
"md5": "eb0fd2ef1e219438e2be22927657209d",
"sha256": "ffcad1e1657a631c79d05ea35ba4ba0a144fedfbf8894343809596a969856d26"
},
"downloads": -1,
"filename": "machine_stats-2.6.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "eb0fd2ef1e219438e2be22927657209d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": null,
"size": 15224,
"upload_time": "2024-01-12T02:20:19",
"upload_time_iso_8601": "2024-01-12T02:20:19.696021Z",
"url": "https://files.pythonhosted.org/packages/26/75/2ef92f8ee575a501f50584eb63fc6df01a0e7347f7c3817c2267702e2ef7/machine_stats-2.6.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "4d9b272a4a240ef42a825e6b8d77798a5cd5e9ef28715ef26d55706998c79b07",
"md5": "767d017e54fff22f815513c7a0b4c1aa",
"sha256": "cf0c77f188ce75d9d43370f7647d233432e9f399f18e4b26068eed7cdf0bb942"
},
"downloads": -1,
"filename": "machine_stats-2.6.0.tar.gz",
"has_sig": false,
"md5_digest": "767d017e54fff22f815513c7a0b4c1aa",
"packagetype": "sdist",
"python_version": "source",
"requires_python": null,
"size": 16455,
"upload_time": "2024-01-12T02:20:21",
"upload_time_iso_8601": "2024-01-12T02:20:21.767471Z",
"url": "https://files.pythonhosted.org/packages/4d/9b/272a4a240ef42a825e6b8d77798a5cd5e9ef28715ef26d55706998c79b07/machine_stats-2.6.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-01-12 02:20:21",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "tidalmigrations",
"github_project": "machine_stats",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "machine-stats"
}
JSON
