# ML-Logger, A Simple and Scalable Logging Utility With a Beautiful Visualization Dashboard That Is Super Fast
[](http://pepy.tech/project/ml-logger)
<img alt="fast" src="figures/ml-dash-v3.gif" align="right" width="50%"/>
ML-Logger makes it easy to:
- save data locally and remotely, as **binary**, in a transparent `pickle` file, with the same API and zero
configuration.
- write from 500+ worker containers to a single instrumentation server
- visualize `matplotlib.pyplot` figures from a remote server locally with `logger.savefig('my_figure.png')`
And ml-logger does all of these with *minimal configuration* — you can use the same logging
code both locally and remotely with no code change.
ML-logger is highly performant -- the remote writes are asynchronous. For this reason it doesn't slow down your training
even with 100+ metric keys.
Why did we built this, you might ask? Because we want to make it easy for people in ML to
use the same logging code in all of they projects, so that it is easy to get started with
someone else's baseline.
## Getting Started!
To **install** `ml_logger`, do:
```bash
pip install ml-logger
```
> The landscape of python modules is a lot messier than that of javascript. The most up-to-date `graphene` requires the following versions:
>
> ```
> yes | pip install graphene==2.1.3
> yes | pip install graphql-core==2.1
> yes | pip install graphql-relay==0.4.5
> yes | pip install graphql-server-core==1.1.1
> ```
Add the following to your `~/.zshrc`
```bash
export ML_LOGGER_ROOT=<your-logging-server-address>
export ML_LOGGER_USER=<your-handle> # make sure it is the same on the cluster.
```
Now you can rock!
```python
from ml_logger import logger
print(logger)
# ~> logging data to /tmp/ml-logger-debug
```
Log key/value pairs, and metrics:
```python
for i in range(1):
logger.log(metrics={'some_val/smooth': 10, 'status': f"step ({i})"}, reward=20, timestep=i)
### flush the data, otherwise the value would be overwritten with new values in the next iteration.
logger.flush()
# outputs ~>
# ╒════════════════════╤════════════════════════════╕
# │ reward │ 20 │
# ├────────────────────┼────────────────────────────┤
...
```
## Simple Logging Example
In your project files, do:
```python
from params_proto import ParamsProto, Proto, Flag
class Args(ParamsProto):
""" CLI Args for the program
Try:
python3 example.py --help
And it should print out the help strings
"""
seed = Proto(1, help="random seed")
D_lr = 5e-4
G_lr = 1e-4
Q_lr = 1e-4
T_lr = 1e-4
plot_interval = 10
verbose = Flag("the verbose flag")
if __name__ == '__main__':
import scipy
import numpy as np
from ml_logger import logger, LOGGER_USER
# Put in your ~/.bashrc
# export ML_LOGGER_ROOT = "http://<your-logging-server>:8081"
# export ML_LOGGER_USER = $USER
logger.configure(prefix=f"{LOGGER_USER}/scratch/your-experiment-prefix!")
logger.log_params(Args=vars(Args))
logger.upload_file(__file__)
for epoch in range(10):
logger.log(step=epoch, D_loss=0.2, G_loss=0.1, mutual_information=0.01)
logger.log_key_value(epoch, 'some string key', 0.0012)
# when the step index updates, logger flushes all of the key-value pairs to file system/logging server
logger.flush()
# Images
face = scipy.misc.face()
face_bw = scipy.misc.face(gray=True)
logger.save_image(face, "figures/face_rgb.png")
logger.save_image(face_bw, "figures/face_bw.png")
image_bw = np.zeros((64, 64, 1))
image_bw_2 = scipy.misc.face(gray=True)[::4, ::4]
logger.save_video([face] * 5, "videos/face.mp4")
```
And it should print out
```bash
✓ created a new logging client
Dashboard: http://app.dash.ml/user-name/your-experiment-prefix!
Log_directory: http://<your-server-ip>:8080
══════════════════════════════════════════
Args
────────────────────┬─────────────────────
seed │ 1
D_lr │ 0.0005
G_lr │ 0.0001
Q_lr │ 0.0001
T_lr │ 0.0001
plot_interval │ 10
════════════════════╧═════════════════════
╒════════════════════╤════════════════════╕
│ step │ 9 │
├────────────────────┼────────────────────┤
│ D loss │ 0.200 │
├────────────────────┼────────────────────┤
│ G loss │ 0.100 │
├────────────────────┼────────────────────┤
│ mutual information │ 0.010 │
╘════════════════════╧════════════════════╛
Process finished with exit code 0
```
### To View the Results
[](https://app.dash.ml/)
Note, the `jwt` based access control has not been implemented yet. So you should leave the token field empty.
### Logging to a Server
**Skip this if you just want to log locally.** When training in parallel, you want to kickstart an logging server (Instrument Server). To do so, run:
```bash
python -m ml_logger.server
```
Use ssh tunnel if you are running on a managed cluster (with SLURM for instance).
**Important**: to set allow remote logging, you need to pass in `--host=0.0.0.0` so that the server accepts non-localhost connections.
```bash
python -m ml_logger.server --host=0.0.0.0
```
### Asynchronously log the summary of LOTs of training metrics
A common scenario is you only want to upload averaged statistics of your metrics. A pattern
that @jachiam uses is the following: `store_metrics()`, `peak_stored_metrics()`, and `log_metrics_summary()`
```python
# You log lots of metrics during training.
for i in range(100):
logger.store_metrics(metrics={'some_val/smooth': 10}, some=20, timestep=i)
# you can peak what's inside the cache and print out a table like this:
logger.peek_stored_metrics(len=4)
# outputs ~>
# some | timestep |some_val/smooth
# ━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━┿━━━━━━━━━━━━━━━
# 20 | 0 | 10
# 20 | 1 | 10
# 20 | 2 | 10
# 20 | 3 | 10
# The metrics are stored in-memory. Now we need to actually log the summaries:
logger.log_metrics_summary(silent=True)
# outputs ~> . (data is now logged to the server)
```
## Table of Contents
- logging `matplotlib.pyplot` figures on an headless server
- [documentation under construction]
## How to Develop
First clone repo, install dev dependencies, and install the module under evaluation mode.
```bash
git clone https://github.com/episodeyang/ml_logger.git
cd ml_logger && cd ml_logger && pip install -r requirements-dev.txt
pip install -e .
```
## Testing local-mode (without a server)
You should be inside ml_logger/ml_logger folder
```bash
pwd # ~> ml_logger/ml_logger
make test
```
## Testing with a server (You need to do both for an PR)
To test with a live server, first run (in a separate console)
```
python -m ml_logger.server --data-dir /tmp/ml-logger-debug
```
or do:
```bash
make start-test-server
```
Then run this test script with the option:
```bash
python -m pytest tests --capture=no --data-dir http://0.0.0.0:8081
```
or do
```bash
make test-with-server
```
Your PR should have both of these two tests working. ToDo: add CI to this repo.
### To Publish
You need `twine`, `rst-lint` etc, which are included in the `requirements-dev.txt` file.
---
### Logging Matplotlib pyplots
### Configuring The Experiment Folder
```python
from ml_logger import logger, ML_Logger
from datetime import datetime
now = datetime.now()
logger.configure("/tmp/ml-logger-demo", "deep_Q_learning", f"{now:%Y%m%d-%H%M%S}")
```
This is a singleton pattern similar to `matplotlib.pyplot`. However, you could also use the logger constructor
```python
logger = ML_Logger(root_dir="/tmp/ml-logger-demo", prefix=f"deep_Q_learning/{now:%Y%m%d-%H%M%S}")
```
### Logging Text, and Metrics
```python
logger.log({"some_var/smooth": 10}, some=Color(0.85, 'yellow', percent), step=3)
```
colored output: (where the values are yellow)
```log
╒════════════════════╤════════════════════╕
│ some var/smooth │ 10 │
├────────────────────┼────────────────────┤
│ some │ 85.0% │
╘════════════════════╧════════════════════╛
```
### Logging Matplotlib Figures
We have optimized ML-Logger, so it supports any format that `pyplot` supports. To save a figure locally or remotely,
```python
import matplotlib.pyplot as plt
import numpy as np
xs = np.linspace(-5, 5)
plt.plot(xs, np.cos(xs), label='Cosine Func')
logger.savefig('cosine_function.pdf')
```
### Logging Videos
It is especially hard to visualize RL training sessions on a remote computer. With ML-Logger this is easy, and
super fast. We optimized the serialization and transport process, so that a large stack of video tensor gets
first compressed by `ffmepg` before getting sent over the wire.
The compression rate (and speed boost) can be 2000:1.
```python
import numpy as np
def im(x, y):
canvas = np.zeros((200, 200))
for i in range(200):
for j in range(200):
if x - 5 < i < x + 5 and y - 5 < j < y + 5:
canvas[i, j] = 1
return canvas
frames = [im(100 + i, 80) for i in range(20)]
logger.log_video(frames, "test_video.mp4")
```
### Saving PyTorch Modules
PyTorch has a very nice module saving and loading API that has inspired the one in `Keras`. We make it easy to save
this state dictionary (`state_dict`) to a server, and load it. This way you can load from 100+ of your previous
experiments, without having to download those weights to your code repository.
```python
# save a module
logger.save_module(cnn, "FastCNN.pkl")
# load a module
logger.load_module(cnn, f"FastCNN.pkl")
```
### Saving Tensorflow Models
The format tensorflow uses to save the models is opaque. I prefer to save model weights in `pickle` as a dictionary.
This way the weight files are transparent. ML_Logger offers easy helper functions to save and load from checkpoints
saved in this format:
```python
## To save checkpoint
from ml_logger import logger
import tensorflow as tf
logger.configure(log_directory="/tmp/ml-logger-demos")
x = tf.get_variable('x', shape=[], initializer=tf.constant_initializer(0.0))
y = tf.get_variable('y', shape=[], initializer=tf.constant_initializer(10.0))
c = tf.Variable(1000)
sess = tf.InteractiveSession()
sess.run(tf.global_variables_initializer())
trainables = tf.trainable_variables()
logger.save_variables(trainables, path="variables.pkl", namespace="checkpoints")
```
which creates a file `checkpoints/variables.pkl` under `/tmp/ml-logger-demos`.
## Visualization
An idea visualization dashboard would be
1. **Fast, instantaneous.** On an AWS headless server? View the plots as if they are on your local computer.
2. **Searchable, performantly.** So that you don't have to remember where an experiment is from last week.
3. **Answer Questions, from 100+ Experiments.** We make available Google's internal hyperparameter visualization tool,
on your own computer.
### Searching for Hyper Parameters
Experiments are identified by the `metrics.pkl` file. You can log multiple times to the same `metrics.pkl` file,
and the later parameter values overwrites earlier ones with the same key. We enforce namespace in this file, so each
key/value argument you pass into the `logger.log_parameters` function call has to be a dictionary.
```python
Args = dict(
learning_rate=10,
hidden_size=200
)
logger.log_parameters(Args=Args)
```
### How to launch the Vis App
**This requires node.js and yarn dev environment** at the moment. We will streamline this process without these
requirements soon.
0. download this repository
1. go to `ml-vis-app` folder
2. Install the dev dependencies
1. install node: [Installation](https://nodejs.org/en/download/)
2. install yarn: [Installation](https://yarnpkg.com/lang/en/docs/install/#mac-stable)
3. install the dependencies of this visualization app:
1. `yarn install`
3. in that folder, run `yarn`.
**The IP address of the server is currently hard coded
[here](https://github.com/episodeyang/ml_logger/blob/master/ml-vis-app/src/App.js#L11).** To use this with your own
instrumentation server, over-write this line. I'm planning on making this configuration more accessible.
## Full Logging API
```python
from ml_logger import logger, Color, percent
logger.log_params(G=dict(some_config="hey"))
logger.log(some=Color(0.1, 'yellow'), step=0)
logger.log(some=Color(0.28571, 'yellow', lambda v: "{:.5f}%".format(v * 100)), step=1)
logger.log(some=Color(0.85, 'yellow', percent), step=2)
logger.log({"some_var/smooth": 10}, some=Color(0.85, 'yellow', percent), step=3)
logger.log(some=Color(10, 'yellow'), step=4)
```
colored output: (where the values are yellow)
```log
╒════════════════════╤════════════════════╕
│ some │ 0.1 │
╘════════════════════╧════════════════════╛
╒════════════════════╤════════════════════╕
│ some │ 28.57100% │
╘════════════════════╧════════════════════╛
╒════════════════════╤════════════════════╕
│ some │ 85.0% │
╘════════════════════╧════════════════════╛
╒════════════════════╤════════════════════╕
│ some var/smooth │ 10 │
├────────────────────┼────────────────────┤
│ some │ 85.0% │
╘════════════════════╧════════════════════╛
```
This version of logger also prints out a tabular printout of the data you are logging to your `stdout`.
- can silence `stdout` per key (per `logger.log` call)
- can print with color: `logger.log(timestep, some_key=green(some_data))`
- can print with custom formatting: `logger.log(timestep, some_key=green(some_data, percent))` where `percent`
- uses the correct `unix` table characters (please stop using `|` and `+`. **Use `│`, `┼` instead**)
A typical print out of this logger look like the following:
```python
from ml_logger import ML_Logger
logger = ML_Logger(root_dir=f"/mnt/bucket/deep_Q_learning/{datetime.now(%Y%m%d-%H%M%S.%f):}")
logger.log_params(G=vars(G), RUN=vars(RUN), Reporting=vars(Reporting))
```
outputs the following
```log
═════════════════════════════════════════════════════
G
───────────────────────────────┬─────────────────────
env_name │ MountainCar-v0
seed │ None
stochastic_action │ True
conv_params │ None
value_params │ (64,)
use_layer_norm │ True
buffer_size │ 50000
replay_batch_size │ 32
prioritized_replay │ True
alpha │ 0.6
beta_start │ 0.4
beta_end │ 1.0
prioritized_replay_eps │ 1e-06
grad_norm_clipping │ 10
double_q │ True
use_dueling │ False
exploration_fraction │ 0.1
final_eps │ 0.1
n_timesteps │ 100000
learning_rate │ 0.001
gamma │ 1.0
learning_start │ 1000
learn_interval │ 1
target_network_update_interval │ 500
═══════════════════════════════╧═════════════════════
RUN
───────────────────────────────┬─────────────────────
log_directory │ /mnt/slab/krypton/machine_learning/ge_dqn/2017-11-20/162048.353909-MountainCar-v0-prioritized_replay(True)
checkpoint │ checkpoint.cp
log_file │ output.log
═══════════════════════════════╧═════════════════════
Reporting
───────────────────────────────┬─────────────────────
checkpoint_interval │ 10000
reward_average │ 100
print_interval │ 10
═══════════════════════════════╧═════════════════════
╒════════════════════╤════════════════════╕
│ timestep │ 1999 │
├────────────────────┼────────────────────┤
│ episode │ 10 │
├────────────────────┼────────────────────┤
│ total reward │ -200.0 │
├────────────────────┼────────────────────┤
│ total reward/mean │ -200.0 │
├────────────────────┼────────────────────┤
│ total reward/max │ -200.0 │
├────────────────────┼────────────────────┤
│time spent exploring│ 82.0% │
├────────────────────┼────────────────────┤
│ replay beta │ 0.41 │
╘════════════════════╧════════════════════╛
```
## TODO:
## Visualization (Preview):boom:
In addition, ml-logger also comes with a powerful visualization dashboard that beats tensorboard in every aspect.
#### An Example Log from ML-Logger
<img alt="example_real_log_output" src="figures/example_log_output.png" align="right"></img>
A common pain that comes after getting to launch ML training jobs on AWS
is a lack of a good way to manage and visualize your data. So far, a common
practice is to upload your experiment data to aws s3 or google cloud buckets.
Then one quickly realizes that downloading data from s3 can be slow. s3 does
not offer diffsync like gcloud-cli's `g rsync`. This makes it hard to sync a
large collection of data that is constantly appended to.
So far the best way we have found for organizing experimental data is to
have a centralized instrumentation server. Compared with managing your data
on S3, a centralized instrumentation server makes it much easier to move
experiments around, run analysis that is co-located with your data, and
hosting visualization dashboards on the same machine. To download data
locally, you can use `sshfs`, `smba`, `rsync` or a variety of remote disks. All
faster than s3.
ML-Logger is the logging utility that allows you to do this. To make ML_logger
easy to use, we made it so that you can use ml-logger with zero configuration,
logging to your local hard-drive by default. When the logging directory field
`logger.configure(log_directory= <your directory>)` is an http end point,
the logger will instantiate a fast, future based logging client that launches
http requests in a separate thread. We optimized the client so that it won't
slow down your training code.
API wise, ML-logger makes it easy for you to log textual printouts, simple
scalars, numpy tensors, image tensors, and `pyplot` figures. Because you might
also want to read data from the instrumentation server, we also made it possible to
load numpy, pickle, text and binary files remotely.
In the future, we will start building an integrated dashboard with fast search,
live figure update and markdown-based reporting/dashboarding to go with ml-logger.
Now give this a try, and profit!
Raw data
{
"_id": null,
"home_page": "https://github.com/geyang/ml-logger",
"name": "ml-logger",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8.0",
"maintainer_email": null,
"keywords": "ml_logger, visualization, logging, debug, debugging",
"author": "Ge Yang",
"author_email": "geyang@mit.edu",
"download_url": "https://files.pythonhosted.org/packages/3f/7e/d2d3edd0b720bbf6ca48356e068381737299ccb0e45296894772ae335576/ml_logger-0.10.24.tar.gz",
"platform": null,
"description": "# ML-Logger, A Simple and Scalable Logging Utility With a Beautiful Visualization Dashboard That Is Super Fast\n\n[](http://pepy.tech/project/ml-logger)\n\n<img alt=\"fast\" src=\"figures/ml-dash-v3.gif\" align=\"right\" width=\"50%\"/>\n\nML-Logger makes it easy to:\n\n- save data locally and remotely, as **binary**, in a transparent `pickle` file, with the same API and zero \nconfiguration.\n- write from 500+ worker containers to a single instrumentation server\n- visualize `matplotlib.pyplot` figures from a remote server locally with `logger.savefig('my_figure.png')`\n\nAnd ml-logger does all of these with *minimal configuration* \u2014 you can use the same logging \ncode both locally and remotely with no code change.\n\nML-logger is highly performant -- the remote writes are asynchronous. For this reason it doesn't slow down your training\neven with 100+ metric keys.\n\nWhy did we built this, you might ask? Because we want to make it easy for people in ML to \nuse the same logging code in all of they projects, so that it is easy to get started with \nsomeone else's baseline.\n\n\n## Getting Started!\n\nTo **install** `ml_logger`, do:\n```bash\npip install ml-logger\n```\n\n> The landscape of python modules is a lot messier than that of javascript. The most up-to-date `graphene`\u00a0requires the following versions:\n> \n> ```\n> yes | pip install graphene==2.1.3\n> yes | pip install graphql-core==2.1\n> yes | pip install graphql-relay==0.4.5\n> yes | pip install graphql-server-core==1.1.1\n> ```\n\nAdd the following to your `~/.zshrc`\n\n```bash\nexport ML_LOGGER_ROOT=<your-logging-server-address>\nexport ML_LOGGER_USER=<your-handle> # make sure it is the same on the cluster.\n```\n\nNow you can rock!\n\n```python\nfrom ml_logger import logger\n\nprint(logger)\n# ~> logging data to /tmp/ml-logger-debug\n```\nLog key/value pairs, and metrics:\n```python\nfor i in range(1):\n logger.log(metrics={'some_val/smooth': 10, 'status': f\"step ({i})\"}, reward=20, timestep=i)\n ### flush the data, otherwise the value would be overwritten with new values in the next iteration.\n logger.flush()\n# outputs ~>\n# \u2552\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2564\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2555\n# \u2502 reward \u2502 20 \u2502\n# \u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n...\n```\n\n## Simple Logging Example\n\nIn your project files, do:\n\n```python\nfrom params_proto import ParamsProto, Proto, Flag\n\nclass Args(ParamsProto):\n \"\"\" CLI Args for the program\n Try:\n python3 example.py --help\n And it should print out the help strings\n \"\"\"\n seed = Proto(1, help=\"random seed\")\n D_lr = 5e-4\n G_lr = 1e-4\n Q_lr = 1e-4\n T_lr = 1e-4\n plot_interval = 10\n verbose = Flag(\"the verbose flag\")\n\nif __name__ == '__main__':\n import scipy\n import numpy as np\n from ml_logger import logger, LOGGER_USER\n\n # Put in your ~/.bashrc\n # export ML_LOGGER_ROOT = \"http://<your-logging-server>:8081\"\n # export ML_LOGGER_USER = $USER\n logger.configure(prefix=f\"{LOGGER_USER}/scratch/your-experiment-prefix!\")\n\n\n logger.log_params(Args=vars(Args))\n logger.upload_file(__file__)\n\n for epoch in range(10):\n logger.log(step=epoch, D_loss=0.2, G_loss=0.1, mutual_information=0.01)\n logger.log_key_value(epoch, 'some string key', 0.0012)\n # when the step index updates, logger flushes all of the key-value pairs to file system/logging server\n\n logger.flush()\n\n # Images\n face = scipy.misc.face()\n face_bw = scipy.misc.face(gray=True)\n logger.save_image(face, \"figures/face_rgb.png\")\n logger.save_image(face_bw, \"figures/face_bw.png\")\n image_bw = np.zeros((64, 64, 1))\n image_bw_2 = scipy.misc.face(gray=True)[::4, ::4]\n\n logger.save_video([face] * 5, \"videos/face.mp4\")\n```\n\nAnd it should print out\n\n\n```bash\n\u2713 created a new logging client\nDashboard: http://app.dash.ml/user-name/your-experiment-prefix!\nLog_directory: http://<your-server-ip>:8080\n\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\n Args \n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n seed \u2502 1 \n D_lr \u2502 0.0005 \n G_lr \u2502 0.0001 \n Q_lr \u2502 0.0001 \n T_lr \u2502 0.0001 \n plot_interval \u2502 10 \n\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2567\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\n\u2552\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2564\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2555\n\u2502 step \u2502 9 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 D loss \u2502 0.200 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 G loss \u2502 0.100 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 mutual information \u2502 0.010 \u2502\n\u2558\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2567\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255b\n\n\nProcess finished with exit code 0\n```\n\n### To View the Results\n\n[](https://app.dash.ml/)\n\nNote, the `jwt` based access control has not been implemented yet. So you should leave the token field empty.\n\n### Logging to a Server\n\n**Skip this if you just want to log locally.** When training in parallel, you want to kickstart an logging server (Instrument Server). To do so, run:\n```bash\npython -m ml_logger.server\n```\nUse ssh tunnel if you are running on a managed cluster (with SLURM for instance).\n**Important**: to set allow remote logging, you need to pass in `--host=0.0.0.0` so that the server accepts non-localhost connections.\n\n```bash\npython -m ml_logger.server --host=0.0.0.0\n```\n\n### Asynchronously log the summary of LOTs of training metrics\n\nA common scenario is you only want to upload averaged statistics of your metrics. A pattern\nthat @jachiam uses is the following: `store_metrics()`, `peak_stored_metrics()`, and `log_metrics_summary()`\n\n```python\n# You log lots of metrics during training.\nfor i in range(100):\n logger.store_metrics(metrics={'some_val/smooth': 10}, some=20, timestep=i)\n# you can peak what's inside the cache and print out a table like this: \nlogger.peek_stored_metrics(len=4)\n# outputs ~>\n# some | timestep |some_val/smooth\n# \u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u253f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u253f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\n# 20 | 0 | 10\n# 20 | 1 | 10\n# 20 | 2 | 10\n# 20 | 3 | 10\n\n# The metrics are stored in-memory. Now we need to actually log the summaries:\nlogger.log_metrics_summary(silent=True)\n# outputs ~> . (data is now logged to the server)\n```\n\n## Table of Contents\n\n- logging `matplotlib.pyplot` figures on an headless server\n- [documentation under construction]\n\n## How to Develop\n\nFirst clone repo, install dev dependencies, and install the module under evaluation mode.\n```bash\ngit clone https://github.com/episodeyang/ml_logger.git\ncd ml_logger && cd ml_logger && pip install -r requirements-dev.txt\npip install -e .\n```\n## Testing local-mode (without a server)\n\nYou should be inside ml_logger/ml_logger folder\n```bash\npwd # ~> ml_logger/ml_logger\nmake test\n```\n\n## Testing with a server (You need to do both for an PR)\n\nTo test with a live server, first run (in a separate console)\n```\npython -m ml_logger.server --data-dir /tmp/ml-logger-debug\n```\nor do:\n```bash\nmake start-test-server\n```\n\nThen run this test script with the option:\n```bash\npython -m pytest tests --capture=no --data-dir http://0.0.0.0:8081\n```\nor do\n```bash\nmake test-with-server\n```\n\nYour PR should have both of these two tests working. ToDo: add CI to this repo.\n\n### To Publish\n\nYou need `twine`, `rst-lint` etc, which are included in the `requirements-dev.txt` file.\n\n---\n\n### Logging Matplotlib pyplots\n\n### Configuring The Experiment Folder\n\n```python\nfrom ml_logger import logger, ML_Logger\nfrom datetime import datetime\n\nnow = datetime.now()\nlogger.configure(\"/tmp/ml-logger-demo\", \"deep_Q_learning\", f\"{now:%Y%m%d-%H%M%S}\")\n```\nThis is a singleton pattern similar to `matplotlib.pyplot`. However, you could also use the logger constructor\n```python\nlogger = ML_Logger(root_dir=\"/tmp/ml-logger-demo\", prefix=f\"deep_Q_learning/{now:%Y%m%d-%H%M%S}\")\n```\n\n### Logging Text, and Metrics\n\n```python\nlogger.log({\"some_var/smooth\": 10}, some=Color(0.85, 'yellow', percent), step=3)\n```\n\ncolored output: (where the values are yellow)\n```log\n\u2552\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2564\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2555\n\u2502 some var/smooth \u2502 10 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 some \u2502 85.0% \u2502\n\u2558\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2567\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255b\n```\n\n### Logging Matplotlib Figures\n\nWe have optimized ML-Logger, so it supports any format that `pyplot` supports. To save a figure locally or remotely, \n```python\nimport matplotlib.pyplot as plt\nimport numpy as np\n\nxs = np.linspace(-5, 5)\n\nplt.plot(xs, np.cos(xs), label='Cosine Func')\nlogger.savefig('cosine_function.pdf')\n```\n\n### Logging Videos\n\nIt is especially hard to visualize RL training sessions on a remote computer. With ML-Logger this is easy, and \nsuper fast. We optimized the serialization and transport process, so that a large stack of video tensor gets\nfirst compressed by `ffmepg` before getting sent over the wire. \n\nThe compression rate (and speed boost) can be 2000:1.\n\n```python\nimport numpy as np\n\ndef im(x, y):\n canvas = np.zeros((200, 200))\n for i in range(200):\n for j in range(200):\n if x - 5 < i < x + 5 and y - 5 < j < y + 5:\n canvas[i, j] = 1\n return canvas\n\nframes = [im(100 + i, 80) for i in range(20)]\n\nlogger.log_video(frames, \"test_video.mp4\")\n```\n\n### Saving PyTorch Modules\n\nPyTorch has a very nice module saving and loading API that has inspired the one in `Keras`. We make it easy to save\nthis state dictionary (`state_dict`) to a server, and load it. This way you can load from 100+ of your previous \nexperiments, without having to download those weights to your code repository.\n\n```python\n# save a module\nlogger.save_module(cnn, \"FastCNN.pkl\")\n\n# load a module\nlogger.load_module(cnn, f\"FastCNN.pkl\")\n```\n\n### Saving Tensorflow Models\n\nThe format tensorflow uses to save the models is opaque. I prefer to save model weights in `pickle` as a dictionary. \nThis way the weight files are transparent. ML_Logger offers easy helper functions to save and load from checkpoints \nsaved in this format:\n\n```python\n## To save checkpoint\nfrom ml_logger import logger\nimport tensorflow as tf\n\nlogger.configure(log_directory=\"/tmp/ml-logger-demos\")\n\nx = tf.get_variable('x', shape=[], initializer=tf.constant_initializer(0.0))\ny = tf.get_variable('y', shape=[], initializer=tf.constant_initializer(10.0))\nc = tf.Variable(1000)\n\nsess = tf.InteractiveSession()\nsess.run(tf.global_variables_initializer())\n\ntrainables = tf.trainable_variables()\nlogger.save_variables(trainables, path=\"variables.pkl\", namespace=\"checkpoints\")\n```\nwhich creates a file `checkpoints/variables.pkl` under `/tmp/ml-logger-demos`.\n\n## Visualization\n\nAn idea visualization dashboard would be\n1. **Fast, instantaneous.** On an AWS headless server? View the plots as if they are on your local computer.\n2. **Searchable, performantly.** So that you don't have to remember where an experiment is from last week.\n3. **Answer Questions, from 100+ Experiments.** We make available Google's internal hyperparameter visualization tool, \non your own computer.\n\n### Searching for Hyper Parameters\n\nExperiments are identified by the `metrics.pkl` file. You can log multiple times to the same `metrics.pkl` file, \nand the later parameter values overwrites earlier ones with the same key. We enforce namespace in this file, so each\nkey/value argument you pass into the `logger.log_parameters` function call has to be a dictionary.\n\n```python\nArgs = dict(\n learning_rate=10,\n hidden_size=200\n)\nlogger.log_parameters(Args=Args)\n```\n\n### How to launch the Vis App\n\n**This requires node.js and yarn dev environment** at the moment. We will streamline this process without these \nrequirements soon.\n\n0. download this repository\n1. go to `ml-vis-app` folder\n2. Install the dev dependencies\n 1. install node: [Installation](https://nodejs.org/en/download/)\n 2. install yarn: [Installation](https://yarnpkg.com/lang/en/docs/install/#mac-stable)\n 3. install the dependencies of this visualization app:\n 1. `yarn install`\n3. in that folder, run `yarn`.\n\n**The IP address of the server is currently hard coded \n[here](https://github.com/episodeyang/ml_logger/blob/master/ml-vis-app/src/App.js#L11).** To use this with your own \ninstrumentation server, over-write this line. I'm planning on making this configuration more accessible.\n\n\n## Full Logging API\n\n```python\nfrom ml_logger import logger, Color, percent\n\nlogger.log_params(G=dict(some_config=\"hey\"))\nlogger.log(some=Color(0.1, 'yellow'), step=0)\nlogger.log(some=Color(0.28571, 'yellow', lambda v: \"{:.5f}%\".format(v * 100)), step=1)\nlogger.log(some=Color(0.85, 'yellow', percent), step=2)\nlogger.log({\"some_var/smooth\": 10}, some=Color(0.85, 'yellow', percent), step=3)\nlogger.log(some=Color(10, 'yellow'), step=4)\n```\n\ncolored output: (where the values are yellow)\n```log\n\u2552\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2564\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2555\n\u2502 some \u2502 0.1 \u2502\n\u2558\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2567\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255b\n\u2552\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2564\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2555\n\u2502 some \u2502 28.57100% \u2502\n\u2558\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2567\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255b\n\u2552\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2564\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2555\n\u2502 some \u2502 85.0% \u2502\n\u2558\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2567\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255b\n\u2552\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2564\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2555\n\u2502 some var/smooth \u2502 10 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 some \u2502 85.0% \u2502\n\u2558\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2567\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255b\n```\n\nThis version of logger also prints out a tabular printout of the data you are logging to your `stdout`.\n- can silence `stdout` per key (per `logger.log` call)\n- can print with color: `logger.log(timestep, some_key=green(some_data))`\n- can print with custom formatting: `logger.log(timestep, some_key=green(some_data, percent))` where `percent`\n- uses the correct `unix` table characters (please stop using `|` and `+`. **Use `\u2502`, `\u253c` instead**)\n\nA typical print out of this logger look like the following:\n\n```python\nfrom ml_logger import ML_Logger\n\nlogger = ML_Logger(root_dir=f\"/mnt/bucket/deep_Q_learning/{datetime.now(%Y%m%d-%H%M%S.%f):}\")\n\nlogger.log_params(G=vars(G), RUN=vars(RUN), Reporting=vars(Reporting))\n```\noutputs the following\n\n```log\n\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\n G \n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n env_name \u2502 MountainCar-v0 \n seed \u2502 None \n stochastic_action \u2502 True \n conv_params \u2502 None \n value_params \u2502 (64,) \n use_layer_norm \u2502 True \n buffer_size \u2502 50000 \n replay_batch_size \u2502 32 \n prioritized_replay \u2502 True \n alpha \u2502 0.6 \n beta_start \u2502 0.4 \n beta_end \u2502 1.0 \n prioritized_replay_eps \u2502 1e-06 \n grad_norm_clipping \u2502 10 \n double_q \u2502 True \n use_dueling \u2502 False \n exploration_fraction \u2502 0.1 \n final_eps \u2502 0.1 \n n_timesteps \u2502 100000 \n learning_rate \u2502 0.001 \n gamma \u2502 1.0 \n learning_start \u2502 1000 \n learn_interval \u2502 1 \ntarget_network_update_interval \u2502 500 \n\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2567\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\n RUN \n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n log_directory \u2502 /mnt/slab/krypton/machine_learning/ge_dqn/2017-11-20/162048.353909-MountainCar-v0-prioritized_replay(True)\n checkpoint \u2502 checkpoint.cp \n log_file \u2502 output.log \n\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2567\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\n Reporting \n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n checkpoint_interval \u2502 10000 \n reward_average \u2502 100 \n print_interval \u2502 10 \n\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2567\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\n\u2552\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2564\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2555\n\u2502 timestep \u2502 1999 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 episode \u2502 10 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 total reward \u2502 -200.0 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 total reward/mean \u2502 -200.0 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 total reward/max \u2502 -200.0 \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502time spent exploring\u2502 82.0% \u2502\n\u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\n\u2502 replay beta \u2502 0.41 \u2502\n\u2558\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2567\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u2550\u255b\n```\n\n\n## TODO:\n\n\n## Visualization (Preview):boom:\n\nIn addition, ml-logger also comes with a powerful visualization dashboard that beats tensorboard in every aspect.\n\n\n\n#### An Example Log from ML-Logger\n<img alt=\"example_real_log_output\" src=\"figures/example_log_output.png\" align=\"right\"></img>\n\nA common pain that comes after getting to launch ML training jobs on AWS\nis a lack of a good way to manage and visualize your data. So far, a common\npractice is to upload your experiment data to aws s3 or google cloud buckets. \nThen one quickly realizes that downloading data from s3 can be slow. s3 does \n\nnot offer diffsync like gcloud-cli's `g rsync`. This makes it hard to sync a \nlarge collection of data that is constantly appended to.\n\nSo far the best way we have found for organizing experimental data is to \nhave a centralized instrumentation server. Compared with managing your data \non S3, a centralized instrumentation server makes it much easier to move \nexperiments around, run analysis that is co-located with your data, and \nhosting visualization dashboards on the same machine. To download data \nlocally, you can use `sshfs`, `smba`, `rsync` or a variety of remote disks. All\nfaster than s3.\n\nML-Logger is the logging utility that allows you to do this. To make ML_logger\neasy to use, we made it so that you can use ml-logger with zero configuration,\nlogging to your local hard-drive by default. When the logging directory field \n`logger.configure(log_directory= <your directory>)` is an http end point, \nthe logger will instantiate a fast, future based logging client that launches \nhttp requests in a separate thread. We optimized the client so that it won't \nslow down your training code.\n\nAPI wise, ML-logger makes it easy for you to log textual printouts, simple \nscalars, numpy tensors, image tensors, and `pyplot` figures. Because you might\nalso want to read data from the instrumentation server, we also made it possible to \nload numpy, pickle, text and binary files remotely.\n\nIn the future, we will start building an integrated dashboard with fast search, \nlive figure update and markdown-based reporting/dashboarding to go with ml-logger.\n\nNow give this a try, and profit!\n",
"bugtrack_url": null,
"license": "MIT",
"summary": null,
"version": "0.10.24",
"project_urls": {
"Homepage": "https://github.com/geyang/ml-logger",
"Repository": "https://github.com/geyang/ml-logger"
},
"split_keywords": [
"ml_logger",
" visualization",
" logging",
" debug",
" debugging"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "4b111e2d04b398ce5009c3c2ec1dc12ed6dab039308ab910e993dc017087dcb5",
"md5": "03a91e79d605cd5a082c03ca80b0f7e3",
"sha256": "e3fd5424e33438279ddfdc4ce153b33004cae1b0547e379c95871ebbef1b4ad1"
},
"downloads": -1,
"filename": "ml_logger-0.10.24-py3-none-any.whl",
"has_sig": false,
"md5_digest": "03a91e79d605cd5a082c03ca80b0f7e3",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8.0",
"size": 67248,
"upload_time": "2024-04-19T07:15:51",
"upload_time_iso_8601": "2024-04-19T07:15:51.647595Z",
"url": "https://files.pythonhosted.org/packages/4b/11/1e2d04b398ce5009c3c2ec1dc12ed6dab039308ab910e993dc017087dcb5/ml_logger-0.10.24-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "3f7ed2d3edd0b720bbf6ca48356e068381737299ccb0e45296894772ae335576",
"md5": "66a6ea18ada3f2be383cd5547330aa5f",
"sha256": "d42c3b83da75bbae60b15d1eb63ecca3cec6f2ba7efadddb653009f594d1a63f"
},
"downloads": -1,
"filename": "ml_logger-0.10.24.tar.gz",
"has_sig": false,
"md5_digest": "66a6ea18ada3f2be383cd5547330aa5f",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8.0",
"size": 67655,
"upload_time": "2024-04-19T07:15:54",
"upload_time_iso_8601": "2024-04-19T07:15:54.490747Z",
"url": "https://files.pythonhosted.org/packages/3f/7e/d2d3edd0b720bbf6ca48356e068381737299ccb0e45296894772ae335576/ml_logger-0.10.24.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-19 07:15:54",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "geyang",
"github_project": "ml-logger",
"github_not_found": true,
"lcname": "ml-logger"
}
JSON
