# ❤️ Lovely Tensors
<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->
## [Read full docs](https://xl0.github.io/lovely-tensors)
### More lovely stuff
##### Working with numbers
- [Numpy](https://numpy.org/): ❤️ [Lovely
NumPy](https://github.com/xl0/lovely-numpy)
- [JAX](https://jax.readthedocs.io/): 💘 [Lovely
`JAX`](https://github.com/xl0/lovely-jax)
- [TinyGrad](https://github.com/tinygrad/tinygrad): 🫀 [Lovely
Grad](https://github.com/xl0/lovely-grad)
##### Community
- [Discord](https://discord.gg/qBaqauUWXP)
## Install
``` sh
pip install lovely-tensors
```
or
``` sh
mamba install lovely-tensors
```
or
``` sh
conda install -c conda-forge lovely-tensors
```
## How to use
How often do you find yourself debugging PyTorch code? You dump a tensor
to the cell output, and see this:
``` python
numbers
```
tensor([[[-0.3541, -0.3369, -0.4054, ..., -0.5596, -0.4739, 2.2489],
[-0.4054, -0.4226, -0.4911, ..., -0.9192, -0.8507, 2.1633],
[-0.4739, -0.4739, -0.5424, ..., -1.0390, -1.0390, 2.1975],
...,
[-0.9020, -0.8335, -0.9363, ..., -1.4672, -1.2959, 2.2318],
[-0.8507, -0.7822, -0.9363, ..., -1.6042, -1.5014, 2.1804],
[-0.8335, -0.8164, -0.9705, ..., -1.6555, -1.5528, 2.1119]],
[[-0.1975, -0.1975, -0.3025, ..., -0.4776, -0.3725, 2.4111],
[-0.2500, -0.2325, -0.3375, ..., -0.7052, -0.6702, 2.3585],
[-0.3025, -0.2850, -0.3901, ..., -0.7402, -0.8102, 2.3761],
...,
[-0.4251, -0.2325, -0.3725, ..., -1.0903, -1.0203, 2.4286],
[-0.3901, -0.2325, -0.4251, ..., -1.2304, -1.2304, 2.4111],
[-0.4076, -0.2850, -0.4776, ..., -1.2829, -1.2829, 2.3410]],
[[-0.6715, -0.9853, -0.8807, ..., -0.9678, -0.6890, 2.3960],
[-0.7238, -1.0724, -0.9678, ..., -1.2467, -1.0201, 2.3263],
[-0.8284, -1.1247, -1.0201, ..., -1.2641, -1.1596, 2.3786],
...,
[-1.2293, -1.4733, -1.3861, ..., -1.5081, -1.2641, 2.5180],
[-1.1944, -1.4559, -1.4210, ..., -1.6476, -1.4733, 2.4308],
[-1.2293, -1.5256, -1.5081, ..., -1.6824, -1.5256, 2.3611]]])
Was it really useful for you, as a human, to see all these numbers?
What is the shape? The size?
What are the statistics?
Are any of the values `nan` or `inf`?
Is it an image of a man holding a tench?
``` python
import lovely_tensors as lt
```
``` python
lt.monkey_patch()
```
## Summary
``` python
numbers # torch.Tensor
```
tensor[3, 196, 196] n=115248 (0.4Mb) x∈[-2.118, 2.640] μ=-0.388 σ=1.073
``` python
numbers.rgb
```
``` python
numbers.plt
```
Better, huh?
``` python
numbers[1,:6,1] # Still shows values if there are not too many.
```
tensor[6] x∈[-0.443, -0.197] μ=-0.311 σ=0.091 [-0.197, -0.232, -0.285, -0.373, -0.443, -0.338]
``` python
spicy = numbers[0,:12,0].clone()
spicy[0] *= 10000
spicy[1] /= 10000
spicy[2] = float('inf')
spicy[3] = float('-inf')
spicy[4] = float('nan')
spicy = spicy.reshape((2,6))
spicy # Spicy stuff
```
tensor[2, 6] n=12 x∈[-3.541e+03, -4.054e-05] μ=-393.842 σ=1.180e+03 +Inf! -Inf! NaN!
``` python
torch.zeros(10, 10) # A zero tensor - make it obvious
```
tensor[10, 10] n=100 all_zeros
``` python
spicy.v # Verbose
```
tensor[2, 6] n=12 x∈[-3.541e+03, -4.054e-05] μ=-393.842 σ=1.180e+03 +Inf! -Inf! NaN!
tensor([[-3.5405e+03, -4.0543e-05, inf, -inf, nan, -6.1093e-01],
[-6.1093e-01, -5.9380e-01, -5.9380e-01, -5.4243e-01, -5.4243e-01, -5.4243e-01]])
``` python
spicy.p # The plain old way
```
tensor([[-3.5405e+03, -4.0543e-05, inf, -inf, nan, -6.1093e-01],
[-6.1093e-01, -5.9380e-01, -5.9380e-01, -5.4243e-01, -5.4243e-01, -5.4243e-01]])
## Named dimensions
``` python
named_numbers = numbers.rename("C", "H","W")
named_numbers
```
/home/xl0/mambaforge/envs/lovely-py31-torch25/lib/python3.10/site-packages/torch/_tensor.py:1420: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at ../c10/core/TensorImpl.h:1925.)
return super().rename(names)
tensor[C=3, H=196, W=196] n=115248 (0.4Mb) x∈[-2.118, 2.640] μ=-0.388 σ=1.073
## Going `.deeper`
``` python
numbers.deeper
```
tensor[3, 196, 196] n=115248 (0.4Mb) x∈[-2.118, 2.640] μ=-0.388 σ=1.073
tensor[196, 196] n=38416 x∈[-2.118, 2.249] μ=-0.324 σ=1.036
tensor[196, 196] n=38416 x∈[-1.966, 2.429] μ=-0.274 σ=0.973
tensor[196, 196] n=38416 x∈[-1.804, 2.640] μ=-0.567 σ=1.178
``` python
# You can go deeper if you need to
# And we can use `.deeper` with named dimensions.
named_numbers.deeper(2)
```
tensor[C=3, H=196, W=196] n=115248 (0.4Mb) x∈[-2.118, 2.640] μ=-0.388 σ=1.073
tensor[H=196, W=196] n=38416 x∈[-2.118, 2.249] μ=-0.324 σ=1.036
tensor[W=196] x∈[-1.912, 2.249] μ=-0.673 σ=0.522
tensor[W=196] x∈[-1.861, 2.163] μ=-0.738 σ=0.418
tensor[W=196] x∈[-1.758, 2.198] μ=-0.806 σ=0.397
tensor[W=196] x∈[-1.656, 2.249] μ=-0.849 σ=0.369
tensor[W=196] x∈[-1.673, 2.198] μ=-0.857 σ=0.357
tensor[W=196] x∈[-1.656, 2.146] μ=-0.848 σ=0.372
tensor[W=196] x∈[-1.433, 2.215] μ=-0.784 σ=0.397
tensor[W=196] x∈[-1.279, 2.249] μ=-0.695 σ=0.486
tensor[W=196] x∈[-1.364, 2.249] μ=-0.637 σ=0.539
...
tensor[H=196, W=196] n=38416 x∈[-1.966, 2.429] μ=-0.274 σ=0.973
tensor[W=196] x∈[-1.861, 2.411] μ=-0.529 σ=0.556
tensor[W=196] x∈[-1.826, 2.359] μ=-0.562 σ=0.473
tensor[W=196] x∈[-1.756, 2.376] μ=-0.622 σ=0.458
tensor[W=196] x∈[-1.633, 2.429] μ=-0.664 σ=0.430
tensor[W=196] x∈[-1.651, 2.376] μ=-0.669 σ=0.399
tensor[W=196] x∈[-1.633, 2.376] μ=-0.701 σ=0.391
tensor[W=196] x∈[-1.563, 2.429] μ=-0.670 σ=0.380
tensor[W=196] x∈[-1.475, 2.429] μ=-0.616 σ=0.386
tensor[W=196] x∈[-1.511, 2.429] μ=-0.593 σ=0.399
...
tensor[H=196, W=196] n=38416 x∈[-1.804, 2.640] μ=-0.567 σ=1.178
tensor[W=196] x∈[-1.717, 2.396] μ=-0.982 σ=0.350
tensor[W=196] x∈[-1.752, 2.326] μ=-1.034 σ=0.314
tensor[W=196] x∈[-1.648, 2.379] μ=-1.086 σ=0.314
tensor[W=196] x∈[-1.630, 2.466] μ=-1.121 σ=0.305
tensor[W=196] x∈[-1.717, 2.448] μ=-1.120 σ=0.302
tensor[W=196] x∈[-1.717, 2.431] μ=-1.166 σ=0.314
tensor[W=196] x∈[-1.560, 2.448] μ=-1.124 σ=0.326
tensor[W=196] x∈[-1.421, 2.431] μ=-1.064 σ=0.383
tensor[W=196] x∈[-1.526, 2.396] μ=-1.047 σ=0.417
...
## Now in `.rgb` color
The important queston - is it our man?
``` python
numbers.rgb
```
*Maaaaybe?* Looks like someone normalized him.
``` python
in_stats = ( (0.485, 0.456, 0.406), # mean
(0.229, 0.224, 0.225) ) # std
# numbers.rgb(in_stats, cl=True) # For channel-last input format
numbers.rgb(in_stats)
```
It’s indeed our hero, the Tenchman!
## `.plt` the statistics
``` python
(numbers+3).plt(center="mean", max_s=1000)
```
``` python
(numbers).plt
```
``` python
(numbers+3).plt(center="range")
```
## See the `.chans`
``` python
# .chans will map values betwen [-1,1] to colors.
# Make our values fit into that range to avoid clipping.
mean = torch.tensor(in_stats[0])[:,None,None]
std = torch.tensor(in_stats[1])[:,None,None]
numbers_01 = (numbers*std + mean)
numbers_01
```
tensor[3, 196, 196] n=115248 (0.4Mb) x∈[0., 1.000] μ=0.361 σ=0.248
``` python
numbers_01.chans
```
Let’s try with a Convolutional Neural Network
``` python
from torchvision.models import vgg11
```
``` python
features: torch.nn.Sequential = vgg11().features
# I saved the first 5 layers in "features.pt"
_ = features.load_state_dict(torch.load("../features.pt", weights_only=True), strict=False)
```
``` python
# Activatons of the second max pool layer of VGG11
acts = (features[:6](numbers[None])[0]/2) # /2 to reduce clipping
acts
```
tensor[128, 49, 49] n=307328 (1.2Mb) x∈[0., 12.508] μ=0.367 σ=0.634 grad DivBackward0
``` python
acts[:4].chans(cmap="coolwarm", scale=4)
```
## Grouping
``` python
# Make 8 images with progressively higher brightness and stack them 2x2x2.
eight_images = (torch.stack([numbers]*8)
.add(torch.linspace(-3, 3, 8)[:,None,None,None])
.mul(torch.tensor(in_stats[1])[:,None,None])
.add(torch.tensor(in_stats[0])[:,None,None])
.clamp(0,1)
.view(2,2,2,3,196,196)
)
eight_images
```
tensor[2, 2, 2, 3, 196, 196] n=921984 (3.5Mb) x∈[0., 1.000] μ=0.411 σ=0.369
``` python
eight_images.rgb
```
``` python
# Weights of the second conv layer of VGG11
features[3].weight
```
Parameter[128, 64, 3, 3] n=73728 (0.3Mb) x∈[-0.783, 0.776] μ=-0.004 σ=0.065 grad
I want +/- 2σ to fall in the range \[-1..1\]
``` python
weights = features[3].weight.data
weights = weights / (2*2*weights.std()) # *2 because we want 2σ on both sides, so 4σ
# weights += weights.std() * 2
weights.plt
```
``` python
# Weights of the second conv layer (64ch -> 128ch) of VGG11,
# grouped per output channel.
weights.chans(frame_px=1, gutter_px=0)
```
It’s a bit hard to see. Scale up 10x, but onyl show the first 4 filters.
``` python
weights[:4].chans(frame_px=1, gutter_px=0, scale=10)
```
## Options \| [Docs](https://xl0.github.io/lovely-tensors/utils.config.html)
``` python
from lovely_tensors import set_config, config, lovely, get_config
```
``` python
set_config(precision=1, sci_mode=True, color=False)
torch.tensor([1, 2, torch.nan])
```
tensor[3] μ=1.5e+00 σ=7.1e-01 NaN! [1.0e+00, 2.0e+00, nan]
``` python
set_config(precision=None, sci_mode=None, color=None) # None -> Reset to defaults
```
``` python
print(torch.tensor([1., 2]))
# Or with config context manager.
with config(sci_mode=True, precision=5):
print(torch.tensor([1., 2]))
print(torch.tensor([1., 2]))
```
tensor[2] μ=1.500 σ=0.707 [1.000, 2.000]
tensor[2] μ=1.50000e+00 σ=7.07107e-01 [1.00000e+00, 2.00000e+00]
tensor[2] μ=1.500 σ=0.707 [1.000, 2.000]
## Without `.monkey_patch`
``` python
lt.lovely(spicy)
```
tensor[2, 6] n=12 x∈[-3.541e+03, -4.054e-05] μ=-393.842 σ=1.180e+03 +Inf! -Inf! NaN!
``` python
lt.lovely(spicy, verbose=True)
```
tensor[2, 6] n=12 x∈[-3.541e+03, -4.054e-05] μ=-393.842 σ=1.180e+03 +Inf! -Inf! NaN!
tensor([[-3.5405e+03, -4.0543e-05, inf, -inf, nan, -6.1093e-01],
[-6.1093e-01, -5.9380e-01, -5.9380e-01, -5.4243e-01, -5.4243e-01, -5.4243e-01]])
``` python
lt.lovely(numbers, depth=1)
```
tensor[3, 196, 196] n=115248 (0.4Mb) x∈[-2.118, 2.640] μ=-0.388 σ=1.073
tensor[196, 196] n=38416 x∈[-2.118, 2.249] μ=-0.324 σ=1.036
tensor[196, 196] n=38416 x∈[-1.966, 2.429] μ=-0.274 σ=0.973
tensor[196, 196] n=38416 x∈[-1.804, 2.640] μ=-0.567 σ=1.178
``` python
lt.rgb(numbers, in_stats)
```
``` python
lt.plot(numbers, center="mean")
```
``` python
lt.chans(numbers_01)
```
## Matplotlib integration \| [Docs](https://xl0.github.io/lovely-tensors/matplotlib.html)
``` python
numbers.rgb(in_stats).fig # matplotlib figure
```
``` python
(numbers*0.3+0.5).chans.fig # matplotlib figure
```
``` python
numbers.plt.fig.savefig('pretty.svg') # Save it
```
``` python
!file pretty.svg; rm pretty.svg
```
pretty.svg: SVG Scalable Vector Graphics image
### Add content to existing Axes
``` python
fig = plt.figure(figsize=(8,3))
fig.set_constrained_layout(True)
gs = fig.add_gridspec(2,2)
ax1 = fig.add_subplot(gs[0, :])
ax2 = fig.add_subplot(gs[1, 0])
ax3 = fig.add_subplot(gs[1,1:])
ax2.set_axis_off()
ax3.set_axis_off()
numbers_01.plt(ax=ax1)
numbers_01.rgb(ax=ax2)
numbers_01.chans(ax=ax3);
```
## torch.compile()
Just works.
``` python
def func(x):
return x*2
if torch.__version__ >= "2.0":
func = torch.compile(func)
func(torch.tensor([1,2,3]))
```
tensor[3] i64 x∈[2, 6] μ=4.000 σ=2.000 [2, 4, 6]
## Inport hook
Lovely tensors installes an import hook. Set `LOVELY_TENSORS=1`, and it
will load automatically, no need to modify the code: \> Note: Don’t set
it globally, or all python scripts you run will import LT and PyTorch,
which will slow things down.
``` python
import torch
x = torch.randn(4, 16)
print(x)
```
``` bash
LOVELY_TENSORS=1 python test.py
```
x: tensor[4, 16] n=64 x∈[-1.652, 1.813] μ=-0.069 σ=0.844
This is especially useful in combination with [Better
Exceptions](https://github.com/Qix-/better-exceptions):
``` python
import torch
x = torch.randn(4, 16)
print(f"x: {x}")
w = torch.randn(15, 8)
y = torch.matmul(x, w) # Dimension mismatch
```
``` bash
BETTER_EXCEPTIONS=1 LOVELY_TENSORS=1 python test.py
```
x: tensor[4, 16] n=64 x∈[-1.834, 2.421] μ=0.103 σ=0.896
Traceback (most recent call last):
File "/home/xl0/work/projects/lovely-tensors/test.py", line 7, in <module>
y = torch.matmul(x, w)
│ │ └ tensor[15, 8] n=120 x∈[-2.355, 2.165] μ=0.142 σ=0.989
│ └ tensor[4, 16] n=64 x∈[-1.834, 2.421] μ=0.103 σ=0.896
└ <module 'torch' from '/home/xl0/mambaforge/envs/torch25-py313/lib/python3.12/site-packages/torch/__init__.py'>
RuntimeError: mat1 and mat2 shapes cannot be multiplied (4x16 and 15x8)
Raw data
{
"_id": null,
"home_page": "https://github.com/xl0/lovely-tensors",
"name": "lovely-tensors",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.7",
"maintainer_email": null,
"keywords": "jupyter pytorch tensor visualisation",
"author": "Alexey Zaytsev",
"author_email": "alexey.zaytsev@gmail.com",
"download_url": "https://files.pythonhosted.org/packages/20/c4/b109b6c912929a471cf03c6a2f02d8c5ab2889ec57177692d7b971c4ec70/lovely_tensors-0.1.18.tar.gz",
"platform": null,
"description": "# \u2764\ufe0f Lovely Tensors\n\n\n<!-- WARNING: THIS FILE WAS AUTOGENERATED! DO NOT EDIT! -->\n\n## [Read full docs](https://xl0.github.io/lovely-tensors)\n\n### More lovely stuff\n\n##### Working with numbers\n\n- [Numpy](https://numpy.org/): \u2764\ufe0f [Lovely\n NumPy](https://github.com/xl0/lovely-numpy)\n- [JAX](https://jax.readthedocs.io/): \ud83d\udc98 [Lovely\n `JAX`](https://github.com/xl0/lovely-jax)\n- [TinyGrad](https://github.com/tinygrad/tinygrad): \ud83e\udec0 [Lovely\n Grad](https://github.com/xl0/lovely-grad)\n\n##### Community\n\n- [Discord](https://discord.gg/qBaqauUWXP)\n\n## Install\n\n``` sh\npip install lovely-tensors\n```\n\nor\n\n``` sh\nmamba install lovely-tensors\n```\n\nor\n\n``` sh\nconda install -c conda-forge lovely-tensors\n```\n\n## How to use\n\nHow often do you find yourself debugging PyTorch code? You dump a tensor\nto the cell output, and see this:\n\n``` python\nnumbers\n```\n\n tensor([[[-0.3541, -0.3369, -0.4054, ..., -0.5596, -0.4739, 2.2489],\n [-0.4054, -0.4226, -0.4911, ..., -0.9192, -0.8507, 2.1633],\n [-0.4739, -0.4739, -0.5424, ..., -1.0390, -1.0390, 2.1975],\n ...,\n [-0.9020, -0.8335, -0.9363, ..., -1.4672, -1.2959, 2.2318],\n [-0.8507, -0.7822, -0.9363, ..., -1.6042, -1.5014, 2.1804],\n [-0.8335, -0.8164, -0.9705, ..., -1.6555, -1.5528, 2.1119]],\n\n [[-0.1975, -0.1975, -0.3025, ..., -0.4776, -0.3725, 2.4111],\n [-0.2500, -0.2325, -0.3375, ..., -0.7052, -0.6702, 2.3585],\n [-0.3025, -0.2850, -0.3901, ..., -0.7402, -0.8102, 2.3761],\n ...,\n [-0.4251, -0.2325, -0.3725, ..., -1.0903, -1.0203, 2.4286],\n [-0.3901, -0.2325, -0.4251, ..., -1.2304, -1.2304, 2.4111],\n [-0.4076, -0.2850, -0.4776, ..., -1.2829, -1.2829, 2.3410]],\n\n [[-0.6715, -0.9853, -0.8807, ..., -0.9678, -0.6890, 2.3960],\n [-0.7238, -1.0724, -0.9678, ..., -1.2467, -1.0201, 2.3263],\n [-0.8284, -1.1247, -1.0201, ..., -1.2641, -1.1596, 2.3786],\n ...,\n [-1.2293, -1.4733, -1.3861, ..., -1.5081, -1.2641, 2.5180],\n [-1.1944, -1.4559, -1.4210, ..., -1.6476, -1.4733, 2.4308],\n [-1.2293, -1.5256, -1.5081, ..., -1.6824, -1.5256, 2.3611]]])\n\nWas it really useful for you, as a human, to see all these numbers?\n\nWhat is the shape? The size? \nWhat are the statistics? \nAre any of the values `nan` or `inf`? \nIs it an image of a man holding a tench?\n\n``` python\nimport lovely_tensors as lt\n```\n\n``` python\nlt.monkey_patch()\n```\n\n## Summary\n\n``` python\nnumbers # torch.Tensor\n```\n\n tensor[3, 196, 196] n=115248 (0.4Mb) x\u2208[-2.118, 2.640] \u03bc=-0.388 \u03c3=1.073\n\n``` python\nnumbers.rgb\n```\n\n\n\n``` python\nnumbers.plt\n```\n\n\n\nBetter, huh?\n\n``` python\nnumbers[1,:6,1] # Still shows values if there are not too many.\n```\n\n tensor[6] x\u2208[-0.443, -0.197] \u03bc=-0.311 \u03c3=0.091 [-0.197, -0.232, -0.285, -0.373, -0.443, -0.338]\n\n``` python\nspicy = numbers[0,:12,0].clone()\n\nspicy[0] *= 10000\nspicy[1] /= 10000\nspicy[2] = float('inf')\nspicy[3] = float('-inf')\nspicy[4] = float('nan')\n\nspicy = spicy.reshape((2,6))\nspicy # Spicy stuff\n```\n\n tensor[2, 6] n=12 x\u2208[-3.541e+03, -4.054e-05] \u03bc=-393.842 \u03c3=1.180e+03 +Inf! -Inf! NaN!\n\n``` python\ntorch.zeros(10, 10) # A zero tensor - make it obvious\n```\n\n tensor[10, 10] n=100 all_zeros\n\n``` python\nspicy.v # Verbose\n```\n\n tensor[2, 6] n=12 x\u2208[-3.541e+03, -4.054e-05] \u03bc=-393.842 \u03c3=1.180e+03 +Inf! -Inf! NaN!\n tensor([[-3.5405e+03, -4.0543e-05, inf, -inf, nan, -6.1093e-01],\n [-6.1093e-01, -5.9380e-01, -5.9380e-01, -5.4243e-01, -5.4243e-01, -5.4243e-01]])\n\n``` python\nspicy.p # The plain old way\n```\n\n tensor([[-3.5405e+03, -4.0543e-05, inf, -inf, nan, -6.1093e-01],\n [-6.1093e-01, -5.9380e-01, -5.9380e-01, -5.4243e-01, -5.4243e-01, -5.4243e-01]])\n\n## Named dimensions\n\n``` python\nnamed_numbers = numbers.rename(\"C\", \"H\",\"W\")\nnamed_numbers\n```\n\n /home/xl0/mambaforge/envs/lovely-py31-torch25/lib/python3.10/site-packages/torch/_tensor.py:1420: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at ../c10/core/TensorImpl.h:1925.)\n return super().rename(names)\n\n tensor[C=3, H=196, W=196] n=115248 (0.4Mb) x\u2208[-2.118, 2.640] \u03bc=-0.388 \u03c3=1.073\n\n## Going `.deeper`\n\n``` python\nnumbers.deeper\n```\n\n tensor[3, 196, 196] n=115248 (0.4Mb) x\u2208[-2.118, 2.640] \u03bc=-0.388 \u03c3=1.073\n tensor[196, 196] n=38416 x\u2208[-2.118, 2.249] \u03bc=-0.324 \u03c3=1.036\n tensor[196, 196] n=38416 x\u2208[-1.966, 2.429] \u03bc=-0.274 \u03c3=0.973\n tensor[196, 196] n=38416 x\u2208[-1.804, 2.640] \u03bc=-0.567 \u03c3=1.178\n\n``` python\n# You can go deeper if you need to\n# And we can use `.deeper` with named dimensions.\n\nnamed_numbers.deeper(2)\n```\n\n tensor[C=3, H=196, W=196] n=115248 (0.4Mb) x\u2208[-2.118, 2.640] \u03bc=-0.388 \u03c3=1.073\n tensor[H=196, W=196] n=38416 x\u2208[-2.118, 2.249] \u03bc=-0.324 \u03c3=1.036\n tensor[W=196] x\u2208[-1.912, 2.249] \u03bc=-0.673 \u03c3=0.522\n tensor[W=196] x\u2208[-1.861, 2.163] \u03bc=-0.738 \u03c3=0.418\n tensor[W=196] x\u2208[-1.758, 2.198] \u03bc=-0.806 \u03c3=0.397\n tensor[W=196] x\u2208[-1.656, 2.249] \u03bc=-0.849 \u03c3=0.369\n tensor[W=196] x\u2208[-1.673, 2.198] \u03bc=-0.857 \u03c3=0.357\n tensor[W=196] x\u2208[-1.656, 2.146] \u03bc=-0.848 \u03c3=0.372\n tensor[W=196] x\u2208[-1.433, 2.215] \u03bc=-0.784 \u03c3=0.397\n tensor[W=196] x\u2208[-1.279, 2.249] \u03bc=-0.695 \u03c3=0.486\n tensor[W=196] x\u2208[-1.364, 2.249] \u03bc=-0.637 \u03c3=0.539\n ...\n tensor[H=196, W=196] n=38416 x\u2208[-1.966, 2.429] \u03bc=-0.274 \u03c3=0.973\n tensor[W=196] x\u2208[-1.861, 2.411] \u03bc=-0.529 \u03c3=0.556\n tensor[W=196] x\u2208[-1.826, 2.359] \u03bc=-0.562 \u03c3=0.473\n tensor[W=196] x\u2208[-1.756, 2.376] \u03bc=-0.622 \u03c3=0.458\n tensor[W=196] x\u2208[-1.633, 2.429] \u03bc=-0.664 \u03c3=0.430\n tensor[W=196] x\u2208[-1.651, 2.376] \u03bc=-0.669 \u03c3=0.399\n tensor[W=196] x\u2208[-1.633, 2.376] \u03bc=-0.701 \u03c3=0.391\n tensor[W=196] x\u2208[-1.563, 2.429] \u03bc=-0.670 \u03c3=0.380\n tensor[W=196] x\u2208[-1.475, 2.429] \u03bc=-0.616 \u03c3=0.386\n tensor[W=196] x\u2208[-1.511, 2.429] \u03bc=-0.593 \u03c3=0.399\n ...\n tensor[H=196, W=196] n=38416 x\u2208[-1.804, 2.640] \u03bc=-0.567 \u03c3=1.178\n tensor[W=196] x\u2208[-1.717, 2.396] \u03bc=-0.982 \u03c3=0.350\n tensor[W=196] x\u2208[-1.752, 2.326] \u03bc=-1.034 \u03c3=0.314\n tensor[W=196] x\u2208[-1.648, 2.379] \u03bc=-1.086 \u03c3=0.314\n tensor[W=196] x\u2208[-1.630, 2.466] \u03bc=-1.121 \u03c3=0.305\n tensor[W=196] x\u2208[-1.717, 2.448] \u03bc=-1.120 \u03c3=0.302\n tensor[W=196] x\u2208[-1.717, 2.431] \u03bc=-1.166 \u03c3=0.314\n tensor[W=196] x\u2208[-1.560, 2.448] \u03bc=-1.124 \u03c3=0.326\n tensor[W=196] x\u2208[-1.421, 2.431] \u03bc=-1.064 \u03c3=0.383\n tensor[W=196] x\u2208[-1.526, 2.396] \u03bc=-1.047 \u03c3=0.417\n ...\n\n## Now in `.rgb` color\n\nThe important queston - is it our man?\n\n``` python\nnumbers.rgb\n```\n\n\n\n*Maaaaybe?* Looks like someone normalized him.\n\n``` python\nin_stats = ( (0.485, 0.456, 0.406), # mean\n (0.229, 0.224, 0.225) ) # std\n\n# numbers.rgb(in_stats, cl=True) # For channel-last input format\nnumbers.rgb(in_stats)\n```\n\n\n\nIt\u2019s indeed our hero, the Tenchman!\n\n## `.plt` the statistics\n\n``` python\n(numbers+3).plt(center=\"mean\", max_s=1000)\n```\n\n\n\n``` python\n(numbers).plt\n```\n\n\n\n``` python\n(numbers+3).plt(center=\"range\")\n```\n\n\n\n## See the `.chans`\n\n``` python\n# .chans will map values betwen [-1,1] to colors.\n# Make our values fit into that range to avoid clipping.\nmean = torch.tensor(in_stats[0])[:,None,None]\nstd = torch.tensor(in_stats[1])[:,None,None]\nnumbers_01 = (numbers*std + mean)\nnumbers_01\n```\n\n tensor[3, 196, 196] n=115248 (0.4Mb) x\u2208[0., 1.000] \u03bc=0.361 \u03c3=0.248\n\n``` python\nnumbers_01.chans\n```\n\n\n\nLet\u2019s try with a Convolutional Neural Network\n\n``` python\nfrom torchvision.models import vgg11\n```\n\n``` python\nfeatures: torch.nn.Sequential = vgg11().features\n\n# I saved the first 5 layers in \"features.pt\"\n_ = features.load_state_dict(torch.load(\"../features.pt\", weights_only=True), strict=False)\n```\n\n``` python\n# Activatons of the second max pool layer of VGG11\nacts = (features[:6](numbers[None])[0]/2) # /2 to reduce clipping\nacts\n```\n\n tensor[128, 49, 49] n=307328 (1.2Mb) x\u2208[0., 12.508] \u03bc=0.367 \u03c3=0.634 grad DivBackward0\n\n``` python\nacts[:4].chans(cmap=\"coolwarm\", scale=4)\n```\n\n\n\n## Grouping\n\n``` python\n# Make 8 images with progressively higher brightness and stack them 2x2x2.\neight_images = (torch.stack([numbers]*8)\n .add(torch.linspace(-3, 3, 8)[:,None,None,None])\n .mul(torch.tensor(in_stats[1])[:,None,None])\n .add(torch.tensor(in_stats[0])[:,None,None])\n .clamp(0,1)\n .view(2,2,2,3,196,196)\n)\neight_images\n```\n\n tensor[2, 2, 2, 3, 196, 196] n=921984 (3.5Mb) x\u2208[0., 1.000] \u03bc=0.411 \u03c3=0.369\n\n``` python\neight_images.rgb\n```\n\n\n\n``` python\n# Weights of the second conv layer of VGG11\nfeatures[3].weight\n```\n\n Parameter[128, 64, 3, 3] n=73728 (0.3Mb) x\u2208[-0.783, 0.776] \u03bc=-0.004 \u03c3=0.065 grad\n\nI want +/- 2\u03c3 to fall in the range \\[-1..1\\]\n\n``` python\nweights = features[3].weight.data\nweights = weights / (2*2*weights.std()) # *2 because we want 2\u03c3 on both sides, so 4\u03c3\n# weights += weights.std() * 2\nweights.plt\n```\n\n\n\n``` python\n# Weights of the second conv layer (64ch -> 128ch) of VGG11,\n# grouped per output channel.\nweights.chans(frame_px=1, gutter_px=0)\n```\n\n\n\nIt\u2019s a bit hard to see. Scale up 10x, but onyl show the first 4 filters.\n\n``` python\nweights[:4].chans(frame_px=1, gutter_px=0, scale=10)\n```\n\n\n\n## Options \\| [Docs](https://xl0.github.io/lovely-tensors/utils.config.html)\n\n``` python\nfrom lovely_tensors import set_config, config, lovely, get_config\n```\n\n``` python\nset_config(precision=1, sci_mode=True, color=False)\ntorch.tensor([1, 2, torch.nan])\n```\n\n tensor[3] \u03bc=1.5e+00 \u03c3=7.1e-01 NaN! [1.0e+00, 2.0e+00, nan]\n\n``` python\nset_config(precision=None, sci_mode=None, color=None) # None -> Reset to defaults\n```\n\n``` python\nprint(torch.tensor([1., 2]))\n# Or with config context manager.\nwith config(sci_mode=True, precision=5):\n print(torch.tensor([1., 2]))\n\nprint(torch.tensor([1., 2]))\n```\n\n tensor[2] \u03bc=1.500 \u03c3=0.707 [1.000, 2.000]\n tensor[2] \u03bc=1.50000e+00 \u03c3=7.07107e-01 [1.00000e+00, 2.00000e+00]\n tensor[2] \u03bc=1.500 \u03c3=0.707 [1.000, 2.000]\n\n## Without `.monkey_patch`\n\n``` python\nlt.lovely(spicy)\n```\n\n tensor[2, 6] n=12 x\u2208[-3.541e+03, -4.054e-05] \u03bc=-393.842 \u03c3=1.180e+03 +Inf! -Inf! NaN!\n\n``` python\nlt.lovely(spicy, verbose=True)\n```\n\n tensor[2, 6] n=12 x\u2208[-3.541e+03, -4.054e-05] \u03bc=-393.842 \u03c3=1.180e+03 +Inf! -Inf! NaN!\n tensor([[-3.5405e+03, -4.0543e-05, inf, -inf, nan, -6.1093e-01],\n [-6.1093e-01, -5.9380e-01, -5.9380e-01, -5.4243e-01, -5.4243e-01, -5.4243e-01]])\n\n``` python\nlt.lovely(numbers, depth=1)\n```\n\n tensor[3, 196, 196] n=115248 (0.4Mb) x\u2208[-2.118, 2.640] \u03bc=-0.388 \u03c3=1.073\n tensor[196, 196] n=38416 x\u2208[-2.118, 2.249] \u03bc=-0.324 \u03c3=1.036\n tensor[196, 196] n=38416 x\u2208[-1.966, 2.429] \u03bc=-0.274 \u03c3=0.973\n tensor[196, 196] n=38416 x\u2208[-1.804, 2.640] \u03bc=-0.567 \u03c3=1.178\n\n``` python\nlt.rgb(numbers, in_stats)\n```\n\n\n\n``` python\nlt.plot(numbers, center=\"mean\")\n```\n\n\n\n``` python\nlt.chans(numbers_01)\n```\n\n\n\n## Matplotlib integration \\| [Docs](https://xl0.github.io/lovely-tensors/matplotlib.html)\n\n``` python\nnumbers.rgb(in_stats).fig # matplotlib figure\n```\n\n\n\n``` python\n(numbers*0.3+0.5).chans.fig # matplotlib figure\n```\n\n\n\n``` python\nnumbers.plt.fig.savefig('pretty.svg') # Save it\n```\n\n``` python\n!file pretty.svg; rm pretty.svg\n```\n\n pretty.svg: SVG Scalable Vector Graphics image\n\n### Add content to existing Axes\n\n``` python\nfig = plt.figure(figsize=(8,3))\nfig.set_constrained_layout(True)\ngs = fig.add_gridspec(2,2)\nax1 = fig.add_subplot(gs[0, :])\nax2 = fig.add_subplot(gs[1, 0])\nax3 = fig.add_subplot(gs[1,1:])\n\nax2.set_axis_off()\nax3.set_axis_off()\n\nnumbers_01.plt(ax=ax1)\nnumbers_01.rgb(ax=ax2)\nnumbers_01.chans(ax=ax3);\n```\n\n\n\n## torch.compile()\n\nJust works.\n\n``` python\ndef func(x):\n return x*2\n\nif torch.__version__ >= \"2.0\":\n func = torch.compile(func)\n\nfunc(torch.tensor([1,2,3]))\n```\n\n tensor[3] i64 x\u2208[2, 6] \u03bc=4.000 \u03c3=2.000 [2, 4, 6]\n\n## Inport hook\n\nLovely tensors installes an import hook. Set `LOVELY_TENSORS=1`, and it\nwill load automatically, no need to modify the code: \\> Note: Don\u2019t set\nit globally, or all python scripts you run will import LT and PyTorch,\nwhich will slow things down.\n\n``` python\nimport torch\n\nx = torch.randn(4, 16)\nprint(x)\n```\n\n``` bash\nLOVELY_TENSORS=1 python test.py\n```\n\n x: tensor[4, 16] n=64 x\u2208[-1.652, 1.813] \u03bc=-0.069 \u03c3=0.844\n\nThis is especially useful in combination with [Better\nExceptions](https://github.com/Qix-/better-exceptions):\n\n``` python\nimport torch\n\nx = torch.randn(4, 16)\nprint(f\"x: {x}\")\n\nw = torch.randn(15, 8) \ny = torch.matmul(x, w) # Dimension mismatch\n```\n\n``` bash\nBETTER_EXCEPTIONS=1 LOVELY_TENSORS=1 python test.py \n```\n\n x: tensor[4, 16] n=64 x\u2208[-1.834, 2.421] \u03bc=0.103 \u03c3=0.896\n Traceback (most recent call last):\n File \"/home/xl0/work/projects/lovely-tensors/test.py\", line 7, in <module>\n y = torch.matmul(x, w)\n \u2502 \u2502 \u2514 tensor[15, 8] n=120 x\u2208[-2.355, 2.165] \u03bc=0.142 \u03c3=0.989\n \u2502 \u2514 tensor[4, 16] n=64 x\u2208[-1.834, 2.421] \u03bc=0.103 \u03c3=0.896\n \u2514 <module 'torch' from '/home/xl0/mambaforge/envs/torch25-py313/lib/python3.12/site-packages/torch/__init__.py'>\n RuntimeError: mat1 and mat2 shapes cannot be multiplied (4x16 and 15x8)\n",
"bugtrack_url": null,
"license": "MIT License",
"summary": "\u2764\ufe0f Lovely Tensors",
"version": "0.1.18",
"project_urls": {
"Homepage": "https://github.com/xl0/lovely-tensors"
},
"split_keywords": [
"jupyter",
"pytorch",
"tensor",
"visualisation"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "71fb688604e72694c597cf236e5ccd9a4bcc34bacc5f8e5dbb2cf7d7ddccb25b",
"md5": "fa760b310751a49d428d67187ee6626f",
"sha256": "91dc30f0d6224364851e6f14497e1677076cd3a59bae4d92c78bbe8684f6f22b"
},
"downloads": -1,
"filename": "lovely_tensors-0.1.18-py3-none-any.whl",
"has_sig": false,
"md5_digest": "fa760b310751a49d428d67187ee6626f",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.7",
"size": 19303,
"upload_time": "2024-11-20T22:00:27",
"upload_time_iso_8601": "2024-11-20T22:00:27.147609Z",
"url": "https://files.pythonhosted.org/packages/71/fb/688604e72694c597cf236e5ccd9a4bcc34bacc5f8e5dbb2cf7d7ddccb25b/lovely_tensors-0.1.18-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "20c4b109b6c912929a471cf03c6a2f02d8c5ab2889ec57177692d7b971c4ec70",
"md5": "8964d652da524c6bf1729a1bff18c443",
"sha256": "afb07d52de9ec6e560e77d9b3e01758bbc921a6f25c661b1c51ebbb3f75ee0c5"
},
"downloads": -1,
"filename": "lovely_tensors-0.1.18.tar.gz",
"has_sig": false,
"md5_digest": "8964d652da524c6bf1729a1bff18c443",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.7",
"size": 21962,
"upload_time": "2024-11-20T22:00:29",
"upload_time_iso_8601": "2024-11-20T22:00:29.209765Z",
"url": "https://files.pythonhosted.org/packages/20/c4/b109b6c912929a471cf03c6a2f02d8c5ab2889ec57177692d7b971c4ec70/lovely_tensors-0.1.18.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-20 22:00:29",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "xl0",
"github_project": "lovely-tensors",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "lovely-tensors"
}
JSON
