|
|
[Aim](https://aimstack.io/#aim) is an interactive tool that helps to visualize and compare several metrics (losses, learning rates, etc.) between different runs.
|
|
|
|
|
|

|
|
|
|
|
|
The main goal of Aim is to help developers during different iterations of model structure or data processing.
|
|
|
|
|
|
## How it works
|
|
|
Aim is a very powerful tool but at the same time is very simple to use. It defines three main interfaces:
|
|
|
|
|
|
### Session
|
|
|
Session class is the core of Aim. During the creation of a session, it is possible to define an experiment name and/or a run name: experiments are useful to group together runs related to the same model.
|
|
|
|
|
|
Inside this template, experiment and run names are handled by the [config file](https://gitlab.fbk.eu/dsip/templates/dl_setup/-/wikis/Configuration-file) and run script helper.
|
|
|
```python
|
|
|
from aim import Session
|
|
|
|
|
|
aim_session = Session(
|
|
|
repo='/aim/repo',
|
|
|
experiment='experiment_name',
|
|
|
run='run_name',
|
|
|
)
|
|
|
```
|
|
|
|
|
|
Example:
|
|
|
|
|
|

|
|
|
|
|
|
The red arrow points to a "filter by experiment", and the green arrow points to runs related to the selected experiment.
|
|
|
|
|
|
### set_params
|
|
|
It is a session method used to set hyper-parameters associated with the run.
|
|
|
|
|
|
Inside dl_setup, set_params is called automatically: all configs and hyper-paramters related to the run are saved. This approach allows to match a specific run (from [config file](https://gitlab.fbk.eu/dsip/templates/dl_setup/-/wikis/Configuration-file)) to related results (losses tracked from aim for example).
|
|
|
```python
|
|
|
aim_session.set_params(
|
|
|
{ 'lr': 0.01, 'optimizer': 'Adam', 'batch_size': 64 },
|
|
|
name="hparams"
|
|
|
)
|
|
|
```
|
|
|
|
|
|
Example:
|
|
|
|
|
|

|
|
|
|
|
|
### track
|
|
|
It is a session method used to track a specific metric over time.
|
|
|
|
|
|
```python
|
|
|
aim_session.track(loss_value, name="loss",
|
|
|
epoch=epoch_number, subset='train_phase')
|
|
|
```
|
|
|
|
|
|
It accepts any argument defined in the form *keyword=value*.
|
|
|
|
|
|
Inside dl_setup, train and validation losses are already being tracked. If the user wants to add another metric (accuracy for example), they should call the `track` method of the session instance.
|
|
|
|
|
|
Example:
|
|
|
|
|
|

|
|
|
|
|
|
The X-axis represents epochs.
|
|
|
The Y-axis represents loss value.
|
|
|
The red line is validation loss.
|
|
|
The green line is train loss.
|
|
|
|
|
|
### Aim on cluster
|
|
|
At the moment there is no option to be able to visualize on a dashboard the aim logs that are stored in the cluster. We are currently developing a feature that enables aim to send the tracked data to an exposed online webserver. Additional info will be provided as soon as possible.
|
|
|
|
|
|
For the moment, in order to visualize aim logs, it is necessary to copy the `.aim` folder on your local machine. |