... | ... | @@ -27,21 +27,21 @@ mkdir -p /path/to/my/dataset |
|
|
## udocker no able to build
|
|
|
Inside the cluster there is no possibility to build a model or misc container with `./run build_model` or `./run build_misc`.
|
|
|
|
|
|
The are two ways to fix this problem:
|
|
|
There are two ways to fix this problem:
|
|
|
|
|
|
1. gitlab:
|
|
|
1. push code to gitlab server;
|
|
|
1. GitLab:
|
|
|
1. push code to GitLab server;
|
|
|
2. automatic building is triggered on push;
|
|
|
3. after some minutes containers will be available on container registry and will be ablet to be fetched with `./run pull_model` and `./run pull_misc`.
|
|
|
2. code as volume: if you need to modify many times your code and you do not want to wait gitlab builds you can use `--development` flag in `./run run_model` and `./run run_misc`. In this way code will be accessible inside the container.
|
|
|
3. after some minutes containers will be available on the container registry and will be able to be fetched with `./run pull_model` and `./run pull_misc`.
|
|
|
2. code as volume: if you need to modify many times your code and you do not want to wait GitLab builds you can use `--development` flag in `./run run_model` and `./run run_misc`. In this way, code will be accessible inside the container.
|
|
|
|
|
|
## pull from gpu node
|
|
|
If you want nvidia gpu support inside the container be sure to call `./run pull_model` and `./run pull_misc` from a cluster node with gpu.
|
|
|
## pull from GPU node
|
|
|
If you want Nvidia GPU support inside the container be sure to call `./run pull_model` and `./run pull_misc` from a cluster node with GPU.
|
|
|
|
|
|
## Port forwarding
|
|
|
In order to be able to have ssh port forwarding follow this guidelines.
|
|
|
In order to be able to have ssh port forwarding follow this guideline.
|
|
|
|
|
|
Disclaimer: this approach assume usage of [jump host](https://gitlab.fbk.eu/di-center/di-club/di-club-technical/-/wikis/A.-intro/1.-DICLUB-and-slurm-intro) technique. For VPN approach this was not tested.
|
|
|
Disclaimer: this approach assumes usage of [jump host](https://gitlab.fbk.eu/di-center/di-club/di-club-technical/-/wikis/A.-intro/1.-DICLUB-and-slurm-intro) technique. For the VPN approach this was not tested.
|
|
|
|
|
|
#### Update ssh config
|
|
|
Update ssh config (usually inside ~/.ssh/config) adding this line:
|
... | ... | @@ -59,12 +59,12 @@ Host diclub |
|
|
LocalForward 65432 localhost:65432
|
|
|
```
|
|
|
|
|
|
Port number 65432 is not mandatory, it is possible to specify any number with the shrewdness of use the same value in steps below
|
|
|
Port number 65432 is not mandatory, it is possible to specify any number with the shrewdness of use the same value in the steps below
|
|
|
|
|
|
#### Get a node
|
|
|
It is possible to use this command to reserve a node inside the cluster:
|
|
|
```bash
|
|
|
srun --gres=gpu:1 -p gpu-K80 --pty bash -i # for a node with a gpu
|
|
|
srun --gres=gpu:1 -p gpu-K80 --pty bash -i # for a node with a GPU
|
|
|
```
|
|
|
|
|
|
#### Identify the node number
|
... | ... | @@ -72,10 +72,10 @@ At this point with |
|
|
```bash
|
|
|
squeue -u $USER
|
|
|
```
|
|
|
user need to take note of node number, for example *node9*, *node11*, etc.
|
|
|
user needs to take note of node number, for example, *node9*, *node11*, etc.
|
|
|
|
|
|
#### Start process
|
|
|
Start process that needs port forwarding. An example may be jupyter notebook:
|
|
|
Start the process that needs port forwarding. An example may be Jupiter notebook:
|
|
|
```bash
|
|
|
d your-project
|
|
|
./run run_misc --notebook --port 65432 --config path_to_config/config.ini
|
... | ... | |