|
|
## udocker setup
|
|
|
On cluster there is no possibility to use plain docker. An alternative to it that allows container usage is udocker. It has different command arguments from command-line interface, for this reason there are two different sections inside [configuration file](configuration-file), one for docker and another for udocker.
|
|
|
On the cluster, there is no possibility to use plain docker. An alternative to it that allows container usage is udocker. It has different command arguments from the command-line interface, for this reason, there are two different sections inside [configuration file](configuration-file), one for docker and another for udocker.
|
|
|
|
|
|
There is a snippet that can be used to set up a udocker environment on cluster:
|
|
|
There is a snippet that can be used to set up a udocker environment on the cluster:
|
|
|
|
|
|
1. go to this [link](https://gitlab.fbk.eu/dsip/templates/dl_setup/-/snippets/65)
|
|
|
2. copy it in a file `udocker_setup.sh` inside your cluster home
|
|
|
3. run `chmod +x udocker_setup.sh`
|
|
|
4. run `./udocker_setup.sh`
|
|
|
5. run `rm udocker_setup.sh`
|
|
|
6. exit and enter again inside cluster
|
|
|
6. exit and enter again inside the cluster
|
|
|
7. [login](https://indigo-dc.gitbook.io/udocker/user_manual#3-23-login) on the GitLab Package Registry (gitlab-registry.fbk.eu) with udocker
|
|
|
|
|
|
## udocker volume directories
|
|
|
Normally, when you pass a volume to docker, it is able to create missing directories related to the passed path. Udocker can not do this: if you write a path inside the [configuration file](configuration-file) and it is not present on disk an exception is raised in which is tell to the user to create all missing directories.
|
|
|
Normally, when you pass a volume to docker, it is able to create missing directories related to the passed path. Udocker can not do this: if you write a path inside the [configuration file](configuration-file) and it is not present on disk an exception is raised.
|
|
|
|
|
|
Example:
|
|
|
```ini
|
... | ... | @@ -19,13 +20,13 @@ Example: |
|
|
[paths]
|
|
|
dataset = /path/to/my/dataset
|
|
|
```
|
|
|
with dataset folder inside /path/to/my does not exist. An error is raised and user can solve it running:
|
|
|
with dataset folder inside /path/to/my does not exist. An error is raised and the user can solve it running:
|
|
|
```bash
|
|
|
mkdir -p /path/to/my/dataset
|
|
|
```
|
|
|
|
|
|
## udocker no able to build
|
|
|
Inside the cluster there is no possibility to build a model or misc container with `./run build_model` or `./run build_misc`.
|
|
|
Inside the cluster, there is no possibility to build a model or misc container with `./run build_model` or `./run build_misc`.
|
|
|
|
|
|
There are two ways to fix this problem:
|
|
|
|
... | ... | @@ -33,7 +34,7 @@ There are two ways to fix this problem: |
|
|
1. push code to GitLab server;
|
|
|
2. automatic building is triggered on push;
|
|
|
3. after some minutes containers will be available on the container registry and will be able to be fetched with `./run pull_model` and `./run pull_misc`.
|
|
|
2. code as volume: if you need to modify many times your code and you do not want to wait GitLab builds you can use `--development` flag in `./run run_model` and `./run run_misc`. In this way, code will be accessible inside the container.
|
|
|
2. code as volume: if you need to modify many times your code and you do not want to wait for GitLab builds, you can use the `--development` flag in `./run run_model` and `./run run_misc`. In this way, code will be accessible inside the container.
|
|
|
|
|
|
## pull from GPU node
|
|
|
If you want Nvidia GPU support inside the container be sure to call `./run pull_model` and `./run pull_misc` from a cluster node with GPU.
|
... | ... | @@ -41,7 +42,7 @@ If you want Nvidia GPU support inside the container be sure to call `./run pull_ |
|
|
## Port forwarding
|
|
|
In order to be able to have ssh port forwarding follow this guideline.
|
|
|
|
|
|
Disclaimer: this approach assumes usage of [jump host](https://gitlab.fbk.eu/di-center/di-club/di-club-technical/-/wikis/A.-intro/1.-DICLUB-and-slurm-intro) technique. For the VPN approach this was not tested.
|
|
|
Disclaimer: this approach assumes usage of [jump host](https://gitlab.fbk.eu/di-center/di-club/di-club-technical/-/wikis/A.-intro/1.-DICLUB-and-slurm-intro) technique. For the VPN approach, this was not tested.
|
|
|
|
|
|
#### Update ssh config
|
|
|
Update ssh config (usually inside ~/.ssh/config) adding this line:
|
... | ... | |