|
|
|
This project template aims to reduce all necessary infrastructure operations during a project initialization. The main idea is to have a simple generic project generator that helps developers focus only on their specific task (in this case one or more DL models) without worrying about building up the infrastructure.
|
|
|
|
The main philosophy behind this project template is reproducibility. This is accomplished using a standardized structure: everyone that has access to the same data, code, and configuration for the same experiment, can also replicate it.
|
|
|
|
#### DL setup
|
|
|
|
|
|
|
|
## How to use
|
|
|
|
This repo is a template and can be used as a starting point for another repo. Some steps are required after copying this repo, they can be found at this [link](How-to-use)
|
|
|
|
- [home](Home-dl_setup)
|
|
|
|
- [how to use](How-to-use)
|
|
|
|
- features
|
|
|
|
- [docker](Docker)
|
|
|
|
- [pytorch](Pytorch)
|
|
|
|
- [agile](Agile)
|
|
|
|
- [aim](Aim)
|
|
|
|
- [configuration file](Configuration-file)
|
|
|
|
- [runner script](Runner-script)
|
|
|
|
|
|
|
|
## Features
|
|
|
|
|
|
|
|
### Docker
|
|
|
|
Docker provides isolated containers in which dependency problems are avoided. It is also a helpful way to build a potentially deployable environment with only the necessary packages required by a project.
|
|
|
|
|
|
|
|
More guidelines about Docker and its usage related to this template can be found at this [link](Docker).
|
|
|
|
|
|
|
|
### PyTorch
|
|
|
|
PyTorch is the main library adopted by the DSIP group to build deep learning models. There are some similar components across the projects:
|
|
|
|
|
|
|
|
* Dataset
|
|
|
|
* Dataloader
|
|
|
|
* Optimizer
|
|
|
|
* Scheduler
|
|
|
|
* Early stopper
|
|
|
|
* ...
|
|
|
|
|
|
|
|
For this reason, the main skeleton, providing more standard and generic pieces of code, is included inside this template. More guidelines about this skeleton and how to use it can be found at this link [link](Pytorch).
|
|
|
|
|
|
|
|
### Agile approach
|
|
|
|
Agile approaches are used inside the industry to keep the code organized and well documented.
|
|
|
|
Agile also helps to avoid bugs through multiple test stages. It is important to underline that this feature is the only one that is not transparent to the developer. A good agile approach requires the user to think in a different way about project development.
|
|
|
|
|
|
|
|
More guidelines about the agile approach adopted can be found at this [link](Agile).
|
|
|
|
|
|
|
|
### Aim
|
|
|
|
Aim helps visualize and compare several DL experiments.
|
|
|
|
|
|
|
|
More guidelines about aim can be found at this [link](Aim).
|
|
|
|
|
|
|
|
### Configuration file
|
|
|
|
All parameters variable over time can be set using a configuration file to have all of them accessible at a single point.
|
|
|
|
|
|
|
|
More guidelines about the configuration file format can be found at this [link](Configuration-file).
|
|
|
|
|
|
|
|
### Runner script
|
|
|
|
The management of all these features can become easily overwhelming. For this reason inside the repo, there is a script on the root folder called *run*. It can be used to run any features explained above.
|
|
|
|
|
|
|
|
More guidelines about the configuration file format can be found at this [link](Configuration-file).
|
|
|
|
|
|
|
|
## Workflow example
|
|
|
|
|
|
|
|
## Proposals |
|
|
|
#### Other
|
|
|
|
- [move azure disks between regions](Move-azure-disks-between-regions) |
