Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
  • Sign in
  • D DSIP
  • Group information
    • Group information
    • Activity
    • Labels
    • Members
  • Epics 2
    • Epics 2
    • List
    • Boards
    • Roadmap
  • Issues 10
    • Issues 10
    • List
    • Boards
    • Milestones
    • Iterations
  • Merge requests 0
    • Merge requests 0
  • Security & Compliance
    • Security & Compliance
    • Security dashboard
    • Vulnerability report
  • Packages and registries
    • Packages and registries
    • Package Registry
  • Analytics
    • Analytics
    • Insights
    • Issue
  • Wiki
    • Wiki
Collapse sidebar
  • DSIP
  • Wiki
  • Home dl_setup

Home dl_setup · Changes

Page history
Update home authored Jun 04, 2021 by Federico Izzo's avatar Federico Izzo
Show whitespace changes
Inline Side-by-side
Home-dl_setup.md 0 → 100644
View page @ e72cea3f
This project template aims to reduce all necessary infrastructure operations during a project initialization. The main idea is to have a simple generic project generator that helps developers focus only on their specific task (in this case one or more DL models) without worrying about building up the infrastructure.
The main philosophy behind this project template is reproducibility. This is accomplished using a standardized structure: everyone that has access to the same data, code, and configuration for the same experiment, can also replicate it.
## How to use
This repo is a template and can be used as a starting point for another repo. Some steps are required after copying this repo, they can be found at this [link](How-to-use)
## Features
### Docker
Docker provides isolated containers in which dependency problems are avoided. It is also a helpful way to build a potentially deployable environment with only the necessary packages required by a project.
More guidelines about Docker and its usage related to this template can be found at this [link](Docker).
### PyTorch
PyTorch is the main library adopted by the DSIP group to build deep learning models. There are some similar components across the projects:
* Dataset
* Dataloader
* Optimizer
* Scheduler
* Early stopper
* ...
For this reason, the main skeleton, providing more standard and generic pieces of code, is included inside this template. More guidelines about this skeleton and how to use it can be found at this link [link](Pytorch).
### Agile approach
Agile approaches are used inside the industry to keep the code organized and well documented.
Agile also helps to avoid bugs through multiple test stages. It is important to underline that this feature is the only one that is not transparent to the developer. A good agile approach requires the user to think in a different way about project development.
More guidelines about the agile approach adopted can be found at this [link](Agile).
### Aim
Aim helps visualize and compare several DL experiments.
More guidelines about aim can be found at this [link](Aim).
### Configuration file
All parameters variable over time can be set using a configuration file to have all of them accessible at a single point.
More guidelines about the configuration file format can be found at this [link](Configuration-file).
### Runner script
The management of all these features can become easily overwhelming. For this reason inside the repo, there is a script on the root folder called *run*. It can be used to run any features explained above.
More guidelines about the configuration file format can be found at this [link](Configuration-file).
## Workflow example
## Proposals
Clone repository

DL setup

  • home
  • how to use
  • features
    • docker
    • pytorch
    • agile
    • aim
    • configuration file
    • runner script

Other

  • move azure disks between regions