About Fidle

This repository contains all the documents and links of the Fidle Training .
Fidle (for Formation Introduction au Deep Learning) is a 3-day training session co-organized
by the 3IA MIAI institute, the CNRS, via the Mission for Transversal and Interdisciplinary
Initiatives (MITI) and the University of Grenoble Alpes (UGA).

The objectives of this training are :

• Understanding the bases of Deep Learning neural networks
• Develop a first experience through simple and representative examples
• Understanding Tensorflow/Keras and Jupyter lab technologies
• Apprehend the academic computing environments Tier-2 or Tier-1 with powerfull GPU

For more information, see https://fidle.cnrs.fr :

• Fidle site
• Presentation of the training
• Detailed program
• Subscribe to the list, to stay informed !
• Find us on youtube
• Corrected notebooks

For more information, you can contact us at :

Current Version : 2.5.4

Course materials

Course slides The course in pdf format	Notebooks     Get a Zip or clone this repository	Datasets All the needed datasets	Videos     Our Youtube channel

Have a look about How to get and install these notebooks and datasets.

Jupyter notebooks

NOTE : The examples marked "obsolete" are still functional under Keras2/Tensorflow, but cannot be run in the proposed environment, now based on Keras3, PyTorch and Lightning.
We have decided to consider Keras2/Tensorflow as pedagogically obsolete, although Keras2 and Tensorflow are still perfectly usable (January 2024).
For these reason, they are kept as examples, while we develop the Keras3/PyTorch versions.
The world of Deep Learning is changing very fast !

Linear and logistic regression

• LINR1 - Linear regression with direct resolution
  Low-level implementation, using numpy, of a direct resolution for a linear regression
• GRAD1 - Linear regression with gradient descent
  Low level implementation of a solution by gradient descent. Basic and stochastic approach.
• POLR1 - Complexity Syndrome
  Illustration of the problem of complexity with the polynomial regression
• LOGR1 - Logistic regression
  Simple example of logistic regression with a sklearn solution

Perceptron Model 1957

• PER57 - Perceptron Model 1957
  Example of use of a Perceptron, with sklearn and IRIS dataset of 1936 !

BHPD regression (DNN), using Keras3/PyTorch

• K3BHPD1 - Regression with a Dense Network (DNN)
  Simple example of a regression with the dataset Boston Housing Prices Dataset (BHPD)
• K3BHPD2 - Regression with a Dense Network (DNN) - Advanced code
  A more advanced implementation of the precedent example, using Keras3

BHPD regression (DNN), using PyTorch

• PBHPD1 - Regression with a Dense Network (DNN)
  A Simple regression with a Dense Neural Network (DNN) using Pytorch - BHPD dataset

Wine Quality prediction (DNN), using Keras3/PyTorch

• K3WINE1 - Wine quality prediction with a Dense Network (DNN)
  Another example of regression, with a wine quality prediction, using Keras 3 and PyTorch

Wine Quality prediction (DNN), using PyTorch/Lightning

• LWINE1 - Wine quality prediction with a Dense Network (DNN)
  Another example of regression, with a wine quality prediction, using PyTorch Lightning

MNIST classification (DNN,CNN), using Keras3/PyTorch

• K3MNIST1 - Simple classification with DNN
  An example of classification using a dense neural network for the famous MNIST dataset
• K3MNIST2 - Simple classification with CNN
  An example of classification using a convolutional neural network for the famous MNIST dataset

MNIST classification (DNN,CNN), using PyTorch

• PMNIST1 - Simple classification with DNN
  Example of classification with a fully connected neural network, using Pytorch

MNIST classification (DNN,CNN), using PyTorch/Lightning

• LMNIST2 - Simple classification with DNN
  An example of classification using a dense neural network for the famous MNIST dataset, using PyTorch Lightning
• LMNIST2 - Simple classification with CNN
  An example of classification using a convolutional neural network for the famous MNIST dataset, using PyTorch Lightning

Images classification GTSRB with Convolutional Neural Networks (CNN), using Keras3/PyTorch

• K3GTSRB1 - Dataset analysis and preparation
  Episode 1 : Analysis of the GTSRB dataset and creation of an enhanced dataset
• K3GTSRB2 - First convolutions
  Episode 2 : First convolutions and first classification of our traffic signs, using Keras3
• K3GTSRB3 - Training monitoring
  Episode 3 : Monitoring, analysis and check points during a training session, using Keras3
• K3GTSRB4 - Hight level example (Keras-cv)
  An example of using a pre-trained model with Keras-cv
• K3GTSRB10 - OAR batch script submission
  Bash script for an OAR batch submission of an ipython code
• K3GTSRB11 - SLURM batch script
  Bash script for a Slurm batch submission of an ipython code

Sentiment analysis with word embedding, using Keras3/PyTorch

• K3IMDB1 - Sentiment analysis with hot-one encoding
  A basic example of sentiment analysis with sparse encoding, using a dataset from Internet Movie Database (IMDB), using Keras 3 on PyTorch
• K3IMDB2 - Sentiment analysis with text embedding
  A very classical example of word embedding with a dataset from Internet Movie Database (IMDB), using Keras 3 on PyTorch
• K3IMDB3 - Reload and reuse a saved model
  Retrieving a saved model to perform a sentiment analysis (movie review), using Keras 3 and PyTorch
• K3IMDB4 - Reload embedded vectors
  Retrieving embedded vectors from our trained model, using Keras 3 and PyTorch
• K3IMDB5 - Sentiment analysis with a RNN network
  Still the same problem, but with a network combining embedding and RNN, using Keras 3 and PyTorch

Time series with Recurrent Neural Network (RNN), using Keras3/PyTorch

• K3LADYB1 - Prediction of a 2D trajectory via RNN
  Artificial dataset generation and prediction attempt via a recurrent network, using Keras 3 and PyTorch

Sentiment analysis with transformer, using PyTorch

• TRANS1 - IMDB, Sentiment analysis with Transformers
  Using a Tranformer to perform a sentiment analysis (IMDB) - Jean Zay version
• TRANS2 - IMDB, Sentiment analysis with Transformers
  Using a Tranformer to perform a sentiment analysis (IMDB) - Colab version

Unsupervised learning with an autoencoder neural network (AE), using Keras2 (obsolete)

• K2AE1 - Prepare a noisy MNIST dataset
  Episode 1: Preparation of a noisy MNIST dataset, using Keras 2 and Tensorflow (obsolete)
• K2AE2 - Building and training an AE denoiser model
  Episode 1 : Construction of a denoising autoencoder and training of it with a noisy MNIST dataset, using Keras 2 and Tensorflow (obsolete)
• K2AE3 - Playing with our denoiser model
  Episode 2 : Using the previously trained autoencoder to denoise data, using Keras 2 and Tensorflow (obsolete)
• K2AE4 - Denoiser and classifier model
  Episode 4 : Construction of a denoiser and classifier model, using Keras 2 and Tensorflow (obsolete)
• K2AE5 - Advanced denoiser and classifier model
  Episode 5 : Construction of an advanced denoiser and classifier model, using Keras 2 and Tensorflow (obsolete)

Generative network with Variational Autoencoder (VAE), using Keras2 (obsolete)

• K2VAE1 - First VAE, using functional API (MNIST dataset)
  Construction and training of a VAE, using functional APPI, with a latent space of small dimension, using Keras 2 and Tensorflow (obsolete)
• K2VAE2 - VAE, using a custom model class (MNIST dataset)
  Construction and training of a VAE, using model subclass, with a latent space of small dimension, using Keras 2 and Tensorflow (obsolete)
• K2VAE3 - Analysis of the VAE's latent space of MNIST dataset
  Visualization and analysis of the VAE's latent space of the dataset MNIST, using Keras 2 and Tensorflow (obsolete)

Generative network with Variational Autoencoder (VAE), using PyTorch Lightning

• LVAE1 - First VAE, using Lightning API (MNIST dataset)
  Construction and training of a VAE, using Lightning API, with a latent space of small dimension, using PyTorch Lightning
• LVAE2 - VAE, using a custom model class (MNIST dataset)
  Construction and training of a VAE, using model subclass, with a latent space of small dimension, using PyTorch Lightninh
• LVAE3 - Analysis of the VAE's latent space of MNIST dataset
  Visualization and analysis of the VAE's latent space of the dataset MNIST, using PyTorch Lightning

Generative Adversarial Networks (GANs), using Lightning

• LSHEEP3 - A DCGAN to Draw a Sheep, using Pytorch Lightning
  "Draw me a sheep", revisited with a DCGAN, using Pytorch Lightning

Diffusion Model (DDPM) using PyTorch

• DDPM1 - Fashion MNIST Generation with DDPM
  Diffusion Model example, to generate Fashion MNIST images.
• DDPM2 - DDPM Python classes
  Python classes used by DDMP Example

Training optimization, using PyTorch

• OPT1 - Training setup optimization
  The goal of this notebook is to go through a typical deep learning model training

Deep Reinforcement Learning (DRL), using PyTorch

• DRL1 - Solving CartPole with DQN
  Using a a Deep Q-Network to play CartPole - an inverted pendulum problem (PyTorch)
• DRL2 - RL Baselines3 Zoo: Training in Colab
  Demo of Stable baseline3 with Colab

Miscellaneous things, but very important!

• NP1 - A short introduction to Numpy
  Numpy is an essential tool for the Scientific Python.
• ACTF1 - Activation functions
  Some activation functions, with their derivatives.
• PANDAS1 - Quelques exemples avec Pandas
  pandas is another essential tool for the Scientific Python.
• PYTORCH1 - Practical Lab : PyTorch
  PyTorch est l'un des principaux framework utilisé dans le Deep Learning
• TSB1 - Tensorboard with/from Jupyter
  4 ways to use Tensorboard from the Jupyter environment
• SCRATCH1 - Scratchbook
  A scratchbook for small examples

NOTE : The examples marked "obsolete" are still functional under Keras2/Tensorflow, but cannot be run in the proposed environment, now based on Keras3, PyTorch and Lightning.
We have decided to consider Keras2/Tensorflow as pedagogically obsolete, although Keras2 and Tensorflow are still perfectly usable (January 2024).
For these resaon, they are kept as examples, while we develop the Keras3/PyTorch versions.
The world of Deep Learning is changing very fast !

Installation

Have a look about How to get and install these notebooks and datasets.

Licence

[en] Attribution - NonCommercial - ShareAlike 4.0 International (CC BY-NC-SA 4.0)
[Fr] Attribution - Pas d’Utilisation Commerciale - Partage dans les Mêmes Conditions 4.0 International
See License.
See Disclaimer.

---