diff --git a/Misc/05-RNN.ipynb b/Misc/05-RNN.ipynb
index 1c61ec98129c3d090985eb1e84a9d906f6bdba36..26254fa922135f33ff6e1790ddf4bdc9fce52917 100644
--- a/Misc/05-RNN.ipynb
+++ b/Misc/05-RNN.ipynb
@@ -1,5 +1,17 @@
{
"cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<img width=\"800px\" src=\"../fidle/img/header.svg\"></img>\n",
+ "\n",
+ "# <!-- TITLE --> [K3LSTM1] - Basic Keras LSTM Layer\n",
+ "<!-- DESC --> A small example of an LSTM layer in Keras\n",
+ "\n",
+ "<!-- AUTHOR : Jean-Luc Parouty (CNRS/SIMaP) -->"
+ ]
+ },
{
"cell_type": "code",
"execution_count": null,
@@ -52,7 +64,17 @@
"execution_count": null,
"metadata": {},
"outputs": [],
- "source": []
+ "source": [
+ "fidle.end()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "---\n",
+ "<img width=\"80px\" src=\"../fidle/img/logo-paysage.svg\"></img>"
+ ]
}
],
"metadata": {
diff --git a/Misc/06-Gradients.ipynb b/Misc/06-Gradients.ipynb
index 26c042b6e47415b3f24430336f1d559a40b045f5..1c7e58e8b5a72a43b2f516342dd6910f0e67715f 100644
--- a/Misc/06-Gradients.ipynb
+++ b/Misc/06-Gradients.ipynb
@@ -1,5 +1,24 @@
{
"cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<img width=\"800px\" src=\"../fidle/img/header.svg\"></img>\n",
+ "\n",
+ "# <!-- TITLE --> [PGRAD1] - Gradient illustration with PyTorch\n",
+ "<!-- DESC --> Exemple de calcul d'un gradient avec PyTorch\n",
+ "\n",
+ "<!-- AUTHOR : Jean-Luc Parouty (CNRS/SIMaP) -->\n",
+ "\n",
+ "## Objectives :\n",
+ " - Exemple de calcul d'un gradient avec PyTorch\n",
+ "\n",
+ "## What we're going to do :\n",
+ "\n",
+ "- Exemple de calcul d'un gradient avec PyTorch"
+ ]
+ },
{
"cell_type": "code",
"execution_count": 1,
diff --git a/Misc/99-Fid-Example.ipynb b/Misc/99-Fid-Example.ipynb
new file mode 100644
index 0000000000000000000000000000000000000000..6e710c952b8d6afe2d5d63c3c67770f6dbede584
--- /dev/null
+++ b/Misc/99-Fid-Example.ipynb
@@ -0,0 +1,157 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "<img width=\"800px\" src=\"../fidle/img/header.svg\"></img>\n",
+ "\n",
+ "# <!-- TITLE --> [FID1] - Exemple de notebook Fidle \n",
+ "<!-- DESC --> Un simple exemple de notebook Fidle\n",
+ "\n",
+ "<!-- AUTHOR : Jean-Luc Parouty (CNRS/SIMaP) -->"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "\n",
+ "> **Note :** Penser à bien remplir les tags TITLE, DESC et AUTHOR de la cellule ci-dessus (Voir source markdown)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Step 1 - Init Python\n",
+ "\n",
+ "> Penser à **importer** le **module Fidle** \n",
+ "> Penser à effectuer **l'initialisation de l'environnement Fidle** \n",
+ "> `FID1` est l'identifiant du notebook (run_id) \n",
+ "> `run_dir` est un dossier où mettre les outputs du notebook (typiquement ./run/<run_id>) \n",
+ "> `datasets_dir` le dossier où sont les datasets Fidle"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import torch\n",
+ "\n",
+ "import fidle\n",
+ "\n",
+ "# Init Fidle environment\n",
+ "run_id, run_dir, datasets_dir = fidle.init('FID1')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Parameters\n",
+ "> Nous avons ici (par exemple) 3 paramètres : scale, x et batch_size"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "scale = 0.1\n",
+ "x=12\n",
+ "batch_size=64"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "> L'appel ci-dessous permet de définir les parametres modifiables lors d'une exécution batch via la commande fid `run_ci...`"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fidle.override('scale', 'x','batch_size')"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Working part...\n",
+ "(Tout ce que fait notre notebook...)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "print('scale=',scale)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## End part"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "> Pour terminer le notebook, on peut :\n",
+ "> - faire un `fidle.end()` pour afficher quelques infos utiles\n",
+ "> - insérer un beau logo en markdown"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "fidle.end()"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "---\n",
+ "<img width=\"80px\" src=\"../fidle/img/logo-paysage.svg\"></img>"
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "fidle-env",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.9.2"
+ }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/README.ipynb b/README.ipynb
index a21d9355b52bf35cfb1b04a3cb924a9ac033dc4c..f462d451a890b0ac006d98c0d5dd4f2516b0a1ac 100644
--- a/README.ipynb
+++ b/README.ipynb
@@ -3,13 +3,13 @@
{
"cell_type": "code",
"execution_count": 1,
- "id": "3f8d27e1",
+ "id": "92eb46fc",
"metadata": {
"execution": {
- "iopub.execute_input": "2024-03-20T20:58:24.985713Z",
- "iopub.status.busy": "2024-03-20T20:58:24.985441Z",
- "iopub.status.idle": "2024-03-20T20:58:24.998771Z",
- "shell.execute_reply": "2024-03-20T20:58:24.997910Z"
+ "iopub.execute_input": "2024-03-22T17:43:14.034809Z",
+ "iopub.status.busy": "2024-03-22T17:43:14.033992Z",
+ "iopub.status.idle": "2024-03-22T17:43:14.048462Z",
+ "shell.execute_reply": "2024-03-22T17:43:14.047262Z"
},
"jupyter": {
"source_hidden": true
@@ -53,7 +53,7 @@
"For more information, you can contact us at : \n",
"[<img width=\"200px\" style=\"vertical-align:middle\" src=\"fidle/img/00-Mail_contact.svg\"></img>](#top)\n",
"\n",
- "Current Version : <!-- VERSION_BEGIN -->3.0.10<!-- VERSION_END -->\n",
+ "Current Version : <!-- VERSION_BEGIN -->3.0.11<!-- VERSION_END -->\n",
"\n",
"\n",
"## Course materials\n",
@@ -68,7 +68,7 @@
"## Jupyter notebooks\n",
"\n",
"<!-- TOC_BEGIN -->\n",
- "<!-- Automatically generated on : 20/03/24 21:58:23 -->\n",
+ "<!-- Automatically generated on : 22/03/24 18:43:13 -->\n",
"\n",
"### Linear and logistic regression\n",
"- **[LINR1](LinearReg/01-Linear-Regression.ipynb)** - [Linear regression with direct resolution](LinearReg/01-Linear-Regression.ipynb) \n",
@@ -154,6 +154,8 @@
"- **[TRANS2](Transformers.PyTorch/02-distilbert_colab.ipynb)** - [IMDB, Sentiment analysis with Transformers ](Transformers.PyTorch/02-distilbert_colab.ipynb) \n",
"Using a Tranformer to perform a sentiment analysis (IMDB) - Colab version\n",
"\n",
+ "### Graph Neural Networks\n",
+ "\n",
"### Unsupervised learning with an autoencoder neural network (AE), using Keras3\n",
"- **[K3AE1](AE.Keras3/01-Prepare-MNIST-dataset.ipynb)** - [Prepare a noisy MNIST dataset](AE.Keras3/01-Prepare-MNIST-dataset.ipynb) \n",
"Episode 1: Preparation of a noisy MNIST dataset\n",
@@ -205,10 +207,12 @@
"PyTorch est l'un des principaux framework utilisé dans le Deep Learning\n",
"- **[TSB1](Misc/04-Using-Tensorboard.ipynb)** - [Tensorboard with/from Jupyter ](Misc/04-Using-Tensorboard.ipynb) \n",
"4 ways to use Tensorboard from the Jupyter environment\n",
- "- **[??](Misc/05-RNN.ipynb)** - [??](Misc/05-RNN.ipynb) \n",
- "??\n",
- "- **[??](Misc/06-Gradients.ipynb)** - [??](Misc/06-Gradients.ipynb) \n",
- "??\n",
+ "- **[K3LSTM1](Misc/05-RNN.ipynb)** - [Basic Keras LSTM Layer](Misc/05-RNN.ipynb) \n",
+ "A small example of an LSTM layer in Keras\n",
+ "- **[PGRAD1](Misc/06-Gradients.ipynb)** - [Gradient illustration with PyTorch](Misc/06-Gradients.ipynb) \n",
+ "Exemple de calcul d'un gradient avec PyTorch\n",
+ "- **[FID1](Misc/99-Fid-Example.ipynb)** - [Exemple de notebook Fidle ](Misc/99-Fid-Example.ipynb) \n",
+ "Un simple exemple de notebook Fidle\n",
"<!-- TOC_END -->\n",
"\n",
"## Installation\n",
@@ -239,7 +243,7 @@
"from IPython.display import display,Markdown\n",
"display(Markdown(open('README.md', 'r').read()))\n",
"#\n",
- "# This README is visible under Jupiter Lab ;-)# Automatically generated on : 20/03/24 21:58:24"
+ "# This README is visible under Jupiter Lab ;-)# Automatically generated on : 22/03/24 18:43:13"
]
}
],
diff --git a/README.md b/README.md
index b44e752f71901981bf0797e08a1c1590b54a043e..71c00a3af946f18d689394acf4c6e2cc69d77cbe 100644
--- a/README.md
+++ b/README.md
@@ -32,7 +32,7 @@ For more information, see **https://fidle.cnrs.fr** :
For more information, you can contact us at :
[<img width="200px" style="vertical-align:middle" src="fidle/img/00-Mail_contact.svg"></img>](#top)
-Current Version : <!-- VERSION_BEGIN -->3.0.10<!-- VERSION_END -->
+Current Version : <!-- VERSION_BEGIN -->3.0.11<!-- VERSION_END -->
## Course materials
@@ -47,7 +47,7 @@ Have a look about **[How to get and install](https://fidle.cnrs.fr/installation)
## Jupyter notebooks
<!-- TOC_BEGIN -->
-<!-- Automatically generated on : 20/03/24 21:58:23 -->
+<!-- Automatically generated on : 22/03/24 18:43:13 -->
### Linear and logistic regression
- **[LINR1](LinearReg/01-Linear-Regression.ipynb)** - [Linear regression with direct resolution](LinearReg/01-Linear-Regression.ipynb)
@@ -133,6 +133,8 @@ Using a Tranformer to perform a sentiment analysis (IMDB) - Jean Zay version
- **[TRANS2](Transformers.PyTorch/02-distilbert_colab.ipynb)** - [IMDB, Sentiment analysis with Transformers ](Transformers.PyTorch/02-distilbert_colab.ipynb)
Using a Tranformer to perform a sentiment analysis (IMDB) - Colab version
+### Graph Neural Networks
+
### Unsupervised learning with an autoencoder neural network (AE), using Keras3
- **[K3AE1](AE.Keras3/01-Prepare-MNIST-dataset.ipynb)** - [Prepare a noisy MNIST dataset](AE.Keras3/01-Prepare-MNIST-dataset.ipynb)
Episode 1: Preparation of a noisy MNIST dataset
@@ -184,10 +186,12 @@ pandas is another essential tool for the Scientific Python.
PyTorch est l'un des principaux framework utilisé dans le Deep Learning
- **[TSB1](Misc/04-Using-Tensorboard.ipynb)** - [Tensorboard with/from Jupyter ](Misc/04-Using-Tensorboard.ipynb)
4 ways to use Tensorboard from the Jupyter environment
-- **[??](Misc/05-RNN.ipynb)** - [??](Misc/05-RNN.ipynb)
-??
-- **[??](Misc/06-Gradients.ipynb)** - [??](Misc/06-Gradients.ipynb)
-??
+- **[K3LSTM1](Misc/05-RNN.ipynb)** - [Basic Keras LSTM Layer](Misc/05-RNN.ipynb)
+A small example of an LSTM layer in Keras
+- **[PGRAD1](Misc/06-Gradients.ipynb)** - [Gradient illustration with PyTorch](Misc/06-Gradients.ipynb)
+Exemple de calcul d'un gradient avec PyTorch
+- **[FID1](Misc/99-Fid-Example.ipynb)** - [Exemple de notebook Fidle ](Misc/99-Fid-Example.ipynb)
+Un simple exemple de notebook Fidle
<!-- TOC_END -->
## Installation
diff --git a/fidle/about.yml b/fidle/about.yml
index c145d7895c19bccd7ba4e3aab9b73d9027c05384..921bb26f8170391708dc1cdac1f8fafd94af4666 100644
--- a/fidle/about.yml
+++ b/fidle/about.yml
@@ -13,7 +13,7 @@
#
# This file describes the notebooks used by the Fidle training.
-version: 3.0.10
+version: 3.0.11
content: notebooks
name: Notebooks Fidle
description: All notebooks used by the Fidle training
@@ -37,6 +37,7 @@ toc:
Embedding.Keras3: Sentiment analysis with word embedding, using Keras3/PyTorch
RNN.Keras3: Time series with Recurrent Neural Network (RNN), using Keras3/PyTorch
Transformers.PyTorch: Sentiment analysis with transformer, using PyTorch
+ GNN.PyTorch: Graph Neural Networks
AE.Keras3: Unsupervised learning with an autoencoder neural network (AE), using Keras3
VAE.Keras3: Generative network with Variational Autoencoder (VAE), using Keras3
DCGAN.Lightning: Generative Adversarial Networks (GANs), using Lightning
diff --git a/fidle/ci/batch_gpu.sh b/fidle/ci/batch_gpu.sh
new file mode 100644
index 0000000000000000000000000000000000000000..3791458e0c99f319f1de260e69b89c0d0a791733
--- /dev/null
+++ b/fidle/ci/batch_gpu.sh
@@ -0,0 +1,67 @@
+#!/bin/bash
+# -----------------------------------------------
+# _ _ _
+# | |__ __ _| |_ ___| |__
+# | '_ \ / _` | __/ __| '_ \
+# | |_) | (_| | || (__| | | |
+# |_.__/ \__,_|\__\___|_| |_|
+# Fidle at IDRIS
+# -----------------------------------------------
+#
+# SLURM batch script
+# Bash script for SLURM batch submission of ci notebooks
+# by Jean-Luc Parouty (CNRS/SIMaP)
+#
+# Soumission : sbatch /(...)/batch_slurm.sh
+# Suivi : squeue -u $USER
+
+# ==== Job parameters ==============================================
+
+#SBATCH --job-name="Fidle ci" # nom du job
+#SBATCH --ntasks=1 # nombre de tâche (un unique processus ici)
+#SBATCH --gres=gpu:1 # nombre de GPU à réserver (un unique GPU ici)
+#SBATCH --cpus-per-task=10 # nombre de coeurs à réserver (un quart du noeud)
+#SBATCH --hint=nomultithread # on réserve des coeurs physiques et non logiques
+#SBATCH --time=05:00:00 # temps exécution maximum demande (HH:MM:SS)
+#SBATCH --output="FIDLE_CI_%j.out" # nom du fichier de sortie
+#SBATCH --error="FIDLE_CI_%j.err" # nom du fichier des erreurs
+#SBATCH --mail-user=Someone@somewhere.fr
+#SBATCH --mail-type=END,FAIL
+
+# ==== Parameters ==================================================
+
+MODULE_ENV="pytorch-gpu/py3/2.1.1"
+RUN_DIR="$WORK/fidle-project/fidle"
+CAMPAIN_PROFILE="./fidle/ci/gpu-scale1.yml"
+FILTER=( '.*' )
+
+# ==================================================================
+
+echo '------------------------------------------------------------'
+echo "Start : $0"
+echo '------------------------------------------------------------'
+echo "Job id : $SLURM_JOB_ID"
+echo "Job name : $SLURM_JOB_NAME"
+echo "Job node list : $SLURM_JOB_NODELIST"
+echo '------------------------------------------------------------'
+echo "module loaded : $MODULE_ENV"
+echo "run dir : $RUN_DIR"
+echo "campain profile : $CAMPAIN_PROFILE"
+echo "filter : ${FILTER[@]}"
+echo '------------------------------------------------------------'
+
+# ---- Module + env.
+
+module purge
+module load "$MODULE_ENV"
+
+export PYTHONUSERBASE=$WORK/local/fidle-k3
+export PATH=$PATH:$PYTHONUSERBASE/bin
+
+# ---- Run it...
+
+cd "$RUN_DIR"
+
+fid run_ci --quiet --campain "$CAMPAIN_PROFILE" --filter ${FILTER[@]}
+
+echo 'Done.'
diff --git a/fidle/ci/cpu-default.yml b/fidle/ci/cpu-default.yml
index af25720e29f70c5f403a060d6b8ed53b993ff654..86fae788e9174721b6f4579f97b16c9157b63e4d 100644
--- a/fidle/ci/cpu-default.yml
+++ b/fidle/ci/cpu-default.yml
@@ -336,3 +336,10 @@ ACTF1:
PANDAS1:
notebook: Misc/02-Using-pandas.ipynb
+
+FID1:
+ notebook: Misc/07-Fid-Example.ipynb
+ overrides:
+ scale: .1
+ x: 345
+ batch_size: default
\ No newline at end of file
diff --git a/fidle/ci/default.yml b/fidle/ci/default.yml
index ae0bd6b0f2de7a69c736b80505a1909c6026876f..54e8e7210890cbe92ab5b85a6772135252ecdbaa 100644
--- a/fidle/ci/default.yml
+++ b/fidle/ci/default.yml
@@ -1,6 +1,6 @@
campain:
version: '1.0'
- description: Automatically generated ci profile (20/03/24 21:58:23)
+ description: Automatically generated ci profile (22/03/24 18:43:13)
directory: ./campains/default
existing_notebook: 'remove # remove|skip'
report_template: 'fidle # fidle|default'
@@ -186,6 +186,11 @@ TRANS1:
TRANS2:
notebook: Transformers.PyTorch/02-distilbert_colab.ipynb
+#
+# ------------ GNN.PyTorch
+#
+{}
+
#
# ------------ AE.Keras3
#
@@ -320,5 +325,13 @@ PYTORCH1:
TSB1:
notebook: Misc/04-Using-Tensorboard.ipynb
overrides: ??
-??:
+K3LSTM1:
+ notebook: Misc/05-RNN.ipynb
+PGRAD1:
notebook: Misc/06-Gradients.ipynb
+FID1:
+ notebook: Misc/99-Fid-Example.ipynb
+ overrides:
+ scale: default
+ x: default
+ batch_size: default
diff --git a/fidle/ci/gpu-scale1.yml b/fidle/ci/gpu-scale1.yml
index 254f875f958e49ef7eaf372dcadc188248b1073b..1ea21c66505095e77f7adbf8597558cf2dc88d13 100644
--- a/fidle/ci/gpu-scale1.yml
+++ b/fidle/ci/gpu-scale1.yml
@@ -346,3 +346,10 @@ ACTF1:
PANDAS1:
notebook: Misc/02-Using-pandas.ipynb
+
+FID1:
+ notebook: Misc/07-Fid-Example.ipynb
+ overrides:
+ scale: 1
+ x: 345
+ batch_size: default