Newer
Older
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img width=\"800px\" src=\"../fidle/img/header.svg\"></img>\n",
"# <!-- TITLE --> [K2AE2] - Building and training an AE denoiser model\n",
"<!-- DESC --> Episode 1 : Construction of a denoising autoencoder and training of it with a noisy MNIST dataset, using Keras 2 and Tensorflow (obsolete)\n",
"\n",
"<!-- AUTHOR : Jean-Luc Parouty (CNRS/SIMaP) -->\n",
"\n",
"## Objectives :\n",
" - Understanding and implementing a denoizing **autoencoder** neurals network (AE)\n",
" - First overview or example of Keras procedural syntax\n",
"The calculation needs being important, it is preferable to use a very simple dataset such as MNIST. \n",
"The use of a GPU is often indispensable.\n",
"\n",
"## What we're going to do :\n",
"\n",
" - Defining a VAE model\n",
" - Build the model\n",
" - Train it\n",
" - Follow the learning process with Tensorboard\n",
" \n",
"## Data Terminology :\n",
"- `clean_train`, `clean_test` for noiseless images \n",
"- `noisy_train`, `noisy_test` for noisy images\n",
"- `denoised_test` for denoised images at the output of the model"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 1 - Init python stuff\n",
"### 1.1 - Init"
]
},
{
"cell_type": "code",
"source": [
"import numpy as np\n",
"\n",
"import tensorflow as tf\n",
"from tensorflow import keras\n",
"from tensorflow.keras import layers\n",
"from tensorflow.keras.callbacks import ModelCheckpoint, TensorBoard\n",
"\n",
"from importlib import reload\n",
"import h5py\n",
"\n",
"from modules.MNIST import MNIST\n",
"from modules.ImagesCallback import ImagesCallback\n",
"run_id, run_dir, datasets_dir = fidle.init('K2AE2')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.2 - Parameters\n",
"`prepared_dataset` : Filename of the prepared dataset (Need 400 Mo, but can be in ./data) \n",
"`dataset_seed` : Random seed for shuffling dataset \n",
"`scale` : % of the dataset to use (1. for 100%) \n",
"`latent_dim` : Dimension of the latent space \n",
"`train_prop` : Percentage for train (the rest being for the test)\n",
"`epochs` : Nb of epochs for training\\\n",
"`fit_verbosity` is the verbosity during training : 0 = silent, 1 = progress bar, 2 = one line per epoch\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"prepared_dataset = './data/mnist-noisy.h5'\n",
"dataset_seed = 123\n",
"train_prop = .8\n",
"batch_size = 128\n",
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Override parameters (batch mode) - Just forget this cell"
]
},
{
"cell_type": "code",
"fidle.override('prepared_dataset', 'dataset_seed', 'scale', 'latent_dim')\n",
"fidle.override('train_prop', 'batch_size', 'epochs', 'fit_verbosity')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 2 - Retrieve dataset\n",
"With our MNIST class, in one call, we can reload, rescale, shuffle and split our previously saved dataset :-)"
]
},
{
"cell_type": "code",
"clean_train,clean_test, noisy_train,noisy_test, _,_ = MNIST.reload_prepared_dataset(scale = scale, \n",
" train_prop = train_prop,\n",
" seed = dataset_seed,\n",
" shuffle = True,\n",
" filename=prepared_dataset )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Encoder"
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [],
"source": [
"inputs = keras.Input(shape=(28, 28, 1))\n",
"x = layers.Conv2D(32, 3, activation=\"relu\", strides=2, padding=\"same\")(inputs)\n",
"x = layers.Conv2D(64, 3, activation=\"relu\", strides=2, padding=\"same\")(x)\n",
"x = layers.Flatten()(x)\n",
"x = layers.Dense(16, activation=\"relu\")(x)\n",
"z = layers.Dense(latent_dim)(x)\n",
"\n",
"encoder = keras.Model(inputs, z, name=\"encoder\")\n",
"# encoder.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Decoder"
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [],
"source": [
"inputs = keras.Input(shape=(latent_dim,))\n",
"x = layers.Dense(7 * 7 * 64, activation=\"relu\")(inputs)\n",
"x = layers.Reshape((7, 7, 64))(x)\n",
"x = layers.Conv2DTranspose(64, 3, activation=\"relu\", strides=2, padding=\"same\")(x)\n",
"x = layers.Conv2DTranspose(32, 3, activation=\"relu\", strides=2, padding=\"same\")(x)\n",
"outputs = layers.Conv2DTranspose(1, 3, activation=\"sigmoid\", padding=\"same\")(x)\n",
"\n",
"decoder = keras.Model(inputs, outputs, name=\"decoder\")\n",
"# decoder.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### AE\n"
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [],
"source": [
"inputs = keras.Input(shape=(28, 28, 1))\n",
"\n",
"latents = encoder(inputs)\n",
"outputs = decoder(latents)\n",
"\n",
"ae = keras.Model(inputs,outputs, name=\"ae\")\n",
"\n",
"ae.compile(optimizer=keras.optimizers.Adam(), loss='binary_crossentropy')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 4 - Train\n",
"20' on a CPU \n",
"1'12 on a GPU (V100, IDRIS)"
]
},
{
"cell_type": "code",
"metadata": {},
"outputs": [],
"source": [
"# ---- Callback : Images\n",
"fidle.utils.mkdir( run_dir + '/images')\n",
"filename = run_dir + '/images/image-{epoch:03d}-{i:02d}.jpg'\n",
"callback_images = ImagesCallback(filename, x=clean_test[:5], encoder=encoder,decoder=decoder)\n",
"\n",
"# ---- Callback : Best model\n",
"fidle.utils.mkdir( run_dir + '/models')\n",
"filename = run_dir + '/models/best_model.h5'\n",
"callback_bestmodel = tf.keras.callbacks.ModelCheckpoint(filepath=filename, verbose=0, save_best_only=True)\n",
"\n",
"# ---- Callback tensorboard\n",
"logdir = run_dir + '/logs'\n",
"callback_tensorboard = TensorBoard(log_dir=logdir, histogram_freq=1)\n",
"# callbacks_list = [callback_images, callback_bestmodel, callback_tensorboard]\n",
"callbacks_list = [callback_images, callback_bestmodel]"
]
},
{
"cell_type": "code",
"chrono = fidle.Chrono()\n",
"chrono.start()\n",
"history = ae.fit(noisy_train, clean_train,\n",
" batch_size = batch_size,\n",
" epochs = epochs,\n",
" validation_data = (noisy_test, clean_test),\n",
" callbacks = callbacks_list )\n",
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 5 - History"
]
},
{
"cell_type": "code",
"fidle.scrawler.history(history, plot={'loss':['loss','val_loss']}, save_as='01-history')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 6 - Denoising progress"
]
},
{
"cell_type": "code",
"for epoch in range(0,epochs,2):\n",
" filename = run_dir + '/images/image-{epoch:03d}-{i:02d}.jpg'.format(epoch=epoch, i=i)\n",
"fidle.utils.subtitle('Real images (clean_test) :')\n",
"fidle.scrawler.images(clean_test[:5], None, indices='all', columns=5, x_size=2,y_size=2, interpolation=None, save_as='02-original-real')\n",
"fidle.utils.subtitle('Noisy images (noisy_test) :')\n",
"fidle.scrawler.images(noisy_test[:5], None, indices='all', columns=5, x_size=2,y_size=2, interpolation=None, save_as='03-original-noisy')\n",
"fidle.utils.subtitle('Evolution during the training period (denoised_test) :')\n",
"fidle.scrawler.images(imgs, None, indices='all', columns=5, x_size=2,y_size=2, interpolation=None, y_padding=0.1, save_as='04-learning')\n",
"fidle.utils.subtitle('Noisy images (noisy_test) :')\n",
"fidle.scrawler.images(noisy_test[:5], None, indices='all', columns=5, x_size=2,y_size=2, interpolation=None, save_as=None)\n",
"fidle.utils.subtitle('Real images (clean_test) :')\n",
"fidle.scrawler.images(clean_test[:5], None, indices='all', columns=5, x_size=2,y_size=2, interpolation=None, save_as=None)\n"
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 7 - Evaluation\n",
"**Note :** We will use the following data:\\\n",
"`clean_train`, `clean_test` for noiseless images \\\n",
"`noisy_train`, `noisy_test` for noisy images\\\n",
"`denoised_test` for denoised images at the output of the model\n",
" \n",
"### 7.1 - Reload our best model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model = keras.models.load_model(f'{run_dir}/models/best_model.h5')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 7.2 - Let's make a prediction"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"denoised_test = model.predict(noisy_test)\n",
"\n",
"print('Denoised images (denoised_test) shape : ',denoised_test.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 7.3 - Denoised images "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"i=random.randint(0,len(denoised_test)-8)\n",
"j=i+8\n",
"\n",
"fidle.utils.subtitle('Noisy test images (input):')\n",
"fidle.scrawler.images(noisy_test[i:j], None, indices='all', columns=8, x_size=2,y_size=2, interpolation=None, save_as='05-test-noisy')\n",
"fidle.utils.subtitle('Denoised images (output):')\n",
"fidle.scrawler.images(denoised_test[i:j], None, indices='all', columns=8, x_size=2,y_size=2, interpolation=None, save_as='06-test-predict')\n",
"fidle.utils.subtitle('Real test images :')\n",
"fidle.scrawler.images(clean_test[i:j], None, indices='all', columns=8, x_size=2,y_size=2, interpolation=None, save_as='07-test-real')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"<img width=\"80px\" src=\"../fidle/img/logo-paysage.svg\"></img>"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.9.2 ('fidle-env')",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.2"
},
"vscode": {
"interpreter": {
"hash": "b3929042cc22c1274d74e3e946c52b845b57cb6d84f2d591ffe0519b38e4896d"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}