02-First-convolutions.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img width=\"800px\" src=\"../fidle/img/header.svg\"></img>\n",
    "\n",
    "# <!-- TITLE --> [K3GTSRB2] - First convolutions\n",
    "<!-- DESC --> Episode 2 : First convolutions and first classification of our traffic signs, using Keras3\n",
    "<!-- AUTHOR : Jean-Luc Parouty (CNRS/SIMaP) -->\n",
    "\n",
    "## Objectives :\n",
    "  - Recognizing traffic signs \n",
    "  - Understand the **principles** and **architecture** of a **convolutional neural network** for image classification\n",
    "  \n",
    "The German Traffic Sign Recognition Benchmark (GTSRB) is a dataset with more than 50,000 photos of road signs from about 40 classes.  \n",
    "The final aim is to recognise them !  \n",
    "\n",
    "Description is available there : http://benchmark.ini.rub.de/?section=gtsrb&subsection=dataset\n",
    "\n",
    "\n",
    "**IMPORTANT :** To be able to use this notebook and the following, **you must have generated the enhanced datasets** in <dataset_dir>/enhanced via the notebook **[01-Preparation-of-data.ipynb](01-Preparation-of-data.ipynb)**  \n",
    "\n",
    "## What we're going to do :\n",
    "\n",
    " - Read H5 dataset\n",
    " - Build a model\n",
    " - Train the model\n",
    " - Evaluate the model\n",
    "\n",
    "## Step 1 - Import and init\n",
    "### 1.1 - Python stuff"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ['KERAS_BACKEND'] = 'torch'\n",
    "\n",
    "import keras\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import h5py\n",
    "import os,time,sys\n",
    "\n",
    "from importlib import reload\n",
    "\n",
    "# Init Fidle environment\n",
    "import fidle\n",
    "\n",
    "run_id, run_dir, datasets_dir = fidle.init('K3GTSRB2')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.2 - Parameters\n",
    "`scale` is the proportion of the dataset that will be used during the training. (1 mean 100%)  \n",
    "A 20% 24x24 dataset, with 5 epochs and a scale of 1, need  **3'30** on a CPU laptop.\\\n",
    "`fit_verbosity` is the verbosity during training : 0 = silent, 1 = progress bar, 2 = one line per epoch"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "enhanced_dir = './data'\n",
    "# enhanced_dir = f'{datasets_dir}/GTSRB/enhanced'\n",
    "\n",
    "dataset_name  = 'set-24x24-L'\n",
    "batch_size    = 64\n",
    "epochs        = 5\n",
    "scale         = 1\n",
    "fit_verbosity = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Override parameters (batch mode) - Just forget this cell"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fidle.override('enhanced_dir', 'dataset_name', 'batch_size', 'epochs', 'scale', 'fit_verbosity')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2 - Load dataset\n",
    "We're going to retrieve a previously recorded dataset.  \n",
    "For example: set-24x24-L"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def read_dataset(enhanced_dir, dataset_name, scale=1):\n",
    "    '''\n",
    "    Reads h5 dataset\n",
    "    Args:\n",
    "        filename     : datasets filename\n",
    "        dataset_name : dataset name, without .h5\n",
    "    Returns:    \n",
    "        x_train,y_train, x_test,y_test data, x_meta,y_meta\n",
    "    '''\n",
    "\n",
    "    # ---- Read dataset\n",
    "    #\n",
    "    chrono=fidle.Chrono()\n",
    "    chrono.start()\n",
    "    filename = f'{enhanced_dir}/{dataset_name}.h5'\n",
    "    with  h5py.File(filename,'r') as f:\n",
    "        x_train = f['x_train'][:]\n",
    "        y_train = f['y_train'][:]\n",
    "        x_test  = f['x_test'][:]\n",
    "        y_test  = f['y_test'][:]\n",
    "        x_meta  = f['x_meta'][:]\n",
    "        y_meta  = f['y_meta'][:]\n",
    "\n",
    "    # ---- Rescale \n",
    "    #\n",
    "    print('Original shape  :', x_train.shape, y_train.shape)\n",
    "    x_train,y_train, x_test,y_test = fidle.utils.rescale_dataset(x_train,y_train,x_test,y_test, scale=scale)\n",
    "    print('Rescaled shape  :', x_train.shape, y_train.shape)\n",
    "\n",
    "    # ---- Shuffle\n",
    "    #\n",
    "    x_train,y_train=fidle.utils.shuffle_np_dataset(x_train,y_train)\n",
    "\n",
    "    # ---- done\n",
    "    #\n",
    "    duration = chrono.get_delay()\n",
    "    size     = fidle.utils.hsize(os.path.getsize(filename))\n",
    "    print(f'\\nDataset \"{dataset_name}\" is loaded and shuffled. ({size} in {duration})')\n",
    "    return x_train,y_train, x_test,y_test, x_meta,y_meta\n",
    "\n",
    "# ---- Read dataset\n",
    "#\n",
    "x_train,y_train,x_test,y_test, x_meta,y_meta = read_dataset(enhanced_dir, dataset_name, scale)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3 - Have a look to the dataset\n",
    "We take a quick look as we go by..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"x_train : \", x_train.shape)\n",
    "print(\"y_train : \", y_train.shape)\n",
    "print(\"x_test  : \", x_test.shape)\n",
    "print(\"y_test  : \", y_test.shape)\n",
    "\n",
    "fidle.scrawler.images(x_train, y_train, range(12), columns=6,  x_size=2, y_size=2, save_as='01-dataset-medium')\n",
    "fidle.scrawler.images(x_train, y_train, range(36), columns=12, x_size=1, y_size=1, save_as='02-dataset-small')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 4 - Create model\n",
    "We will now build a model and train it...\n",
    "\n",
    "Some models :"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# ------------------------------------------------------------------\n",
    "# -- A simple model, for 24x24 or 48x48 images                    --\n",
    "# ------------------------------------------------------------------\n",
    "#\n",
    "def get_model_01(lx,ly,lz):\n",
    "    \n",
    "    model = keras.models.Sequential()\n",
    "\n",
    "    model.add( keras.layers.Input((lx,ly,lz)) )\n",
    "    \n",
    "    model.add( keras.layers.Conv2D(96, (3,3), activation='relu' ))\n",
    "    model.add( keras.layers.MaxPooling2D((2, 2)))\n",
    "    model.add( keras.layers.Dropout(0.2))\n",
    "\n",
    "    model.add( keras.layers.Conv2D(192, (3, 3), activation='relu'))\n",
    "    model.add( keras.layers.MaxPooling2D((2, 2)))\n",
    "    model.add( keras.layers.Dropout(0.2))\n",
    "\n",
    "    model.add( keras.layers.Flatten()) \n",
    "    model.add( keras.layers.Dense(1500, activation='relu'))\n",
    "    model.add( keras.layers.Dropout(0.5))\n",
    "\n",
    "    model.add( keras.layers.Dense(43, activation='softmax'))\n",
    "    return model\n",
    "    \n",
    "\n",
    "# ------------------------------------------------------------------\n",
    "# -- A more sophisticated model, for 48x48 images                 --\n",
    "# ------------------------------------------------------------------\n",
    "#\n",
    "def get_model_02(lx,ly,lz):\n",
    "    model = keras.models.Sequential()\n",
    "    \n",
    "    model.add( keras.layers.Input((lx,ly,lz)) )\n",
    "    \n",
    "    model.add( keras.layers.Conv2D(32, (3,3),   activation='relu'))\n",
    "    model.add( keras.layers.MaxPooling2D((2, 2)))\n",
    "    model.add( keras.layers.Dropout(0.5))\n",
    "\n",
    "    model.add( keras.layers.Conv2D(64, (3, 3), activation='relu'))\n",
    "    model.add( keras.layers.MaxPooling2D((2, 2)))\n",
    "    model.add( keras.layers.Dropout(0.5))\n",
    "\n",
    "    model.add( keras.layers.Conv2D(128, (3, 3), activation='relu'))\n",
    "    model.add( keras.layers.MaxPooling2D((2, 2)))\n",
    "    model.add( keras.layers.Dropout(0.5))\n",
    "\n",
    "    model.add( keras.layers.Conv2D(256, (3, 3), activation='relu'))\n",
    "    model.add( keras.layers.MaxPooling2D((2, 2)))\n",
    "    model.add( keras.layers.Dropout(0.5))\n",
    "\n",
    "    model.add( keras.layers.Flatten()) \n",
    "    model.add( keras.layers.Dense(1152, activation='relu'))\n",
    "    model.add( keras.layers.Dropout(0.5))\n",
    "\n",
    "    model.add( keras.layers.Dense(43, activation='softmax'))\n",
    "    return model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5 - Train the model\n",
    "**Get the shape of my data :**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(n,lx,ly,lz) = x_train.shape\n",
    "print(\"Images of the dataset have this folowing shape : \",(lx,ly,lz))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Get and compile a model, with the data shape :**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = get_model_01(lx,ly,lz)\n",
    "\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer = 'adam',\n",
    "              loss      = 'sparse_categorical_crossentropy',\n",
    "              metrics   = ['accuracy'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Train it :**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "chrono=fidle.Chrono()\n",
    "chrono.start()\n",
    "\n",
    "# ---- Shuffle train data\n",
    "x_train,y_train=fidle.utils.shuffle_np_dataset(x_train,y_train)\n",
    "\n",
    "# ---- Train\n",
    "history = model.fit(  x_train, y_train,\n",
    "                      batch_size      = batch_size,\n",
    "                      epochs          = epochs,\n",
    "                      verbose         = fit_verbosity,\n",
    "                      validation_data = (x_test, y_test))\n",
    "\n",
    "chrono.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 5 - Evaluate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "max_val_accuracy = max(history.history[\"val_accuracy\"])\n",
    "print(\"Max validation accuracy is : {:.4f}\".format(max_val_accuracy))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "score = model.evaluate(x_test, y_test, verbose=0)\n",
    "\n",
    "print('Test loss      : {:5.4f}'.format(score[0]))\n",
    "print('Test accuracy  : {:5.4f}'.format(score[1]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fidle.end()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"todo\">\n",
    "    What you can do:\n",
    "    <ul>\n",
    "        <li>Try the different models</li>\n",
    "        <li>Try with different datasets</li>\n",
    "        <li>Test different hyperparameters (epochs, batch size, optimization, etc.)</li>\n",
    "        <li>Create your own model</li>\n",
    "    </ul>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "<img width=\"80px\" src=\"../fidle/img/logo-paysage.svg\"></img>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9.2 ('fidle-env')",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  },
  "vscode": {
   "interpreter": {
    "hash": "b3929042cc22c1274d74e3e946c52b845b57cb6d84f2d591ffe0519b38e4896d"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}