GTSRB update

Former-commit-id: bb636b27

GTSRB/README.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "German Traffic Sign Recognition Benchmark (GTSRB)\n",
+    "=================================================\n",
+    "\n",
+    "---\n",
+    "Introduction au Deep Learning  (IDLE)  \n",
+    "S. Aria, E. Maldonado, JL. Parouty  \n",
+    "CNRS/SARI/DEVLOG - 2020\n",
+    "\n",
+    "Objectives of this practical work\n",
+    "---------------------------------\n",
+    "   \n",
+    "Traffic sign classification with **CNN**, using Tensorflow and **Keras**  \n",
+    "\n",
+    "\n",
+    "Prerequisite\n",
+    "------------\n",
+    "\n",
+    "Environment, with the following packages :\n",
+    " - Python 3.6\n",
+    " - numpy\n",
+    " - Matplotlib\n",
+    " - Tensorflow 2.0\n",
+    " - scikit-image\n",
+    " \n",
+    "You can create it from the `environment.yml` file :\n",
+    "```\n",
+    "# conda env create -f environment.yml\n",
+    "```\n",
+    "\n",
+    "About the dataset\n",
+    "-----------------\n",
+    "\n",
+    "Name : [German Traffic Sign Recognition Benchmark (GTSRB)](http://benchmark.ini.rub.de/?section=gtsrb)  \n",
+    "Available [here](https://sid.erda.dk/public/archives/daaeac0d7ce1152aea9b61d9f1e19370/published-archive.html) \n",
+    "or on **[kaggle](https://www.kaggle.com/meowmeowmeowmeowmeow/gtsrb-german-traffic-sign)**  \n",
+    "\n",
+    "A nice example from : [Alex Staravoitau](https://navoshta.com/traffic-signs-classification/)  \n",
+    "\n",
+    "In few words :\n",
+    " - Images : Variable dimensions, rgb\n",
+    " - Train set : 39209 images  \n",
+    " - Test set : 12630 images\n",
+    " - Classes : 0 to 42\n",
+    "\n",
+    "Episodes\n",
+    "--------\n",
+    "   \n",
+    "**[01 - Preparation of data](01-Preparation-of-data.ipynb)**\n",
+    " - Understanding the dataset\n",
+    " - Preparing and formatting data\n",
+    " - Organize and backup data\n",
+    " \n",
+    "**[02 - First convolutions](02-First-convolutions.ipynb)**\n",
+    " - Read dataset\n",
+    " - Build a model\n",
+    " - Train the model\n",
+    " - Model evaluation\n",
+    " "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.6.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

GTSRB/README.md

+German Traffic Sign Recognition Benchmark (GTSRB)
+=================================================
+
+---
+pjluc 2019 - CNRS/DEVLOG - Formation Deep Learning
+
+
+1/ Objectives
+   ----------
+   
+Traffic sign classification with **CNN**, using Tensorflow and **Keras**  
+
+2/ About the dataset
+   -----------------
+
+Name : [German Traffic Sign Recognition Benchmark (GTSRB)](http://benchmark.ini.rub.de/?section=gtsrb)  
+Available [here](https://sid.erda.dk/public/archives/daaeac0d7ce1152aea9b61d9f1e19370/published-archive.html) 
+or on **[kaggle](https://www.kaggle.com/meowmeowmeowmeowmeow/gtsrb-german-traffic-sign)**  
+A nice example from : [Alex Staravoitau](https://navoshta.com/traffic-signs-classification/)  
+
+In few words :
+ - Images : Variable dimensions, rgb
+ - Train set : 39209 images  
+ - Test set : 12630 images
+ - Classes : 0 to 42
+
+3/ Episodes
+   --------
+   
+   01 - Dataset preparation  
+   - Undestand the data
+   02 - First convolutions  
\ No newline at end of file

GTSRB/deepmods/pwk.py → GTSRB/idle/pwk.py

@@ -25,25 +25,25 @@ from tensorflow import keras
 import matplotlib
 import matplotlib.pyplot as plt
 
-VERSION='0.1.0'
+VERSION='0.1.1'
 
 
 # -------------------------------------------------------------
 # init_all
 # -------------------------------------------------------------
 #
-def init(mplstyle='deepmods/talk.mplstyle'):
+def init(mplstyle='idle/talk.mplstyle'):
     global VERSION
     # ---- matplotlib
     matplotlib.style.use(mplstyle)
     # ---- Hello world
     now = datetime.datetime.now()
-    print('\nDeepmod/pwk by pjluc 2019')
+    print('IDLE 2020 - Practical Work Module')
     print('  Version            :', VERSION)
     print('  Run time           : {}'.format(now.strftime("%A %-d %B %Y, %H:%M:%S")))
     print('  Matplotlib style   :', mplstyle)
-    print('  TensorFlow version : ',tf.__version__)
-    print('  Keras version      : ',tf.keras.__version__)
+    print('  TensorFlow version :',tf.__version__)
+    print('  Keras version      :',tf.keras.__version__)
           
 # -------------------------------------------------------------
 # init_folder
@@ -52,12 +52,25 @@ def init(mplstyle='deepmods/talk.mplstyle'):
 def init_folder(path):
     os.makedirs(path, mode=0o750, exist_ok=True)
       
+def get_directory_size(path):
+    """
+    Return the directory size, but only 1 level
+    args:
+        path : directory path
+    return:
+        size in Mo
+    """
+    size=0
+    for f in os.listdir(path):
+        if os.path.isfile(path+'/'+f):
+            size+=os.path.getsize(path+'/'+f)
+    return size/(1024*1024)
 
 # -------------------------------------------------------------
 # shuffle_dataset
 # -------------------------------------------------------------
 #
-def shuffle_dataset(x, y):
+def shuffle_np_dataset(x, y):
     assert (len(x) == len(y)), "x and y must have same size"
     p = np.random.permutation(len(x))
     return x[p], y[p]
@@ -70,7 +83,7 @@ def update_progress(what,i,imax):
     progress  = float(i/imax)
     block     = int(round(bar_length * progress))
     endofline = '\r' if progress<1 else '\n'
-    text = "{:16s} [{}] {:.1f}%".format( what, "#"*block+"-"*(bar_length-block), progress*100)
+    text = "{:16s} [{}] {:>5.1f}% of {}".format( what, "#"*block+"-"*(bar_length-block), progress*100, imax)
     print(text, end=endofline)
 
 

environment.yml

+name: deeplearning2
+channels:
+  - defaults
+dependencies:
+  - _libgcc_mutex=0.1=main
+  - _tflow_select=2.3.0=mkl
+  - absl-py=0.8.1=py36_0
+  - astor=0.8.0=py36_0
+  - attrs=19.3.0=py_0
+  - backcall=0.1.0=py36_0
+  - blas=1.0=mkl
+  - bleach=3.1.0=py36_0
+  - c-ares=1.15.0=h7b6447c_1001
+  - ca-certificates=2019.11.27=0
+  - certifi=2019.11.28=py36_0
+  - cloudpickle=1.2.2=py_0
+  - cycler=0.10.0=py36_0
+  - cytoolz=0.10.1=py36h7b6447c_0
+  - dask-core=2.9.0=py_0
+  - dbus=1.13.12=h746ee38_0
+  - decorator=4.4.1=py_0
+  - defusedxml=0.6.0=py_0
+  - entrypoints=0.3=py36_0
+  - expat=2.2.6=he6710b0_0
+  - fontconfig=2.13.0=h9420a91_0
+  - freetype=2.9.1=h8a8886c_1
+  - gast=0.2.2=py36_0
+  - glib=2.63.1=h5a9c865_0
+  - gmp=6.1.2=h6c8ec71_1
+  - google-pasta=0.1.8=py_0
+  - grpcio=1.16.1=py36hf8bcb03_1
+  - gst-plugins-base=1.14.0=hbbd80ab_1
+  - gstreamer=1.14.0=hb453b48_1
+  - h5py=2.9.0=py36h7918eee_0
+  - hdf5=1.10.4=hb1b8bf9_0
+  - icu=58.2=h9c2bf20_1
+  - imageio=2.6.1=py36_0
+  - importlib_metadata=1.3.0=py36_0
+  - intel-openmp=2019.4=243
+  - ipykernel=5.1.3=py36h39e3cac_0
+  - ipython=7.10.2=py36h39e3cac_0
+  - ipython_genutils=0.2.0=py36_0
+  - jedi=0.15.1=py36_0
+  - jinja2=2.10.3=py_0
+  - jpeg=9b=h024ee3a_2
+  - json5=0.8.5=py_0
+  - jsonschema=3.2.0=py36_0
+  - jupyter_client=5.3.4=py36_0
+  - jupyter_core=4.6.1=py36_0
+  - jupyterlab=1.2.4=pyhf63ae98_0
+  - jupyterlab_server=1.0.6=py_0
+  - keras-applications=1.0.8=py_0
+  - keras-preprocessing=1.1.0=py_1
+  - kiwisolver=1.1.0=py36he6710b0_0
+  - libedit=3.1.20181209=hc058e9b_0
+  - libffi=3.2.1=hd88cf55_4
+  - libgcc-ng=9.1.0=hdf63c60_0
+  - libgfortran-ng=7.3.0=hdf63c60_0
+  - libpng=1.6.37=hbc83047_0
+  - libprotobuf=3.11.2=hd408876_0
+  - libsodium=1.0.16=h1bed415_0
+  - libstdcxx-ng=9.1.0=hdf63c60_0
+  - libtiff=4.1.0=h2733197_0
+  - libuuid=1.0.3=h1bed415_2
+  - libxcb=1.13=h1bed415_1
+  - libxml2=2.9.9=hea5a465_1
+  - markdown=3.1.1=py36_0
+  - markupsafe=1.1.1=py36h7b6447c_0
+  - matplotlib=3.1.1=py36h5429711_0
+  - mistune=0.8.4=py36h7b6447c_0
+  - mkl=2019.4=243
+  - mkl-service=2.3.0=py36he904b0f_0
+  - mkl_fft=1.0.15=py36ha843d7b_0
+  - mkl_random=1.1.0=py36hd6b4f25_0
+  - more-itertools=8.0.2=py_0
+  - nbconvert=5.6.1=py36_0
+  - nbformat=4.4.0=py36_0
+  - ncurses=6.1=he6710b0_1
+  - networkx=2.4=py_0
+  - notebook=6.0.2=py36_0
+  - numpy=1.17.4=py36hc1035e2_0
+  - numpy-base=1.17.4=py36hde5b4d6_0
+  - olefile=0.46=py36_0
+  - openssl=1.1.1d=h7b6447c_3
+  - opt_einsum=3.1.0=py_0
+  - pandoc=2.2.3.2=0
+  - pandocfilters=1.4.2=py36_1
+  - parso=0.5.2=py_0
+  - pcre=8.43=he6710b0_0
+  - pexpect=4.7.0=py36_0
+  - pickleshare=0.7.5=py36_0
+  - pillow=6.2.1=py36h34e0f95_0
+  - pip=19.3.1=py36_0
+  - prometheus_client=0.7.1=py_0
+  - prompt_toolkit=3.0.2=py_0
+  - protobuf=3.11.2=py36he6710b0_0
+  - ptyprocess=0.6.0=py36_0
+  - pygments=2.5.2=py_0
+  - pyparsing=2.4.5=py_0
+  - pyqt=5.9.2=py36h05f1152_2
+  - pyrsistent=0.15.6=py36h7b6447c_0
+  - python=3.6.9=h265db76_0
+  - python-dateutil=2.8.1=py_0
+  - pytz=2019.3=py_0
+  - pywavelets=1.1.1=py36h7b6447c_0
+  - pyzmq=18.1.0=py36he6710b0_0
+  - qt=5.9.7=h5867ecd_1
+  - readline=7.0=h7b6447c_5
+  - scikit-image=0.15.0=py36he6710b0_0
+  - scipy=1.3.2=py36h7c811a0_0
+  - send2trash=1.5.0=py36_0
+  - setuptools=42.0.2=py36_0
+  - sip=4.19.8=py36hf484d3e_0
+  - six=1.13.0=py36_0
+  - sqlite=3.30.1=h7b6447c_0
+  - tensorboard=2.0.0=pyhb38c66f_1
+  - tensorflow=2.0.0=mkl_py36hef7ec59_0
+  - tensorflow-base=2.0.0=mkl_py36h9204916_0
+  - tensorflow-estimator=2.0.0=pyh2649769_0
+  - termcolor=1.1.0=py36_1
+  - terminado=0.8.3=py36_0
+  - testpath=0.4.4=py_0
+  - tk=8.6.8=hbc83047_0
+  - toolz=0.10.0=py_0
+  - tornado=6.0.3=py36h7b6447c_0
+  - traitlets=4.3.3=py36_0
+  - wcwidth=0.1.7=py36_0
+  - webencodings=0.5.1=py36_1
+  - werkzeug=0.16.0=py_0
+  - wheel=0.33.6=py36_0
+  - wrapt=1.11.2=py36h7b6447c_0
+  - xz=5.2.4=h14c3975_4
+  - zeromq=4.3.1=he6710b0_3
+  - zipp=0.6.0=py_0
+  - zlib=1.2.11=h7b6447c_3
+  - zstd=1.3.7=h0b5b093_0
+prefix: /home/pjluc/anaconda3/envs/deeplearning2
+