Skip to content
Snippets Groups Projects
Commit c31b70e4 authored by Jean-Luc Parouty's avatar Jean-Luc Parouty
Browse files

update GTSRB

parent 26ceb97d
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
GTSRB/03-Tracking-and-visualizing.ipynb
+ 181
− 81
View file @ c31b70e4
%% Cell type:markdown id: tags:
German Traffic Sign Recognition Benchmark (GTSRB)
=================================================
---
Introduction au Deep Learning (IDLE) - S. Aria, E. Maldonado, JL. Parouty - CNRS/SARI/DEVLOG - 2020
## Episode 3 : Tracking and visualizing
Our main steps:
- Monitoring and understanding our model training
- Analyze the results
- Improving our model
- Add recovery points
## 1/ Import and init
%% Cell type:code id: tags:
``` python
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.callbacks import TensorBoard
import numpy as np
import matplotlib
import matplotlib.pyplot as plt
import time
from importlib import reload
import idle.pwk as ooo
ooo.init()
```
%% Output
IDLE 2020 - Practical Work Module
Version : 0.1.1
Run time : Monday 6 January 2020, 23:42:11
Run time : Tuesday 7 January 2020, 17:38:16
Matplotlib style : idle/talk.mplstyle
TensorFlow version : 2.0.0
Keras version : 2.2.4-tf
%% Cell type:markdown id: tags:
## 2/ Reload dataset
Dataset is one of the saved dataset: RGB25, RGB35, L25, L35, etc.
First of all, we're going to use the dataset : **L25**
(with a GPU, it only takes 35'' compared to more than 5' with a CPU !)
%% Cell type:code id: tags:
``` python
%%time
dataset ='L25'
img_lx = 25
img_ly = 25
img_lz = 1
# ---- Read dataset
x_train = np.load('./data/{}/x_train.npy'.format(dataset))
y_train = np.load('./data/{}/y_train.npy'.format(dataset))
x_test = np.load('./data/{}/x_test.npy'.format(dataset))
y_test = np.load('./data/{}/y_test.npy'.format(dataset))
# ---- Reshape data
x_train = x_train.reshape( x_train.shape[0], img_lx, img_ly, img_lz)
x_test = x_test.reshape( x_test.shape[0], img_lx, img_ly, img_lz)
input_shape = (img_lx, img_ly, img_lz)
print("Dataset loaded, size={:.1f} Mo\n".format(ooo.get_directory_size('./data/'+dataset)))
```
%% Output
Dataset loaded, size=247.6 Mo
CPU times: user 0 ns, sys: 344 ms, total: 344 ms
Wall time: 498 ms
CPU times: user 0 ns, sys: 312 ms, total: 312 ms
Wall time: 321 ms
%% Cell type:markdown id: tags:
## 3/ Have a look to the dataset
Note: Data must be reshape for matplotlib
%% Cell type:code id: tags:
``` python
print("x_train : ", x_train.shape)
print("y_train : ", y_train.shape)
print("x_test : ", x_test.shape)
print("y_test : ", y_test.shape)
if img_lz>1:
ooo.plot_images(x_train.reshape(-1,img_lx,img_ly,img_lz), y_train, range(6), columns=3, x_size=4, y_size=3)
ooo.plot_images(x_train.reshape(-1,img_lx,img_ly,img_lz), y_train, range(36), columns=12, x_size=1, y_size=1)
else:
ooo.plot_images(x_train.reshape(-1,img_lx,img_ly), y_train, range(6), columns=6, x_size=2, y_size=2)
ooo.plot_images(x_train.reshape(-1,img_lx,img_ly), y_train, range(36), columns=12, x_size=1, y_size=1)
```
%% Output
x_train : (39209, 25, 25, 1)
y_train : (39209,)
x_test : (12630, 25, 25, 1)
y_test : (12630,)
%% Cell type:markdown id: tags:
## 4/ Create model
%% Cell type:code id: tags:
``` python
batch_size = 64
num_classes = 43
epochs = 10
epochs = 20
```
%% Cell type:code id: tags:
``` python
tf.keras.backend.clear_session()
model = keras.models.Sequential()
model.add( keras.layers.Conv2D(96, (3,3), activation='relu', input_shape=(img_lx, img_ly, img_lz)))
model.add( keras.layers.MaxPooling2D((2, 2)))
model.add( keras.layers.Conv2D(192, (3, 3), activation='relu'))
model.add( keras.layers.MaxPooling2D((2, 2)))
model.add( keras.layers.Flatten())
model.add( keras.layers.Dense(3072, activation='relu'))
model.add( keras.layers.Dense(500, activation='relu'))
model.add( keras.layers.Dense(500, activation='relu'))
model.add( keras.layers.Dense(43, activation='softmax'))
model.summary()
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
```
%% Output
Model: "sequential"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv2d (Conv2D) (None, 23, 23, 96) 960
_________________________________________________________________
max_pooling2d (MaxPooling2D) (None, 11, 11, 96) 0
_________________________________________________________________
conv2d_1 (Conv2D) (None, 9, 9, 192) 166080
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 4, 4, 192) 0
_________________________________________________________________
flatten (Flatten) (None, 3072) 0
_________________________________________________________________
dense (Dense) (None, 3072) 9440256
_________________________________________________________________
dense_1 (Dense) (None, 500) 1536500
_________________________________________________________________
dense_2 (Dense) (None, 500) 250500
_________________________________________________________________
dense_3 (Dense) (None, 43) 21543
=================================================================
Total params: 11,415,839
Trainable params: 11,415,839
Non-trainable params: 0
_________________________________________________________________
%% Cell type:markdown id: tags:
## 5/ Add callbacks
Nous allons ajouter 2 callbacks :
- **TensorBoard**
Training logs, which can be visualised with Tensorboard.
`#tensorboard --logdir ./run/logs`
IMPORTANT : Relancer tensorboard à chaque run
- **model backup**
- **Model backup**
It is possible to save the model each xx epoch or at each improvement.
The model can be saved completely or partially (weight).
For full format, we can use HDF5 format.
%% Cell type:code id: tags:
``` python
# To clean logs and saved model, run this cell
!/bin/rm -r ./run/logs ./run/models
!/bin/ls -l ./run
```
%% Output
total 0
%% Cell type:code id: tags:
``` python
ooo.mkdir('./run/models')
ooo.mkdir('./run/logs')
# ---- Callback tensorboard
log_dir = "./run/logs/" + ooo.tag_now()
log_dir = "./run/logs/tb_" + ooo.tag_now()
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
# ---- Callback ModelCheckpoint
save_dir = "./run/models"
checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(filepath=save_dir, verbose=0, monitor='accuracy', save_best_only=True)
# ---- Callback ModelCheckpoint - Save best model
save_dir = "./run/models/best-model.h5"
bestmodel_callback = tf.keras.callbacks.ModelCheckpoint(filepath=save_dir, verbose=0, monitor='accuracy', save_best_only=True)
# ---- Callback ModelCheckpoint - Save model each epochs
save_dir = "./run/models/model-{epoch:04d}.h5"
savemodel_callback = tf.keras.callbacks.ModelCheckpoint(filepath=save_dir, verbose=0, save_freq=2000*5)
```
%% Cell type:markdown id: tags:
## 5/ Run model
%% Cell type:code id: tags:
``` python
%%time
history = model.fit( x_train[:3000], y_train[:3000],
history = model.fit( x_train[:2000], y_train[:2000],
batch_size=batch_size,
epochs=epochs,
verbose=1,
validation_data=(x_test[:500], y_test[:500]),
callbacks=[tensorboard_callback, checkpoint_callback] )
validation_data=(x_test[:200], y_test[:200]),
callbacks=[tensorboard_callback, bestmodel_callback, savemodel_callback] )
model.save('./run/models/last-model.h5')
```
%% Output
Train on 3000 samples, validate on 500 samples
Epoch 1/10
2944/3000 [============================>.] - ETA: 0s - loss: 3.4893 - accuracy: 0.0676INFO:tensorflow:Assets written to: ./run/models/assets
3000/3000 [==============================] - 8s 3ms/sample - loss: 3.4847 - accuracy: 0.0690 - val_loss: 3.2739 - val_accuracy: 0.1640
Epoch 2/10
2944/3000 [============================>.] - ETA: 0s - loss: 2.4907 - accuracy: 0.3227INFO:tensorflow:Assets written to: ./run/models/assets
3000/3000 [==============================] - 7s 2ms/sample - loss: 2.4802 - accuracy: 0.3250 - val_loss: 2.0143 - val_accuracy: 0.3900
Epoch 3/10
2944/3000 [============================>.] - ETA: 0s - loss: 1.3810 - accuracy: 0.5591INFO:tensorflow:Assets written to: ./run/models/assets
3000/3000 [==============================] - 7s 2ms/sample - loss: 1.3794 - accuracy: 0.5593 - val_loss: 1.3322 - val_accuracy: 0.6200
Epoch 4/10
2944/3000 [============================>.] - ETA: 0s - loss: 0.8501 - accuracy: 0.7286INFO:tensorflow:Assets written to: ./run/models/assets
3000/3000 [==============================] - 7s 2ms/sample - loss: 0.8426 - accuracy: 0.7323 - val_loss: 1.1705 - val_accuracy: 0.6580
Epoch 5/10
2944/3000 [============================>.] - ETA: 0s - loss: 0.5132 - accuracy: 0.8400INFO:tensorflow:Assets written to: ./run/models/assets
3000/3000 [==============================] - 8s 3ms/sample - loss: 0.5087 - accuracy: 0.8413 - val_loss: 0.9281 - val_accuracy: 0.7360
Epoch 6/10
2944/3000 [============================>.] - ETA: 0s - loss: 0.3740 - accuracy: 0.8787INFO:tensorflow:Assets written to: ./run/models/assets
3000/3000 [==============================] - 8s 3ms/sample - loss: 0.3763 - accuracy: 0.8783 - val_loss: 0.9252 - val_accuracy: 0.7520
Epoch 7/10
2944/3000 [============================>.] - ETA: 0s - loss: 0.2671 - accuracy: 0.9141INFO:tensorflow:Assets written to: ./run/models/assets
3000/3000 [==============================] - 8s 3ms/sample - loss: 0.2677 - accuracy: 0.9140 - val_loss: 0.8153 - val_accuracy: 0.8000
Epoch 8/10
2944/3000 [============================>.] - ETA: 0s - loss: 0.1599 - accuracy: 0.9535INFO:tensorflow:Assets written to: ./run/models/assets
3000/3000 [==============================] - 8s 3ms/sample - loss: 0.1588 - accuracy: 0.9543 - val_loss: 0.7000 - val_accuracy: 0.8500
Epoch 9/10
2944/3000 [============================>.] - ETA: 0s - loss: 0.1282 - accuracy: 0.9626INFO:tensorflow:Assets written to: ./run/models/assets
3000/3000 [==============================] - 8s 3ms/sample - loss: 0.1271 - accuracy: 0.9633 - val_loss: 0.7833 - val_accuracy: 0.8080
Epoch 10/10
2944/3000 [============================>.] - ETA: 0s - loss: 0.0979 - accuracy: 0.9715INFO:tensorflow:Assets written to: ./run/models/assets
3000/3000 [==============================] - 8s 3ms/sample - loss: 0.0971 - accuracy: 0.9720 - val_loss: 0.7634 - val_accuracy: 0.8420
CPU times: user 4min 26s, sys: 1min 31s, total: 5min 58s
Wall time: 1min 15s
Train on 2000 samples, validate on 200 samples
Epoch 1/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 3.6179 - accuracy: 0.0595 - val_loss: 3.5203 - val_accuracy: 0.0550
Epoch 2/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 3.4567 - accuracy: 0.0600 - val_loss: 3.4113 - val_accuracy: 0.0750
Epoch 3/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 2.9967 - accuracy: 0.2045 - val_loss: 2.7160 - val_accuracy: 0.2450
Epoch 4/20
2000/2000 [==============================] - 5s 2ms/sample - loss: 2.1231 - accuracy: 0.3710 - val_loss: 2.2007 - val_accuracy: 0.3550
Epoch 5/20
2000/2000 [==============================] - 5s 2ms/sample - loss: 1.5418 - accuracy: 0.5290 - val_loss: 1.6167 - val_accuracy: 0.5050
Epoch 6/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 1.1403 - accuracy: 0.6315 - val_loss: 1.6735 - val_accuracy: 0.5100
Epoch 7/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 0.8817 - accuracy: 0.7070 - val_loss: 1.1985 - val_accuracy: 0.6300
Epoch 8/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 0.6063 - accuracy: 0.8005 - val_loss: 1.1051 - val_accuracy: 0.7050
Epoch 9/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 0.4553 - accuracy: 0.8540 - val_loss: 1.1474 - val_accuracy: 0.6850
Epoch 10/20
2000/2000 [==============================] - 6s 3ms/sample - loss: 0.3155 - accuracy: 0.9005 - val_loss: 0.9822 - val_accuracy: 0.7500
Epoch 11/20
2000/2000 [==============================] - 6s 3ms/sample - loss: 0.2466 - accuracy: 0.9220 - val_loss: 0.9238 - val_accuracy: 0.7550
Epoch 12/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 0.1964 - accuracy: 0.9365 - val_loss: 1.0252 - val_accuracy: 0.7600
Epoch 13/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 0.1690 - accuracy: 0.9445 - val_loss: 0.9506 - val_accuracy: 0.7550
Epoch 14/20
2000/2000 [==============================] - 6s 3ms/sample - loss: 0.1053 - accuracy: 0.9690 - val_loss: 1.1243 - val_accuracy: 0.7300
Epoch 15/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 0.1054 - accuracy: 0.9730 - val_loss: 0.8967 - val_accuracy: 0.8000
Epoch 16/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 0.1285 - accuracy: 0.9620 - val_loss: 0.9004 - val_accuracy: 0.7850
Epoch 17/20
2000/2000 [==============================] - 6s 3ms/sample - loss: 0.0744 - accuracy: 0.9805 - val_loss: 0.8287 - val_accuracy: 0.8200
Epoch 18/20
2000/2000 [==============================] - 6s 3ms/sample - loss: 0.0400 - accuracy: 0.9915 - val_loss: 1.0527 - val_accuracy: 0.8150
Epoch 19/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 0.0618 - accuracy: 0.9805 - val_loss: 0.9737 - val_accuracy: 0.8000
Epoch 20/20
2000/2000 [==============================] - 5s 3ms/sample - loss: 0.0806 - accuracy: 0.9755 - val_loss: 1.1634 - val_accuracy: 0.8050
CPU times: user 6min 23s, sys: 2min 26s, total: 8min 49s
Wall time: 1min 47s
%% Cell type:markdown id: tags:
## 6/ Evaluation
## 6/ History
The return of model.fit() returns us the learning history
%% Cell type:code id: tags:
``` python
score = model.evaluate(x_test, y_test, verbose=0)
ooo.plot_history(history)
```
print('Test loss : {:5.4f}'.format(score[0]))
print('Test accuracy : {:5.4f}'.format(score[1]))
%% Output
%% Cell type:markdown id: tags:
## 7/ Restore and evaluate
List of saved models :
%% Cell type:code id: tags:
``` python
!find ./run/models/
```
%% Output
./run/models/
./run/models/best-model.h5
./run/models/last-model.h5
%% Cell type:markdown id: tags:
## 7/ History
The return of model.fit() returns us the learning history
Restore current model and evaluate it..
%% Cell type:code id: tags:
``` python
ooo.plot_history(history)
best_model = tf.keras.models.load_model('./run/models/best-model.h5')
# best_model.summary()
score = best_model.evaluate(x_test, y_test, verbose=0)
print('Test loss : {:5.4f}'.format(score[0]))
print('Test accuracy : {:5.4f}'.format(score[1]))
```
%% Output
Test loss : 0.8608
Test accuracy : 0.8466
%% Cell type:markdown id: tags:
---
### Results :
L25 : size=250 Mo, 93.15%
...
### A faire :
- Restauration
- Reprise apprentissage
- Evaluation
- Matrice de confusion
%% Cell type:code id: tags:
``` python
!ls -l {save_dir}
!tree {save_dir}
predictions = best_model.predict(x_test[3004:3005])
with np.printoptions(precision=2, suppress=True, linewidth=95):
print(predictions*100)
```
%% Output
total 176
drwxr-xr-x 1 pjluc pjluc 512 Jan 7 00:16 assets
-rw-r--r-- 1 pjluc pjluc 168427 Jan 7 00:17 saved_model.pb
drwxr-xr-x 1 pjluc pjluc 512 Jan 7 00:17 variables
./run/models
├── assets
├── saved_model.pb
└── variables
├── variables.data-00000-of-00001
└── variables.index
2 directories, 3 files
[[ 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 2.07 0. 0. 0.
0. 0. 0. 0. 0. 0. 0.15 0. 0. 0. 0. 0. 1.64 1.56 0.
94.58 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. ]]
%% Cell type:markdown id: tags:
%% Cell type:code id: tags:
A suivre : https://www.tensorflow.org/tutorials/keras/save_and_load
``` python
```
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment