Skip to content
Snippets Groups Projects
Commit 969a596c authored by Jean-Luc Parouty Jean-Luc.Parouty@simap.grenoble-inp.fr's avatar Jean-Luc Parouty Jean-Luc.Parouty@simap.grenoble-inp.fr
Browse files

Add data augmentation

parent d5434aea
No related branches found
No related tags found
No related merge requests found
Expand all Hide whitespace changes
Inline Side-by-side
GTSRB/03-Tracking-and-visualizing.ipynb
+ 0
− 1
View file @ 969a596c
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
German Traffic Sign Recognition Benchmark (GTSRB) German Traffic Sign Recognition Benchmark (GTSRB)
================================================= =================================================
--- ---
Introduction au Deep Learning (IDLE) - S. Aria, E. Maldonado, JL. Parouty - CNRS/SARI/DEVLOG - 2020 Introduction au Deep Learning (IDLE) - S. Aria, E. Maldonado, JL. Parouty - CNRS/SARI/DEVLOG - 2020
## Episode 3 : Tracking, visualizing and save models ## Episode 3 : Tracking, visualizing and save models
Our main steps: Our main steps:
- Monitoring and understanding our model training - Monitoring and understanding our model training
- Add recovery points - Add recovery points
- Analyze the results - Analyze the results
- Restore and run recovery pont - Restore and run recovery pont
## 1/ Import and init ## 1/ Import and init
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import tensorflow as tf import tensorflow as tf
from tensorflow import keras from tensorflow import keras
from tensorflow.keras.callbacks import TensorBoard from tensorflow.keras.callbacks import TensorBoard
import numpy as np import numpy as np
import h5py import h5py
from sklearn.metrics import confusion_matrix from sklearn.metrics import confusion_matrix
import matplotlib.pyplot as plt import matplotlib.pyplot as plt
import seaborn as sn import seaborn as sn
import os, time, random import os, time, random
import idle.pwk as ooo import idle.pwk as ooo
from importlib import reload from importlib import reload
ooo.init() ooo.init()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## 2/ Load dataset ## 2/ Load dataset
Dataset is one of the saved dataset: RGB25, RGB35, L25, L35, etc. Dataset is one of the saved dataset: RGB25, RGB35, L25, L35, etc.
First of all, we're going to use a smart dataset : **set-24x24-L** First of all, we're going to use a smart dataset : **set-24x24-L**
(with a GPU, it only takes 35'' compared to more than 5' with a CPU !) (with a GPU, it only takes 35'' compared to more than 5' with a CPU !)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
%%time %%time
def read_dataset(name): def read_dataset(name):
'''Reads h5 dataset from ./data '''Reads h5 dataset from ./data
Arguments: dataset name, without .h5 Arguments: dataset name, without .h5
Returns: x_train,y_train,x_test,y_test data''' Returns: x_train,y_train,x_test,y_test data'''
# ---- Read dataset # ---- Read dataset
filename='./data/'+name+'.h5' filename='./data/'+name+'.h5'
with h5py.File(filename) as f: with h5py.File(filename) as f:
x_train = f['x_train'][:] x_train = f['x_train'][:]
y_train = f['y_train'][:] y_train = f['y_train'][:]
x_test = f['x_test'][:] x_test = f['x_test'][:]
y_test = f['y_test'][:] y_test = f['y_test'][:]
# ---- done # ---- done
print('Dataset "{}" is loaded. ({:.1f} Mo)\n'.format(name,os.path.getsize(filename)/(1024*1024))) print('Dataset "{}" is loaded. ({:.1f} Mo)\n'.format(name,os.path.getsize(filename)/(1024*1024)))
return x_train,y_train,x_test,y_test return x_train,y_train,x_test,y_test
x_train,y_train,x_test,y_test = read_dataset('set-24x24-L') x_train,y_train,x_test,y_test = read_dataset('set-24x24-L')
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## 3/ Have a look to the dataset ## 3/ Have a look to the dataset
Note: Data must be reshape for matplotlib Note: Data must be reshape for matplotlib
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
print("x_train : ", x_train.shape) print("x_train : ", x_train.shape)
print("y_train : ", y_train.shape) print("y_train : ", y_train.shape)
print("x_test : ", x_test.shape) print("x_test : ", x_test.shape)
print("y_test : ", y_test.shape) print("y_test : ", y_test.shape)
ooo.plot_images(x_train, y_train, range(12), columns=6, x_size=2, y_size=2) ooo.plot_images(x_train, y_train, range(12), columns=6, x_size=2, y_size=2)
ooo.plot_images(x_train, y_train, range(36), columns=12, x_size=1, y_size=1) ooo.plot_images(x_train, y_train, range(36), columns=12, x_size=1, y_size=1)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## 4/ Create model ## 4/ Create model
We will now build a model and train it... We will now build a model and train it...
Some models... Some models...
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# A basic model # A basic model
# #
def get_model_v1(lx,ly,lz): def get_model_v1(lx,ly,lz):
model = keras.models.Sequential() model = keras.models.Sequential()
model.add( keras.layers.Conv2D(96, (3,3), activation='relu', input_shape=(lx,ly,lz))) model.add( keras.layers.Conv2D(96, (3,3), activation='relu', input_shape=(lx,ly,lz)))
model.add( keras.layers.MaxPooling2D((2, 2))) model.add( keras.layers.MaxPooling2D((2, 2)))
model.add( keras.layers.Dropout(0.2)) model.add( keras.layers.Dropout(0.2))
model.add( keras.layers.Conv2D(192, (3, 3), activation='relu')) model.add( keras.layers.Conv2D(192, (3, 3), activation='relu'))
model.add( keras.layers.MaxPooling2D((2, 2))) model.add( keras.layers.MaxPooling2D((2, 2)))
model.add( keras.layers.Dropout(0.2)) model.add( keras.layers.Dropout(0.2))
model.add( keras.layers.Flatten()) model.add( keras.layers.Flatten())
model.add( keras.layers.Dense(1500, activation='relu')) model.add( keras.layers.Dense(1500, activation='relu'))
model.add( keras.layers.Dropout(0.5)) model.add( keras.layers.Dropout(0.5))
model.add( keras.layers.Dense(43, activation='softmax')) model.add( keras.layers.Dense(43, activation='softmax'))
return model return model
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## 5/ Prepare callbacks ## 5/ Prepare callbacks
We will add 2 callbacks : We will add 2 callbacks :
- **TensorBoard** - **TensorBoard**
Training logs, which can be visualised with Tensorboard. Training logs, which can be visualised with Tensorboard.
`#tensorboard --logdir ./run/logs` `#tensorboard --logdir ./run/logs`
IMPORTANT : Relancer tensorboard à chaque run IMPORTANT : Relancer tensorboard à chaque run
- **Model backup** - **Model backup**
It is possible to save the model each xx epoch or at each improvement. It is possible to save the model each xx epoch or at each improvement.
The model can be saved completely or partially (weight). The model can be saved completely or partially (weight).
For full format, we can use HDF5 format. For full format, we can use HDF5 format.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
%%bash %%bash
# To clean old logs and saved model, run this cell # To clean old logs and saved model, run this cell
# #
#/bin/rm -r ./run/logs 2>/dev/null #/bin/rm -r ./run/logs 2>/dev/null
#/bin/rm -r ./run/models 2>/dev/null #/bin/rm -r ./run/models 2>/dev/null
/bin/ls -l ./run 2>/dev/null /bin/ls -l ./run 2>/dev/null
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
ooo.mkdir('./run/models') ooo.mkdir('./run/models')
ooo.mkdir('./run/logs') ooo.mkdir('./run/logs')
# ---- Callback tensorboard # ---- Callback tensorboard
log_dir = "./run/logs/tb_" + ooo.tag_now() log_dir = "./run/logs/tb_" + ooo.tag_now()
tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1) tensorboard_callback = tf.keras.callbacks.TensorBoard(log_dir=log_dir, histogram_freq=1)
# ---- Callback ModelCheckpoint - Save best model # ---- Callback ModelCheckpoint - Save best model
save_dir = "./run/models/best-model.h5" save_dir = "./run/models/best-model.h5"
bestmodel_callback = tf.keras.callbacks.ModelCheckpoint(filepath=save_dir, verbose=0, monitor='accuracy', save_best_only=True) bestmodel_callback = tf.keras.callbacks.ModelCheckpoint(filepath=save_dir, verbose=0, monitor='accuracy', save_best_only=True)
# ---- Callback ModelCheckpoint - Save model each epochs # ---- Callback ModelCheckpoint - Save model each epochs
save_dir = "./run/models/model-{epoch:04d}.h5" save_dir = "./run/models/model-{epoch:04d}.h5"
savemodel_callback = tf.keras.callbacks.ModelCheckpoint(filepath=save_dir, verbose=0, save_freq=2000*5) savemodel_callback = tf.keras.callbacks.ModelCheckpoint(filepath=save_dir, verbose=0, save_freq=2000*5)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## 5/ Train the model ## 5/ Train the model
**Get the shape of my data :** **Get the shape of my data :**
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
(n,lx,ly,lz) = x_train.shape (n,lx,ly,lz) = x_train.shape
print("Images of the dataset have this folowing shape : ",(lx,ly,lz)) print("Images of the dataset have this folowing shape : ",(lx,ly,lz))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**Get and compile a model, with the data shape :** **Get and compile a model, with the data shape :**
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
model = get_model_v1(lx,ly,lz) model = get_model_v1(lx,ly,lz)
# model.summary() # model.summary()
model.compile(optimizer='adam', model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy', loss='sparse_categorical_crossentropy',
metrics=['accuracy']) metrics=['accuracy'])
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**Train it :** **Train it :**
Note : La courbe d'apprentissage est visible en temps réel avec Tensorboard : Note : La courbe d'apprentissage est visible en temps réel avec Tensorboard :
`#tensorboard --logdir ./run/logs` `#tensorboard --logdir ./run/logs`
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
%%time %%time
batch_size = 64 batch_size = 64
epochs = 30 epochs = 30
# ---- Shuffle train data # ---- Shuffle train data
x_train,y_train=ooo.shuffle_np_dataset(x_train,y_train) x_train,y_train=ooo.shuffle_np_dataset(x_train,y_train)
# ---- Train # ---- Train
# Note: To be faster in our example, we can take only 2000 values # Note: To be faster in our example, we can take only 2000 values
# #
history = model.fit( x_train, y_train, history = model.fit( x_train, y_train,
batch_size=batch_size, batch_size=batch_size,
epochs=epochs, epochs=epochs,
verbose=1, verbose=1,
validation_data=(x_test[:200], y_test[:200]), validation_data=(x_test[:200], y_test[:200]),
callbacks=[tensorboard_callback, bestmodel_callback, savemodel_callback] ) callbacks=[tensorboard_callback, bestmodel_callback, savemodel_callback] )
model.save('./run/models/last-model.h5') model.save('./run/models/last-model.h5')
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**Evaluate it :** **Evaluate it :**
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
max_val_accuracy = max(history.history["val_accuracy"]) max_val_accuracy = max(history.history["val_accuracy"])
print("Max validation accuracy is : {:.4f}".format(max_val_accuracy)) print("Max validation accuracy is : {:.4f}".format(max_val_accuracy))
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
score = model.evaluate(x_test, y_test, verbose=0) score = model.evaluate(x_test, y_test, verbose=0)
print('Test loss : {:5.4f}'.format(score[0])) print('Test loss : {:5.4f}'.format(score[0]))
print('Test accuracy : {:5.4f}'.format(score[1])) print('Test accuracy : {:5.4f}'.format(score[1]))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## 6/ History ## 6/ History
The return of model.fit() returns us the learning history The return of model.fit() returns us the learning history
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
ooo.plot_history(history) ooo.plot_history(history)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## 7/ Evaluation and confusion ## 7/ Evaluation and confusion
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
reload(ooo)
y_pred = model.predict_classes(x_test) y_pred = model.predict_classes(x_test)
conf_mat = confusion_matrix(y_test,y_pred, normalize="true", labels=range(43)) conf_mat = confusion_matrix(y_test,y_pred, normalize="true", labels=range(43))
ooo.plot_confusion_matrix(conf_mat) ooo.plot_confusion_matrix(conf_mat)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## 8/ Restore and evaluate ## 8/ Restore and evaluate
### 8.1/ List saved models : ### 8.1/ List saved models :
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
!find ./run/models/ !find ./run/models/
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### 8.2/ Restore a model : ### 8.2/ Restore a model :
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
loaded_model = tf.keras.models.load_model('./run/models/best-model.h5') loaded_model = tf.keras.models.load_model('./run/models/best-model.h5')
# best_model.summary() # best_model.summary()
print("Loaded.") print("Loaded.")
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### 8.3/ Evaluate it : ### 8.3/ Evaluate it :
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
score = loaded_model.evaluate(x_test, y_test, verbose=0) score = loaded_model.evaluate(x_test, y_test, verbose=0)
print('Test loss : {:5.4f}'.format(score[0])) print('Test loss : {:5.4f}'.format(score[0]))
print('Test accuracy : {:5.4f}'.format(score[1])) print('Test accuracy : {:5.4f}'.format(score[1]))
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### 8.4/ Make a prediction : ### 8.4/ Make a prediction :
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# ---- Get a random image # ---- Get a random image
# #
i = random.randint(1,len(x_test)) i = random.randint(1,len(x_test))
x,y = x_test[i], y_test[i] x,y = x_test[i], y_test[i]
# ---- Do prediction # ---- Do prediction
# #
predictions = loaded_model.predict( np.array([x]) ) predictions = loaded_model.predict( np.array([x]) )
# ---- A prediction is just the output layer # ---- A prediction is just the output layer
# #
print("\nOutput layer from model is (x100) :\n") print("\nOutput layer from model is (x100) :\n")
with np.printoptions(precision=2, suppress=True, linewidth=95): with np.printoptions(precision=2, suppress=True, linewidth=95):
print(predictions*100) print(predictions*100)
# ---- Graphic visualisation # ---- Graphic visualisation
# #
print("\nGraphically :\n") print("\nGraphically :\n")
plt.figure(figsize=(12,2)) plt.figure(figsize=(12,2))
plt.bar(range(43), predictions[0], align='center', alpha=0.5) plt.bar(range(43), predictions[0], align='center', alpha=0.5)
plt.ylabel('Probability') plt.ylabel('Probability')
plt.ylim((0,1)) plt.ylim((0,1))
plt.xlabel('Class') plt.xlabel('Class')
plt.title('Trafic Sign prediction') plt.title('Trafic Sign prediction')
plt.show() plt.show()
# ---- Predict class # ---- Predict class
# #
p = np.argmax(predictions) p = np.argmax(predictions)
# ---- Show result # ---- Show result
# #
print("\nPrediction on the left, real stuff on the right :\n") print("\nPrediction on the left, real stuff on the right :\n")
ooo.plot_images([x,x_meta[y]], [p,y], range(2), columns=3, x_size=3, y_size=2) ooo.plot_images([x,x_meta[y]], [p,y], range(2), columns=3, x_size=3, y_size=2)
if p==y: if p==y:
print("YEEES ! that's right!") print("YEEES ! that's right!")
else: else:
print("oups, that's wrong ;-(") print("oups, that's wrong ;-(")
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
--- ---
That's all folks ! That's all folks !
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
``` ```
......
GTSRB/04-Data-augmentation.ipynb
+ 461
− 141
View file @ 969a596c
source diff could not be displayed: it is too large. Options to address this: view the blob.
GTSRB/04.1-Data-augmentation.ipynb 0 → 100644
+ 353
− 0
View file @ 969a596c
This diff is collapsed.
GTSRB/99 Run-Tensorboard.ipynb
+ 75
− 7
View file @ 969a596c
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Running Tensorboard from Jupyter lab Running Tensorboard from Jupyter lab
==================================== ====================================
--- ---
Introduction au Deep Learning (IDLE) - S. Arias, E. Maldonado, JL. Parouty - CNRS/SARI/DEVLOG - 2020 Introduction au Deep Learning (IDLE) - S. Arias, E. Maldonado, JL. Parouty - CNRS/SARI/DEVLOG - 2020
Vesion : 1.0 Vesion : 1.0
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Méthode 1 : Shell execute ## Méthode 1 : Shell execute
**Start** **Start**
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
%%bash %%bash
tensorboard_start --logdir ./run/logs tensorboard_start --logdir ./run/logs
``` ```
%% Output %% Output
Tensorbord started with pid 84593 Tensorbord started with pid 80451
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
%%bash %%bash
tensorboard_stop tensorboard_stop
``` ```
%% Output %% Output
Tensorboard process not found... Tensorboard process not found...
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
%%bash %%bash
# ---- Port number, # ---- Port number,
PORT_JPY="$(id -u)" PORT_JPY="$(id -u)"
PORT_TSB="$(( $PORT_J + 10000 ))" PORT_TSB="$(( $PORT_J + 10000 ))"
HOST_G="$(hostname)" HOST_G="$(hostname)"
SSH_CMD="/usr/bin/ssh -NL 8888:$HOST_G:$PORT_J -L 6006:$HOST_G:$PORT_T dahu.ciment" SSH_CMD="/usr/bin/ssh -NL 8888:$HOST_G:$PORT_J -L 6006:$HOST_G:$PORT_T dahu.ciment"
tensorboard --port 21277 --host 0.0.0.0 --logdir ./run/logs &>/dev/null & # tensorboard --port 21277 --host 0.0.0.0 --logdir ./run/logs &>/dev/null &
tensorboard --port 21277 --host 0.0.0.0 --logdir ./run/logs 2>&1
``` ```
%% Output
W0113 22:02:28.498013 140212267140864 plugin_event_accumulator.py:294] Found more than one graph event per run, or there was a metagraph containing a graph_def, as well as one or more graph events. Overwriting the graph with the newest event.
TensorBoard 2.0.0 at http://0.0.0.0:21277/ (Press CTRL+C to quit)
Traceback (most recent call last):
File "/home/paroutyj/.conda/envs/deeplearning2/bin/tensorboard", line 10, in <module>
sys.exit(run_main())
File "/home/paroutyj/.conda/envs/deeplearning2/lib/python3.7/site-packages/tensorboard/main.py", line 64, in run_main
app.run(tensorboard.main, flags_parser=tensorboard.configure)
File "/home/paroutyj/.conda/envs/deeplearning2/lib/python3.7/site-packages/absl/app.py", line 299, in run
_run_main(main, args)
File "/home/paroutyj/.conda/envs/deeplearning2/lib/python3.7/site-packages/absl/app.py", line 250, in _run_main
sys.exit(main(argv))
File "/home/paroutyj/.conda/envs/deeplearning2/lib/python3.7/site-packages/tensorboard/program.py", line 222, in main
self._register_info(server)
File "/home/paroutyj/.conda/envs/deeplearning2/lib/python3.7/site-packages/tensorboard/program.py", line 268, in _register_info
manager.write_info_file(info)
File "/home/paroutyj/.conda/envs/deeplearning2/lib/python3.7/site-packages/tensorboard/manager.py", line 268, in write_info_file
with open(_get_info_file_path(), "w") as outfile:
PermissionError: [Errno 13] Permission denied: '/tmp/.tensorboard-info/pid-94212.info'
---------------------------------------------------------------------------
CalledProcessError Traceback (most recent call last)
<ipython-input-19-d5c18c938787> in <module>
----> 1 get_ipython().run_cell_magic('bash', '', '\n# ---- Port number, \nPORT_JPY="$(id -u)"\nPORT_TSB="$(( $PORT_J + 10000 ))"\nHOST_G="$(hostname)"\nSSH_CMD="/usr/bin/ssh -NL 8888:$HOST_G:$PORT_J -L 6006:$HOST_G:$PORT_T dahu.ciment"\n\n# tensorboard --port 21277 --host 0.0.0.0 --logdir ./run/logs &>/dev/null &\ntensorboard --port 21277 --host 0.0.0.0 --logdir ./run/logs 2>&1\n')
~/.conda/envs/deeplearning2/lib/python3.7/site-packages/IPython/core/interactiveshell.py in run_cell_magic(self, magic_name, line, cell)
2350 with self.builtin_trap:
2351 args = (magic_arg_s, cell)
-> 2352 result = fn(*args, **kwargs)
2353 return result
2354
~/.conda/envs/deeplearning2/lib/python3.7/site-packages/IPython/core/magics/script.py in named_script_magic(line, cell)
140 else:
141 line = script
--> 142 return self.shebang(line, cell)
143
144 # write a basic docstring:
</home/paroutyj/.conda/envs/deeplearning2/lib/python3.7/site-packages/decorator.py:decorator-gen-110> in shebang(self, line, cell)
~/.conda/envs/deeplearning2/lib/python3.7/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
185 # but it's overkill for just that one bit of state.
186 def magic_deco(arg):
--> 187 call = lambda f, *a, **k: f(*a, **k)
188
189 if callable(arg):
~/.conda/envs/deeplearning2/lib/python3.7/site-packages/IPython/core/magics/script.py in shebang(self, line, cell)
243 sys.stderr.flush()
244 if args.raise_error and p.returncode!=0:
--> 245 raise CalledProcessError(p.returncode, cell, output=out, stderr=err)
246
247 def _run_script(self, p, cell, to_close):
CalledProcessError: Command 'b'\n# ---- Port number, \nPORT_JPY="$(id -u)"\nPORT_TSB="$(( $PORT_J + 10000 ))"\nHOST_G="$(hostname)"\nSSH_CMD="/usr/bin/ssh -NL 8888:$HOST_G:$PORT_J -L 6006:$HOST_G:$PORT_T dahu.ciment"\n\n# tensorboard --port 21277 --host 0.0.0.0 --logdir ./run/logs &>/dev/null &\ntensorboard --port 21277 --host 0.0.0.0 --logdir ./run/logs 2>&1\n'' returned non-zero exit status 1.
%% Cell type:code id: tags:
``` python
!echo "This a test">/tmp/.tensorboard-info/foobar-655465.txt
#!cat /tmp/foobar-655465.txt
#!rm /tmp/foobar-655465.txt
!ls -ld /tmp/.tensorboard-info
!ls -ld /tmp/.tensorboard-info/*
See : https://github.com/tensorflow/tensorboard/issues/2010
```
%% Output
/bin/sh: 1: cannot create /tmp/.tensorboard-info/foobar-655465.txt: Permission denied
drwxr-xr-x 2 juanbaldonado l-inria 60 Jan 13 21:00 /tmp/.tensorboard-info
-rw-r--r-- 1 juanbaldonado l-inria 346 Jan 13 21:00 /tmp/.tensorboard-info/pid-74153.info
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**Check** **Check**
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
!echo $(ps ax | grep 'tensorboard --port 21277' | grep -v grep | awk '{print $1}') !echo $(ps ax | grep 'tensorboard --port 21277' | grep -v grep | awk '{print $1}')
``` ```
%% Output %% Output
84593
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
!ps ax | grep tensorboard !ps ax | grep tensorboard
!pwd !pwd
``` ```
%% Output %% Output
83294 pts/0 Ss+ 0:00 /bin/sh -c ps ax | grep tensorboard 83294 pts/0 Ss+ 0:00 /bin/sh -c ps ax | grep tensorboard
83296 pts/0 S+ 0:00 grep tensorboard 83296 pts/0 S+ 0:00 grep tensorboard
/home/paroutyj/fidle/GTSRB /home/paroutyj/fidle/GTSRB
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**Stop** **Stop**
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
%%bash %%bash
p="$(ps ax | grep 'tensorboard --port 21277' | grep -v grep | awk '{print $1}')" p="$(ps ax | grep 'tensorboard --port 21277' | grep -v grep | awk '{print $1}')"
if [ -z ${p} ]; then echo "No process"; else kill $p; echo "Ended ($p)" ; fi if [ -z ${p} ]; then echo "No process"; else kill $p; echo "Ended ($p)" ; fi
``` ```
%% Output %% Output
Ended (82555) Ended (82555)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
## Méthode 2 : Magic command ## Méthode 2 : Magic command
**Start** **Start**
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
%load_ext tensorboard %load_ext tensorboard
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
%tensorboard --port 21277 --host 0.0.0.0 --logdir ./run/logs %tensorboard --port 21277 --host 0.0.0.0 --logdir ./run/logs
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**Stop** **Stop**
No way... use bash method No way... use bash method
## Methode 3 : Tensorboard module ## Methode 3 : Tensorboard module
**Start** **Start**
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import tensorboard.notebook as tsb import tensorboard.notebook as tsb
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
tsb.start('--port 21277 --host 0.0.0.0 --logdir ./run/logs') tsb.start('--port 21277 --host 0.0.0.0 --logdir ./run/logs')
``` ```
%% Output %% Output
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**Check** **Check**
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
a=tsb.list() a=tsb.list()
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
**Stop** **Stop**
No way... use bash method No way... use bash method
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment