{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img width=\"800px\" src=\"../fidle/img/header.svg\"></img>\n",
"\n",
"# <!-- TITLE --> [K3IMDB2] - Sentiment analysis with text embedding\n",
"<!-- DESC --> A very classical example of word embedding with a dataset from Internet Movie Database (IMDB), using Keras 3 on PyTorch\n",
"<!-- AUTHOR : Jean-Luc Parouty (CNRS/SIMaP) -->\n",
"\n",
"## Objectives :\n",
" - The objective is to guess whether film reviews are **positive or negative** based on the analysis of the text. \n",
" - Understand the management of **textual data** and **sentiment analysis**\n",
"\n",
"Original dataset can be find **[there](http://ai.stanford.edu/~amaas/data/sentiment/)** \n",
"Note that [IMDb.com](https://imdb.com) offers several easy-to-use [datasets](https://www.imdb.com/interfaces/) \n",
"For simplicity's sake, we'll use the dataset directly [embedded in Keras](https://keras.io/datasets)\n",
"\n",
"## What we're going to do :\n",
"\n",
" - Retrieve data\n",
" - Preparing the data\n",
" - Build a model\n",
" - Train the model\n",
" - Evaluate the result\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 1 - Import and init\n",
"### 1.1 - Python stuff"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"os.environ['KERAS_BACKEND'] = 'torch'\n",
"\n",
"import keras\n",
"import keras.datasets.imdb as imdb\n",
"\n",
"import h5py,json\n",
"import numpy as np\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"\n",
"import fidle\n",
"\n",
"# Init Fidle environment\n",
"run_id, run_dir, datasets_dir = fidle.init('K3IMDB2')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.2 - Parameters\n",
"The words in the vocabulary are classified from the most frequent to the rarest. \n",
"`vocab_size` is the number of words we will remember in our vocabulary (the other words will be considered as unknown). \n",
"`hide_most_frequently` is the number of ignored words, among the most common ones \n",
"`review_len` is the review length \n",
"`dense_vector_size` is the size of the generated dense vectors \n",
"`output_dir` is where we will go to save our dictionaries. (./data is a good choice)\\\n",
"`fit_verbosity` is the verbosity during training : 0 = silent, 1 = progress bar, 2 = one line per epoch"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"vocab_size = 5000\n",
"hide_most_frequently = 0\n",
"\n",
"review_len = 256\n",
"dense_vector_size = 32\n",
"\n",
"epochs = 30\n",
"batch_size = 512\n",
"\n",
"output_dir = './data'\n",
"fit_verbosity = 1"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Override parameters (batch mode) - Just forget this cell"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"fidle.override('vocab_size', 'hide_most_frequently', 'review_len', 'dense_vector_size')\n",
"fidle.override('batch_size', 'epochs', 'output_dir', 'fit_verbosity')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 2 - Retrieve data\n",
"\n",
"IMDb dataset can bet get directly from Keras - see [documentation](https://keras.io/api/datasets) \n",
"Note : Due to their nature, textual data can be somewhat complex.\n",
"\n",
"For more details about the management of this dataset, see notebook [IMDB1](01-One-hot-encoding.ipynb)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2.2 - Get dataset"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# ----- Retrieve x,y\n",
"#\n",
"start_char = 1 # Start of a sequence (padding is 0)\n",
"oov_char = 2 # Out-of-vocabulary\n",
"index_from = 3 # First word id\n",
"\n",
"(x_train, y_train), (x_test, y_test) = imdb.load_data( num_words = vocab_size, \n",
" skip_top = hide_most_frequently,\n",
" start_char = start_char, \n",
" oov_char = oov_char, \n",
" index_from = index_from)\n",
"\n",
"# ---- About\n",
"#\n",
"print(\"Max(x_train,x_test) : \", fidle.utils.rmax([x_train,x_test]) )\n",
"print(\"Min(x_train,x_test) : \", fidle.utils.rmin([x_train,x_test]) )\n",
"print(\"Len(x_train) : \", len(x_train))\n",
"print(\"Len(x_test) : \", len(x_test))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2.2 - Load dictionary\n",
"Not essential, but nice if you want to take a closer look at our reviews ;-)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# ---- Retrieve dictionary {word:index}, and encode it in ascii\n",
"# Shift the dictionary from +3\n",
"# Add <pad>, <start> and <unknown> tags\n",
"# Create a reverse dictionary : {index:word}\n",
"#\n",
"word_index = imdb.get_word_index()\n",
"word_index = {w:(i+index_from) for w,i in word_index.items()}\n",
"word_index.update( {'<pad>':0, '<start>':1, '<unknown>':2, '<undef>':3,} )\n",
"index_word = {index:word for word,index in word_index.items()} \n",
"\n",
"# ---- A nice function to transpose :\n",
"#\n",
"def dataset2text(review):\n",
" return ' '.join([index_word.get(i, '?') for i in review])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 3 - Preprocess the data (padding)\n",
"In order to be processed by an NN, all entries must have the **same length.** \n",
"We chose a review length of **review_len** \n",
"We will therefore complete them with a padding (of 0 as \\<pad\\>\\) "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"x_train = keras.preprocessing.sequence.pad_sequences(x_train,\n",
" value = 0,\n",
" padding = 'post',\n",
" maxlen = review_len)\n",
"\n",
"x_test = keras.preprocessing.sequence.pad_sequences(x_test,\n",
" value = 0 ,\n",
" padding = 'post',\n",
" maxlen = review_len)\n",
"\n",
"fidle.utils.subtitle('After padding :')\n",
"print(x_train[12])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Save dataset and dictionary (For future use but not mandatory)**"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# ---- Write dataset in a h5 file, could be usefull\n",
"#\n",
"fidle.utils.mkdir(output_dir)\n",
"\n",
"with h5py.File(f'{output_dir}/dataset_imdb.h5', 'w') as f:\n",
" f.create_dataset(\"x_train\", data=x_train)\n",
" f.create_dataset(\"y_train\", data=y_train)\n",
" f.create_dataset(\"x_test\", data=x_test)\n",
" f.create_dataset(\"y_test\", data=y_test)\n",
" print('Dataset h5 file saved.')\n",
"\n",
"with open(f'{output_dir}/word_index.json', 'w') as fp:\n",
" json.dump(word_index, fp)\n",
" print('Word to index saved.')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 4 - Build the model\n",
"\n",
"More documentation about this model functions :\n",
" - [Embedding](https://keras.io/api/layers/core_layers/embedding/)\n",
" - [GlobalAveragePooling1D](https://keras.io/api/layers/pooling_layers/global_average_pooling1d/)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model = keras.Sequential(name='Embedding model')\n",
"\n",
"model.add(keras.layers.Input( shape=(review_len,) ))\n",
"model.add(keras.layers.Embedding( input_dim = vocab_size,\n",
" output_dim = dense_vector_size))\n",
"model.add(keras.layers.GlobalAveragePooling1D())\n",
"model.add(keras.layers.Dense(dense_vector_size, activation='relu'))\n",
"model.add(keras.layers.Dense(1, activation='sigmoid'))\n",
"\n",
"model.compile( optimizer = 'adam',\n",
" loss = 'binary_crossentropy',\n",
" metrics = ['accuracy'])\n",
"\n",
"model.summary()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 5 - Train the model\n",
"### 5.1 Add Callbacks"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"os.makedirs(f'{run_dir}/models', mode=0o750, exist_ok=True)\n",
"save_dir = f'{run_dir}/models/best_model.keras'\n",
"\n",
"savemodel_callback = keras.callbacks.ModelCheckpoint( filepath=save_dir, monitor='val_accuracy', mode='max', save_best_only=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 5.2 - Train it"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%%time\n",
"\n",
"history = model.fit(x_train,\n",
" y_train,\n",
" epochs = epochs,\n",
" batch_size = batch_size,\n",
" validation_data = (x_test, y_test),\n",
" verbose = fit_verbosity,\n",
" callbacks = [savemodel_callback])\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 6 - Evaluate\n",
"### 6.1 - Training history"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"fidle.scrawler.history(history, save_as='02-history')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 6.2 - Reload and evaluate best model"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model = keras.models.load_model(f'{run_dir}/models/best_model.keras')\n",
"\n",
"# ---- Evaluate\n",
"score = model.evaluate(x_test, y_test, verbose=0)\n",
"\n",
"print('x_test / loss : {:5.4f}'.format(score[0]))\n",
"print('x_test / accuracy : {:5.4f}'.format(score[1]))\n",
"\n",
"values=[score[1], 1-score[1]]\n",
"fidle.scrawler.donut(values,[\"Accuracy\",\"Errors\"], title=\"#### Accuracy donut is :\", save_as='03-donut')\n",
"\n",
"# ---- Confusion matrix\n",
"\n",
"y_sigmoid = model.predict(x_test, verbose=fit_verbosity)\n",
"\n",
"y_pred = y_sigmoid.copy()\n",
"y_pred[ y_sigmoid< 0.5 ] = 0\n",
"y_pred[ y_sigmoid>=0.5 ] = 1 \n",
"\n",
"fidle.scrawler.confusion_matrix_txt(y_test,y_pred,labels=range(2))\n",
"fidle.scrawler.confusion_matrix(y_test,y_pred,range(2), figsize=(8, 8),normalize=False, save_as='04-confusion-matrix')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"fidle.end()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"<img width=\"80px\" src=\"../fidle/img/logo-paysage.svg\"></img>"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.9.2 ('fidle-env')",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.2"
},
"vscode": {
"interpreter": {
"hash": "b3929042cc22c1274d74e3e946c52b845b57cb6d84f2d591ffe0519b38e4896d"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}