Newer
Older
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img width=\"800px\" src=\"../fidle/img/00-Fidle-header-01.svg\"></img>\n",
"\n",
"# <!-- TITLE --> [IMDB4] - Reload embedded vectors\n",
"<!-- DESC --> Retrieving embedded vectors from our trained model\n",
"<!-- AUTHOR : Jean-Luc Parouty (CNRS/SIMaP) -->\n",
"\n",
"## Objectives :\n",
" - The objective is to retrieve and visualize our embedded vectors\n",
" - For this, we will use our **previously saved model**.\n",
"\n",
"## What we're going to do :\n",
"\n",
" - Retrieve our saved model\n",
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 1 - Init python stuff"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"\n",
"import tensorflow as tf\n",
"import tensorflow.keras as keras\n",
"import tensorflow.keras.datasets.imdb as imdb\n",
"\n",
"import matplotlib.pyplot as plt\n",
"import matplotlib\n",
"import pandas as pd\n",
"\n",
"import os,sys,h5py,json,re\n",
"\n",
"from importlib import reload\n",
"\n",
"run_id, run_dir, datasets_dir = fidle.init('IMDB4')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 1.2 - Parameters\n",
"The words in the vocabulary are classified from the most frequent to the rarest. \n",
"`vocab_size` is the number of words we will remember in our vocabulary (the other words will be considered as unknown). \n",
"`review_len` is the review length \n",
"`saved_models` where our models were previously saved \n",
"`dictionaries_dir` is where we will go to save our dictionaries. (./data is a good choice)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"dictionaries_dir = './data'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Override parameters (batch mode) - Just forget this cell"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"fidle.override('vocab_size', 'review_len', 'saved_models', 'dictionaries_dir')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 2 - Get the embedding vectors !"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2.1 - Load model and dictionaries"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model = keras.models.load_model(f'{saved_models}/models/best_model.h5')\n",
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
"print('Model loaded.')\n",
"\n",
"with open(f'{dictionaries_dir}/index_word.json', 'r') as fp:\n",
" index_word = json.load(fp)\n",
" index_word = { int(i):w for i,w in index_word.items() }\n",
" word_index = { w:int(i) for i,w in index_word.items() }\n",
" print('Dictionary loaded.')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2.2 - Retrieve embeddings"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"embeddings = model.layers[0].get_weights()[0]\n",
"print('Shape of embeddings : ',embeddings.shape)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 2.3 - Build a nice dictionary"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"word_embedding = { index_word[i]:embeddings[i] for i in range(vocab_size) }"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 3 - Have a look !\n",
"#### Show embedding of a word :"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"word_embedding['nice']"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Few usefull functions to play with"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Return a l2 distance between 2 words\n",
"#\n",
"def l2w(w1,w2):\n",
" v1=word_embedding[w1]\n",
" v2=word_embedding[w2]\n",
" return np.linalg.norm(v2-v1)\n",
"\n",
"# Show distance between 2 words \n",
"#\n",
"def show_l2(w1,w2):\n",
" print(f'\\nL2 between [{w1}] and [{w2}] : ',l2w(w1,w2))\n",
"\n",
"# Displays the 15 closest words to a given word\n",
"#\n",
"def neighbors(w1):\n",
" v1=word_embedding[w1]\n",
" dd={}\n",
" for i in range(4, 1000):\n",
" w2=index_word[i]\n",
" dd[w2]=l2w(w1,w2)\n",
" dd= {k: v for k, v in sorted(dd.items(), key=lambda item: item[1])}\n",
" print(f'\\nNeighbors of [{w1}] : ', list(dd.keys())[1:15])\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Examples"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"show_l2('nice', 'pleasant')\n",
"show_l2('nice', 'horrible')\n",
"\n",
"neighbors('horrible')\n",
"neighbors('great')\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"<img width=\"80px\" src=\"../fidle/img/00-Fidle-logo-01.svg\"></img>"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3.9.2 ('fidle-env')",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.2"
},
"vscode": {
"interpreter": {
"hash": "b3929042cc22c1274d74e3e946c52b845b57cb6d84f2d591ffe0519b38e4896d"
}
}
},
"nbformat": 4,
"nbformat_minor": 4
}