02-DNN-Regression-Premium.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img width=\"800px\" src=\"../fidle/img/00-Fidle-header-01.svg\"></img>\n",
    "\n",
    "# <!-- TITLE --> [BHP2] - Regression with a Dense Network (DNN) - Advanced code\n",
    "  <!-- DESC -->  More advanced example of DNN network code - BHPD dataset\n",
    "  <!-- AUTHOR : Jean-Luc Parouty (CNRS/SIMaP) -->\n",
    "\n",
    "## Objectives :\n",
    " - Predicts **housing prices** from a set of house features. \n",
    " - Understanding the principle and the architecture of a regression with a dense neural network with backup and restore of the trained model. \n",
    "\n",
    "The **[Boston Housing Dataset](https://www.cs.toronto.edu/~delve/data/boston/bostonDetail.html)** consists of price of houses in various places in Boston.  \n",
    "Alongside with price, the dataset also provide these information :\n",
    "\n",
    " - CRIM: This is the per capita crime rate by town\n",
    " - ZN: This is the proportion of residential land zoned for lots larger than 25,000 sq.ft\n",
    " - INDUS: This is the proportion of non-retail business acres per town\n",
    " - CHAS: This is the Charles River dummy variable (this is equal to 1 if tract bounds river; 0 otherwise)\n",
    " - NOX: This is the nitric oxides concentration (parts per 10 million)\n",
    " - RM: This is the average number of rooms per dwelling\n",
    " - AGE: This is the proportion of owner-occupied units built prior to 1940\n",
    " - DIS: This is the weighted distances to five Boston employment centers\n",
    " - RAD: This is the index of accessibility to radial highways\n",
    " - TAX: This is the full-value property-tax rate per 10,000 dollars\n",
    " - PTRATIO: This is the pupil-teacher ratio by town\n",
    " - B: This is calculated as 1000(Bk — 0.63)^2, where Bk is the proportion of people of African American descent by town\n",
    " - LSTAT: This is the percentage lower status of the population\n",
    " - MEDV: This is the median value of owner-occupied homes in 1000 dollars\n",
    "\n",
    "## What we're going to do :\n",
    "\n",
    " - (Retrieve data)\n",
    " - (Preparing the data)\n",
    " - (Build a model)\n",
    " - Train and save the model\n",
    " - Restore saved model\n",
    " - Evaluate the model\n",
    " - Make some predictions\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1 - Import and init"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       "\n",
       "div.warn {    \n",
       "    background-color: #fcf2f2;\n",
       "    border-color: #dFb5b4;\n",
       "    border-left: 5px solid #dfb5b4;\n",
       "    padding: 0.5em;\n",
       "    font-weight: bold;\n",
       "    font-size: 1.1em;;\n",
       "    }\n",
       "\n",
       "\n",
       "\n",
       "div.nota {    \n",
       "    background-color: #DAFFDE;\n",
       "    border-left: 5px solid #92CC99;\n",
       "    padding: 0.5em;\n",
       "    }\n",
       "\n",
       "div.todo:before { content:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSI1My44OTEyIiBoZWlnaHQ9IjE0My4zOTAyIiB2aWV3Qm94PSIwIDAgNTMuODkxMiAxNDMuMzkwMiI+PHRpdGxlPjAwLUJvYi10b2RvPC90aXRsZT48cGF0aCBkPSJNMjMuNDU2OCwxMTQuMzAxNmExLjgwNjMsMS44MDYzLDAsMSwxLDEuODE1NywxLjgyNEExLjgyMDksMS44MjA5LDAsMCwxLDIzLjQ1NjgsMTE0LjMwMTZabS0xMC42NjEyLDEuODIyQTEuODI3MiwxLjgyNzIsMCwxLDAsMTAuOTgsMTE0LjMsMS44MiwxLjgyLDAsMCwwLDEyLjc5NTYsMTE2LjEyMzZabS03LjcwNyw0LjU4NzR2LTVzLjQ4NjMtOS4xMjIzLDguMDIxNS0xMS45Njc1YTE5LjIwODIsMTkuMjA4MiwwLDAsMSw2LjA0ODYtMS4yNDU0LDE5LjE3NzgsMTkuMTc3OCwwLDAsMSw2LjA0ODcsMS4yNDc1YzcuNTM1MSwyLjgzNDcsOC4wMTc0LDExLjk2NzQsOC4wMTc0LDExLjk2NzR2NS4wMjM0bC4wMDQyLDcuNjgydjIuNGMuMDE2Ny4xOTkyLjAzMzYuMzkyMS4wMzM2LjU4NzEsMCwuMjEzOC0uMDE2OC40MTA5LS4wMzM2LjYzMzJ2LjA1ODdoLS4wMDg0YTguMzcxOSw4LjM3MTksMCwwLDEtNy4zNzM4LDcuNjU0N3MtLjk5NTMsMy42MzgtNi42OTMzLDMuNjM4LTYuNjkzNC0zLjYzOC02LjY5MzQtMy42MzhhOC4zNyw4LjM3LDAsMCwxLTcuMzcxNi03LjY1NDdINS4wODQzdi0uMDU4N2MtLjAxODktLjIyLS4wMjk0LS40MTk0LS4wMjk0LS42MzMyLDAtLjE5MjkuMDE2Ny0uMzgzNy4wMjk0LS41ODcxdi0yLjRtMTguMDkzNy00LjA0YTEuMTU2NSwxLjE1NjUsMCwxLDAtMi4zMTI2LDAsMS4xNTY0LDEuMTU2NCwwLDEsMCwyLjMxMjYsMFptNC4wODM0LDBhMS4xNTk1LDEuMTU5NSwwLDEsMC0xLjE2MzYsMS4xN0ExLjE3NSwxLjE3NSwwLDAsMCwyNy4yNjE0LDEyNC4zNzc5Wk05LjM3MzksMTE0LjYzNWMwLDMuMTA5MywyLjQxMzIsMy4zMSwyLjQxMzIsMy4zMWExMzMuOTI0MywxMzMuOTI0MywwLDAsMCwxNC43MzQ4LDBzMi40MTExLS4xOTI5LDIuNDExMS0zLjMxYTguMDc3Myw4LjA3NzMsMCwwLDAtMi40MTExLTUuNTUxOWMtNC41LTMuNTAzMy05LjkxMjYtMy41MDMzLTE0Ljc0MTEsMEE4LjA4NTEsOC4wODUxLDAsMCwwLDkuMzczOSwxMTQuNjM1WiIgc3R5bGU9ImZpbGw6IzAxMDEwMSIvPjxjaXJjbGUgY3g9IjMzLjE0MzYiIGN5PSIxMjQuNTM0IiByPSIzLjgzNjMiIHN0eWxlPSJmaWxsOiMwMTAxMDEiLz48cmVjdCB4PSIzNS42NjU5IiB5PSIxMTIuOTYyNSIgd2lkdGg9IjIuMDc3IiBoZWlnaHQ9IjEwLjU0NTgiIHRyYW5zZm9ybT0idHJhbnNsYXRlKDIxLjYgMjQxLjExMjEpIHJvdGF0ZSgtMTU1Ljc0NikiIHN0eWxlPSJmaWxsOiMwMTAxMDEiLz48Y2lyY2xlIGN4PSIzOC44NzA0IiBjeT0iMTEzLjQyNzkiIHI9IjIuNDA4NSIgc3R5bGU9ImZpbGw6IzAxMDEwMSIvPjxjaXJjbGUgY3g9IjUuMjI0OCIgY3k9IjEyNC41MzQiIHI9IjMuODM2MyIgc3R5bGU9ImZpbGw6IzAxMDEwMSIvPjxyZWN0IHg9IjEuNDE2NCIgeT0iMTI0LjYzMDEiIHdpZHRoPSIyLjA3NyIgaGVpZ2h0PSIxMC41NDU4IiB0cmFuc2Zvcm09InRyYW5zbGF0ZSg0LjkwOTcgMjU5LjgwNikgcm90YXRlKC0xODApIiBzdHlsZT0iZmlsbDojMDEwMTAxIi8+PGNpcmNsZSBjeD0iMi40MDkxIiBjeT0iMTM3LjA5OTYiIHI9IjIuNDA4NSIgc3R5bGU9ImZpbGw6IzAxMDEwMSIvPjxwYXRoIGQ9Ik0xOC4wNTExLDEwMC4xMDY2aC0uMDE0NlYxMDIuNjFoMi4zdi0yLjQyNzlhMi40MjI5LDIuNDIyOSwwLDEsMC0yLjI4NTQtLjA3NTVaIiBzdHlsZT0iZmlsbDojMDEwMTAxIi8+PHBhdGggZD0iTTM5LjQyMTQsMjcuMjU4djEuMDVBMTEuOTQ1MiwxMS45NDUyLDAsMCwwLDQ0LjU5NTQsNS43OWEuMjQ0OS4yNDQ5LDAsMCwxLS4wMjM1LS40MjI3TDQ2Ljc1LDMuOTUxNWEuMzg5Mi4zODkyLDAsMCwxLC40MjYyLDAsMTQuODQ0MiwxNC44NDQyLDAsMCwxLTcuNzU0MywyNy4yNTkxdjEuMDY3YS40NS40NSwwLDAsMS0uNzA0Ny4zNzU4bC0zLjg0MTktMi41MWEuNDUuNDUsMCwwLDEsMC0uNzUxNmwzLjg0MTktMi41MWEuNDUuNDUsMCwwLDEsLjY5NDYuMzc1OFpNNDMuMjMsMi41ODkyLDM5LjM4NzguMDc5NGEuNDUuNDUsMCwwLDAtLjcwNDYuMzc1OHYxLjA2N2ExNC44NDQyLDE0Ljg0NDIsMCwwLDAtNy43NTQzLDI3LjI1OTEuMzg5LjM4OSwwLDAsMCwuNDI2MSwwbDIuMTc3Ny0xLjQxOTNhLjI0NS4yNDUsMCwwLDAtLjAyMzUtLjQyMjgsMTEuOTQ1MSwxMS45NDUxLDAsMCwxLDUuMTc0LTIyLjUxNDZ2MS4wNWEuNDUuNDUsMCwwLDAsLjcwNDYuMzc1OGwzLjg1NTMtMi41MWEuNDUuNDUsMCwwLDAsMC0uNzUxNlpNMzkuMDUyMywxNC4yNDU4YTIuMTIwNiwyLjEyMDYsMCwxLDAsMi4xMjA2LDIuMTIwNmgwQTIuMTI0LDIuMTI0LDAsMCwwLDM5LjA1MjMsMTQuMjQ1OFptNi4wNzMyLTQuNzc4MS44MjU0LjgyNTVhMS4wNTY4LDEuMDU2OCwwLDAsMSwuMTE3NSwxLjM0MjFsLS44MDIsMS4xNDQyYTcuMTAxOCw3LjEwMTgsMCwwLDEsLjcxMTQsMS43MTEybDEuMzc1Ny4yNDE2YTEuMDU2OSwxLjA1NjksMCwwLDEsLjg3NTcsMS4wNHYxLjE2NDNhMS4wNTY5LDEuMDU2OSwwLDAsMS0uODc1NywxLjA0bC0xLjM3MjQuMjQxNkE3LjExLDcuMTEsMCwwLDEsNDUuMjcsMTkuOTNsLjgwMTksMS4xNDQyYTEuMDU3LDEuMDU3LDAsMCwxLS4xMTc0LDEuMzQyMmwtLjgyODguODQ4OWExLjA1NywxLjA1NywwLDAsMS0xLjM0MjEuMTE3NGwtMS4xNDQyLS44MDE5YTcuMTMzOCw3LjEzMzgsMCwwLDEtMS43MTEzLjcxMTNsLS4yNDE2LDEuMzcyNGExLjA1NjgsMS4wNTY4LDAsMCwxLTEuMDQuODc1N0gzOC40Njg0YTEuMDU2OCwxLjA1NjgsMCwwLDEtMS4wNC0uODc1N2wtLjI0MTYtMS4zNzI0YTcuMTM1NSw3LjEzNTUsMCwwLDEtMS43MTEzLS43MTEzbC0xLjE0NDEuODAxOWExLjA1NzEsMS4wNTcxLDAsMCwxLTEuMzQyMi0uMTE3NGwtLjgzNTUtLjgyNTVhMS4wNTcsMS4wNTcsMCwwLDEtLjExNzQtMS4zNDIxbC44MDE5LTEuMTQ0MmE3LjEyMSw3LjEyMSwwLDAsMS0uNzExMy0xLjcxMTJsLTEuMzcyNC0uMjQxNmExLjA1NjksMS4wNTY5LDAsMCwxLS44NzU3LTEuMDRWMTUuNzgyNmExLjA1NjksMS4wNTY5LDAsMCwxLC44NzU3LTEuMDRsMS4zNzU3LS4yNDE2YTcuMTEsNy4xMSwwLDAsMSwuNzExNC0xLjcxMTJsLS44MDItMS4xNDQyYTEuMDU3LDEuMDU3LDAsMCwxLC4xMTc1LTEuMzQyMmwuODI1NC0uODI1NEExLjA1NjgsMS4wNTY4LDAsMCwxLDM0LjMyNDUsOS4zNmwxLjE0NDIuODAxOUE3LjEzNTUsNy4xMzU1LDAsMCwxLDM3LjE4LDkuNDUxbC4yNDE2LTEuMzcyNGExLjA1NjgsMS4wNTY4LDAsMCwxLDEuMDQtLjg3NTdoMS4xNjc3YTEuMDU2OSwxLjA1NjksMCwwLDEsMS4wNC44NzU3bC4yNDE2LDEuMzcyNGE3LjEyNSw3LjEyNSwwLDAsMSwxLjcxMTIuNzExM0w0My43NjY2LDkuMzZBMS4wNTY5LDEuMDU2OSwwLDAsMSw0NS4xMjU1LDkuNDY3N1ptLTIuMDMsNi44OTg3QTQuMDQzMyw0LjA0MzMsMCwxLDAsMzkuMDUyMywyMC40MWgwQTQuMDQ2NSw0LjA0NjUsMCwwLDAsNDMuMDk1NSwxNi4zNjY0WiIgc3R5bGU9ImZpbGw6I2UxMjIyOSIvPjxwb2x5Z29uIHBvaW50cz0iMzkuNDEzIDM0Ljc1NyAzOS41MzcgMzQuNzU3IDM5LjY3NSAzNC43NTcgMzkuNjc1IDEwOS41MSAzOS41MzcgMTA5LjUxIDM5LjQxMyAxMDkuNTEgMzkuNDEzIDM0Ljc1NyAzOS40MTMgMzQuNzU3IiBzdHlsZT0iZmlsbDpub25lO3N0cm9rZTojOTk5O3N0cm9rZS1saW5lY2FwOnJvdW5kO3N0cm9rZS1taXRlcmxpbWl0OjEwO3N0cm9rZS13aWR0aDowLjMwODg1NDQ1MDU2MDE2MThweDtmaWxsLXJ1bGU6ZXZlbm9kZCIvPjwvc3ZnPg==);\n",
       "    float:left;\n",
       "    margin-right:20px;\n",
       "    margin-top:-20px;\n",
       "    margin-bottom:20px;\n",
       "}\n",
       "div.todo{\n",
       "    font-weight: bold;\n",
       "    font-size: 1.1em;\n",
       "    margin-top:40px;\n",
       "}\n",
       "div.todo ul{\n",
       "    margin: 0.2em;\n",
       "}\n",
       "div.todo li{\n",
       "    margin-left:60px;\n",
       "    margin-top:0;\n",
       "    margin-bottom:0;\n",
       "}\n",
       "\n",
       "\n",
       "</style>\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "FIDLE 2020 - Practical Work Module\n",
      "Version              : 0.57 DEV\n",
      "Run time             : Wednesday 9 September 2020, 14:44:23\n",
      "TensorFlow version   : 2.2.0\n",
      "Keras version        : 2.3.0-tf\n",
      "Current place        : Fidle at IDRIS\n",
      "Dataset dir          : /gpfswork/rech/mlh/commun/datasets\n",
      "Update keras cache   : Done\n"
     ]
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import pandas as pd\n",
    "import os,sys\n",
    "\n",
    "from IPython.display import Markdown\n",
    "from importlib import reload\n",
    "\n",
    "sys.path.append('..')\n",
    "import fidle.pwk as ooo\n",
    "\n",
    "place, datasets_dir = ooo.init()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2 - Retrieve data\n",
    "\n",
    "### 2.1 - Option 1  : From Keras\n",
    "Boston housing is a famous historic dataset, so we can get it directly from [Keras datasets](https://www.tensorflow.org/api_docs/python/tf/keras/datasets)  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# (x_train, y_train), (x_test, y_test) = keras.datasets.boston_housing.load_data(test_split=0.2, seed=113)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2 - Option 2 : From a csv file\n",
    "More fun !"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style  type=\"text/css\" >\n",
       "</style><table id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3\" ><thead>    <tr>        <th class=\"blank level0\" ></th>        <th class=\"col_heading level0 col0\" >crim</th>        <th class=\"col_heading level0 col1\" >zn</th>        <th class=\"col_heading level0 col2\" >indus</th>        <th class=\"col_heading level0 col3\" >chas</th>        <th class=\"col_heading level0 col4\" >nox</th>        <th class=\"col_heading level0 col5\" >rm</th>        <th class=\"col_heading level0 col6\" >age</th>        <th class=\"col_heading level0 col7\" >dis</th>        <th class=\"col_heading level0 col8\" >rad</th>        <th class=\"col_heading level0 col9\" >tax</th>        <th class=\"col_heading level0 col10\" >ptratio</th>        <th class=\"col_heading level0 col11\" >b</th>        <th class=\"col_heading level0 col12\" >lstat</th>        <th class=\"col_heading level0 col13\" >medv</th>    </tr></thead><tbody>\n",
       "                <tr>\n",
       "                        <th id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3level0_row0\" class=\"row_heading level0 row0\" >0</th>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col0\" class=\"data row0 col0\" >0.01</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col1\" class=\"data row0 col1\" >18.00</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col2\" class=\"data row0 col2\" >2.31</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col3\" class=\"data row0 col3\" >0.00</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col4\" class=\"data row0 col4\" >0.54</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col5\" class=\"data row0 col5\" >6.58</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col6\" class=\"data row0 col6\" >65.20</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col7\" class=\"data row0 col7\" >4.09</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col8\" class=\"data row0 col8\" >1.00</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col9\" class=\"data row0 col9\" >296.00</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col10\" class=\"data row0 col10\" >15.30</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col11\" class=\"data row0 col11\" >396.90</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col12\" class=\"data row0 col12\" >4.98</td>\n",
       "                        <td id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3row0_col13\" class=\"data row0 col13\" >24.00</td>\n",
       "            </tr>\n",
       "            <tr>\n",
       "                        <th id=\"T_3037227a_f29a_11ea_b9ca_0cc47af5a3b3level0_row1\" class=\"row_heading level0 row1\" >1</th>\n",