01-Linear-Regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img width=\"800px\" src=\"../fidle/img/00-Fidle-header-01.svg\"></img>\n",
    "\n",
    "# <!-- TITLE --> [LINR1] - Linear regression with direct resolution\n",
    "<!-- DESC --> Direct determination of linear regression \n",
    "<!-- AUTHOR : Jean-Luc Parouty (CNRS/SIMaP) -->\n",
    "\n",
    "## Objectives :\n",
    " - Just one, the illustration of a direct resolution :-)\n",
    "\n",
    "## What we're going to do :\n",
    "\n",
    "Equation : $ Y = X.\\theta + N$  \n",
    "Where N is a noise vector\n",
    "and $\\theta = (a,b)$ a vector as y = a.x + b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 1 - Import and init"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<style>\n",
       "\n",
       "div.warn {    \n",
       "    background-color: #fcf2f2;\n",
       "    border-color: #dFb5b4;\n",
       "    border-left: 5px solid #dfb5b4;\n",
       "    padding: 0.5em;\n",
       "    font-weight: bold;\n",
       "    font-size: 1.1em;;\n",
       "    }\n",
       "\n",
       "\n",
       "\n",
       "div.nota {    \n",
       "    background-color: #DAFFDE;\n",
       "    border-left: 5px solid #92CC99;\n",
       "    padding: 0.5em;\n",
       "    }\n",
       "\n",
       "div.todo:before { content:url(data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSI1My44OTEyIiBoZWlnaHQ9IjE0My4zOTAyIiB2aWV3Qm94PSIwIDAgNTMuODkxMiAxNDMuMzkwMiI+PHRpdGxlPjAwLUJvYi10b2RvPC90aXRsZT48cGF0aCBkPSJNMjMuNDU2OCwxMTQuMzAxNmExLjgwNjMsMS44MDYzLDAsMSwxLDEuODE1NywxLjgyNEExLjgyMDksMS44MjA5LDAsMCwxLDIzLjQ1NjgsMTE0LjMwMTZabS0xMC42NjEyLDEuODIyQTEuODI3MiwxLjgyNzIsMCwxLDAsMTAuOTgsMTE0LjMsMS44MiwxLjgyLDAsMCwwLDEyLjc5NTYsMTE2LjEyMzZabS03LjcwNyw0LjU4NzR2LTVzLjQ4NjMtOS4xMjIzLDguMDIxNS0xMS45Njc1YTE5LjIwODIsMTkuMjA4MiwwLDAsMSw2LjA0ODYtMS4yNDU0LDE5LjE3NzgsMTkuMTc3OCwwLDAsMSw2LjA0ODcsMS4yNDc1YzcuNTM1MSwyLjgzNDcsOC4wMTc0LDExLjk2NzQsOC4wMTc0LDExLjk2NzR2NS4wMjM0bC4wMDQyLDcuNjgydjIuNGMuMDE2Ny4xOTkyLjAzMzYuMzkyMS4wMzM2LjU4NzEsMCwuMjEzOC0uMDE2OC40MTA5LS4wMzM2LjYzMzJ2LjA1ODdoLS4wMDg0YTguMzcxOSw4LjM3MTksMCwwLDEtNy4zNzM4LDcuNjU0N3MtLjk5NTMsMy42MzgtNi42OTMzLDMuNjM4LTYuNjkzNC0zLjYzOC02LjY5MzQtMy42MzhhOC4zNyw4LjM3LDAsMCwxLTcuMzcxNi03LjY1NDdINS4wODQzdi0uMDU4N2MtLjAxODktLjIyLS4wMjk0LS40MTk0LS4wMjk0LS42MzMyLDAtLjE5MjkuMDE2Ny0uMzgzNy4wMjk0LS41ODcxdi0yLjRtMTguMDkzNy00LjA0YTEuMTU2NSwxLjE1NjUsMCwxLDAtMi4zMTI2LDAsMS4xNTY0LDEuMTU2NCwwLDEsMCwyLjMxMjYsMFptNC4wODM0LDBhMS4xNTk1LDEuMTU5NSwwLDEsMC0xLjE2MzYsMS4xN0ExLjE3NSwxLjE3NSwwLDAsMCwyNy4yNjE0LDEyNC4zNzc5Wk05LjM3MzksMTE0LjYzNWMwLDMuMTA5MywyLjQxMzIsMy4zMSwyLjQxMzIsMy4zMWExMzMuOTI0MywxMzMuOTI0MywwLDAsMCwxNC43MzQ4LDBzMi40MTExLS4xOTI5LDIuNDExMS0zLjMxYTguMDc3Myw4LjA3NzMsMCwwLDAtMi40MTExLTUuNTUxOWMtNC41LTMuNTAzMy05LjkxMjYtMy41MDMzLTE0Ljc0MTEsMEE4LjA4NTEsOC4wODUxLDAsMCwwLDkuMzczOSwxMTQuNjM1WiIgc3R5bGU9ImZpbGw6IzAxMDEwMSIvPjxjaXJjbGUgY3g9IjMzLjE0MzYiIGN5PSIxMjQuNTM0IiByPSIzLjgzNjMiIHN0eWxlPSJmaWxsOiMwMTAxMDEiLz48cmVjdCB4PSIzNS42NjU5IiB5PSIxMTIuOTYyNSIgd2lkdGg9IjIuMDc3IiBoZWlnaHQ9IjEwLjU0NTgiIHRyYW5zZm9ybT0idHJhbnNsYXRlKDIxLjYgMjQxLjExMjEpIHJvdGF0ZSgtMTU1Ljc0NikiIHN0eWxlPSJmaWxsOiMwMTAxMDEiLz48Y2lyY2xlIGN4PSIzOC44NzA0IiBjeT0iMTEzLjQyNzkiIHI9IjIuNDA4NSIgc3R5bGU9ImZpbGw6IzAxMDEwMSIvPjxjaXJjbGUgY3g9IjUuMjI0OCIgY3k9IjEyNC41MzQiIHI9IjMuODM2MyIgc3R5bGU9ImZpbGw6IzAxMDEwMSIvPjxyZWN0IHg9IjEuNDE2NCIgeT0iMTI0LjYzMDEiIHdpZHRoPSIyLjA3NyIgaGVpZ2h0PSIxMC41NDU4IiB0cmFuc2Zvcm09InRyYW5zbGF0ZSg0LjkwOTcgMjU5LjgwNikgcm90YXRlKC0xODApIiBzdHlsZT0iZmlsbDojMDEwMTAxIi8+PGNpcmNsZSBjeD0iMi40MDkxIiBjeT0iMTM3LjA5OTYiIHI9IjIuNDA4NSIgc3R5bGU9ImZpbGw6IzAxMDEwMSIvPjxwYXRoIGQ9Ik0xOC4wNTExLDEwMC4xMDY2aC0uMDE0NlYxMDIuNjFoMi4zdi0yLjQyNzlhMi40MjI5LDIuNDIyOSwwLDEsMC0yLjI4NTQtLjA3NTVaIiBzdHlsZT0iZmlsbDojMDEwMTAxIi8+PHBhdGggZD0iTTM5LjQyMTQsMjcuMjU4djEuMDVBMTEuOTQ1MiwxMS45NDUyLDAsMCwwLDQ0LjU5NTQsNS43OWEuMjQ0OS4yNDQ5LDAsMCwxLS4wMjM1LS40MjI3TDQ2Ljc1LDMuOTUxNWEuMzg5Mi4zODkyLDAsMCwxLC40MjYyLDAsMTQuODQ0MiwxNC44NDQyLDAsMCwxLTcuNzU0MywyNy4yNTkxdjEuMDY3YS40NS40NSwwLDAsMS0uNzA0Ny4zNzU4bC0zLjg0MTktMi41MWEuNDUuNDUsMCwwLDEsMC0uNzUxNmwzLjg0MTktMi41MWEuNDUuNDUsMCwwLDEsLjY5NDYuMzc1OFpNNDMuMjMsMi41ODkyLDM5LjM4NzguMDc5NGEuNDUuNDUsMCwwLDAtLjcwNDYuMzc1OHYxLjA2N2ExNC44NDQyLDE0Ljg0NDIsMCwwLDAtNy43NTQzLDI3LjI1OTEuMzg5LjM4OSwwLDAsMCwuNDI2MSwwbDIuMTc3Ny0xLjQxOTNhLjI0NS4yNDUsMCwwLDAtLjAyMzUtLjQyMjgsMTEuOTQ1MSwxMS45NDUxLDAsMCwxLDUuMTc0LTIyLjUxNDZ2MS4wNWEuNDUuNDUsMCwwLDAsLjcwNDYuMzc1OGwzLjg1NTMtMi41MWEuNDUuNDUsMCwwLDAsMC0uNzUxNlpNMzkuMDUyMywxNC4yNDU4YTIuMTIwNiwyLjEyMDYsMCwxLDAsMi4xMjA2LDIuMTIwNmgwQTIuMTI0LDIuMTI0LDAsMCwwLDM5LjA1MjMsMTQuMjQ1OFptNi4wNzMyLTQuNzc4MS44MjU0LjgyNTVhMS4wNTY4LDEuMDU2OCwwLDAsMSwuMTE3NSwxLjM0MjFsLS44MDIsMS4xNDQyYTcuMTAxOCw3LjEwMTgsMCwwLDEsLjcxMTQsMS43MTEybDEuMzc1Ny4yNDE2YTEuMDU2OSwxLjA1NjksMCwwLDEsLjg3NTcsMS4wNHYxLjE2NDNhMS4wNTY5LDEuMDU2OSwwLDAsMS0uODc1NywxLjA0bC0xLjM3MjQuMjQxNkE3LjExLDcuMTEsMCwwLDEsNDUuMjcsMTkuOTNsLjgwMTksMS4xNDQyYTEuMDU3LDEuMDU3LDAsMCwxLS4xMTc0LDEuMzQyMmwtLjgyODguODQ4OWExLjA1NywxLjA1NywwLDAsMS0xLjM0MjEuMTE3NGwtMS4xNDQyLS44MDE5YTcuMTMzOCw3LjEzMzgsMCwwLDEtMS43MTEzLjcxMTNsLS4yNDE2LDEuMzcyNGExLjA1NjgsMS4wNTY4LDAsMCwxLTEuMDQuODc1N0gzOC40Njg0YTEuMDU2OCwxLjA1NjgsMCwwLDEtMS4wNC0uODc1N2wtLjI0MTYtMS4zNzI0YTcuMTM1NSw3LjEzNTUsMCwwLDEtMS43MTEzLS43MTEzbC0xLjE0NDEuODAxOWExLjA1NzEsMS4wNTcxLDAsMCwxLTEuMzQyMi0uMTE3NGwtLjgzNTUtLjgyNTVhMS4wNTcsMS4wNTcsMCwwLDEtLjExNzQtMS4zNDIxbC44MDE5LTEuMTQ0MmE3LjEyMSw3LjEyMSwwLDAsMS0uNzExMy0xLjcxMTJsLTEuMzcyNC0uMjQxNmExLjA1NjksMS4wNTY5LDAsMCwxLS44NzU3LTEuMDRWMTUuNzgyNmExLjA1NjksMS4wNTY5LDAsMCwxLC44NzU3LTEuMDRsMS4zNzU3LS4yNDE2YTcuMTEsNy4xMSwwLDAsMSwuNzExNC0xLjcxMTJsLS44MDItMS4xNDQyYTEuMDU3LDEuMDU3LDAsMCwxLC4xMTc1LTEuMzQyMmwuODI1NC0uODI1NEExLjA1NjgsMS4wNTY4LDAsMCwxLDM0LjMyNDUsOS4zNmwxLjE0NDIuODAxOUE3LjEzNTUsNy4xMzU1LDAsMCwxLDM3LjE4LDkuNDUxbC4yNDE2LTEuMzcyNGExLjA1NjgsMS4wNTY4LDAsMCwxLDEuMDQtLjg3NTdoMS4xNjc3YTEuMDU2OSwxLjA1NjksMCwwLDEsMS4wNC44NzU3bC4yNDE2LDEuMzcyNGE3LjEyNSw3LjEyNSwwLDAsMSwxLjcxMTIuNzExM0w0My43NjY2LDkuMzZBMS4wNTY5LDEuMDU2OSwwLDAsMSw0NS4xMjU1LDkuNDY3N1ptLTIuMDMsNi44OTg3QTQuMDQzMyw0LjA0MzMsMCwxLDAsMzkuMDUyMywyMC40MWgwQTQuMDQ2NSw0LjA0NjUsMCwwLDAsNDMuMDk1NSwxNi4zNjY0WiIgc3R5bGU9ImZpbGw6I2UxMjIyOSIvPjxwb2x5Z29uIHBvaW50cz0iMzkuNDEzIDM0Ljc1NyAzOS41MzcgMzQuNzU3IDM5LjY3NSAzNC43NTcgMzkuNjc1IDEwOS41MSAzOS41MzcgMTA5LjUxIDM5LjQxMyAxMDkuNTEgMzkuNDEzIDM0Ljc1NyAzOS40MTMgMzQuNzU3IiBzdHlsZT0iZmlsbDpub25lO3N0cm9rZTojOTk5O3N0cm9rZS1saW5lY2FwOnJvdW5kO3N0cm9rZS1taXRlcmxpbWl0OjEwO3N0cm9rZS13aWR0aDowLjMwODg1NDQ1MDU2MDE2MThweDtmaWxsLXJ1bGU6ZXZlbm9kZCIvPjwvc3ZnPg==);\n",
       "    float:left;\n",
       "    margin-right:20px;\n",
       "    margin-top:-20px;\n",
       "    margin-bottom:20px;\n",
       "}\n",
       "div.todo{\n",
       "    font-weight: bold;\n",
       "    font-size: 1.1em;\n",
       "    margin-top:40px;\n",
       "}\n",
       "div.todo ul{\n",
       "    margin: 0.2em;\n",
       "}\n",
       "div.todo li{\n",
       "    margin-left:60px;\n",
       "    margin-top:0;\n",
       "    margin-bottom:0;\n",
       "}\n",
       "\n",
       "div .comment{\n",
       "    font-size:0.8em;\n",
       "    color:#696969;\n",
       "}\n",
       "\n",
       "\n",
       "\n",
       "</style>\n",
       "\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "**FIDLE 2020 - Practical Work Module**"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Version              : 0.6.1 DEV\n",
      "Notebook id          : LINR1\n",
      "Run time             : Thursday 17 December 2020, 16:30:24\n",
      "TensorFlow version   : 2.1.0\n",
      "Keras version        : 2.2.4-tf\n",
      "Datasets dir         : /gpfswork/rech/mlh/uja62cb/datasets\n",
      "Running mode         : full\n",
      "Update keras cache   : False\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import math\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import sys\n",
    "\n",
    "sys.path.append('..')\n",
    "import fidle.pwk as pwk\n",
    "\n",
    "datasets_dir = pwk.init('LINR1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 2 - Retrieve a set of points"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ---- Paramètres\n",
    "nb    = 100     # Nombre de points\n",
    "xmin  = 0       # Distribution / x\n",
    "xmax  = 10\n",
    "a     = 4       # Distribution / y\n",
    "b     = 2       # y= a.x + b (+ bruit)\n",
    "noise = 7       # bruit\n",
    "\n",
    "theta = np.array([[a],[b]])\n",
    "\n",
    "# ---- Vecteur X  (1,x) x nb\n",
    "#      la premiere colonne est a 1 afin que X.theta <=> 1.b + x.a\n",
    "\n",
    "Xc1 = np.ones((nb,1))\n",
    "Xc2 = np.random.uniform(xmin,xmax,(nb,1))\n",
    "X = np.c_[ Xc1, Xc2 ]\n",
    "\n",
    "# ---- Noise\n",
    "# N = np.random.uniform(-noise,noise,(nb,1))\n",
    "N = noise * np.random.normal(0,1,(nb,1))\n",
    "\n",
    "# ---- Vecteur Y\n",
    "Y = (X @ theta) + N\n",
    "\n",
    "# print(\"X:\\n\",X,\"\\nY:\\n \",Y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Show it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsIAAAFrCAYAAADb6e9iAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3dfZTld10f8PfsZNi11piA7q5GOJJsloCPUbfZUqKAWj2nCqgohCg+HAX9HgrVnlMItkeRNqn1obbo14pYKbJGWkUKRzxaVIhIN10tCEcIm00M4GJmEVhWy9llM3P7x93Bu5PZ2bkz9+H38Hqdw1nmzsy938l9+L2/n+/TwmAwCAAA9M2ueTcAAADmQRAGAKCXBGEAAHpJEAYAoJcEYQAAemkuQbiUMiil2K4CAIC5uWLOjy8MAwAwbQsb3WhqBAAAvSQIAwDQS4IwAAC9JAgDANBLgjAAAL0kCAMA0EuCMAAAvSQIAwDQS4IwAAC9JAgDANBL8z5iGQCAbVhZHeTYiVO578EzuW7/lTl0YG8Wd214kjCXIAgDALTMyuogLz1yd+45eTrnzq9k99Jibrjmqtx+603C8BhMjQAAaJljJ07lnpOnc/b8SgZJzp5fyT0nT+fYiVPzblqrCMIAAC1z34Nncu78ykW3nTu/kvuXz8ypRe0kCAMAtMx1+6/M7qXFi27bvbSYa/ddOacWtZMgDADQMocO7M0N11yVPUuLWUiy58Ic4UMH9s67aa1isRwAQMss7lrI7bfelGMnTuX+5TO5dp9dI7ZDEAYAaKHFXQs5fHBfDh/cN++mtJapEQAA9JIgDABALwnCAAD0kiAMAEAvCcIAAPSSIAwAQC8JwgAA9JIgDABALwnCAAD0kiAMAEAvCcIAAPTSFfNuAABAX6ysDnLsxKnc9+CZXLf/yhw6sDeLuxbm3azeEoQBAGZgZXWQlx65O/ecPJ1z51eye2kxN1xzVW6/9SZheE5MjQAAmIFjJ07lnpOnc/b8SgZJzp5fyT0nT+fYiVPzblpvCcIAADNw34Nncu78ykW3nTu/kvuXz8ypRQjCAAAzcN3+K7N7afGi23YvLebafVfOqUUIwgAAM3DowN7ccM1V2bO0mIUkey7MET50YO+8m9ZbFssBAMzA4q6F3H7rTTl24lTuXz6Ta/fZNWLeBGEAgBlZ3LWQwwf35fDBffNuCjE1AgCAnhKEAQDoJUEYAIBeEoQBAOglQRgAgF4ShAEA6CVBGACAXhKEAQDoJUEYAIBeEoQBAOglQRgAgF4ShAEA6CVBGACAXhKEAQDoJUEYAIBeEoQBAOglQRgAgF4ShAEA6CVBGACAXhKEAQDoJUEYAIBeEoQBAOglQRgAgF4ShAEA6CVBGACAXhKEAQDoJUEYAIBeEoQBAOglQRgAgF4ShAEA6CVBGACAXhKEAQDoJUEYAIBeEoQBAOglQRgAgF4ShAEA6CVBGACAXrpi3g0AgKZZWR3k2IlTue/BM7lu/5U5dGBvFnctzLtZwIQJwgAwYmV1kJceuTv3nDydc+dXsntpMTdcc1Vuv/UmYRg6xtQIABhx7MSp3HPydM6eX8kgydnzK7nn5OkcO3Fq3k0DJkwQBoAR9z14JufOr1x027nzK7l/+cycWgRMiyAMACOu239ldi8tXnTb7qXFXLvvyjm1CJgWQRgARhw6sDc3XHNV9iwtZiHJngtzhA8d2DvvpgETZrEcAIxY3LWQ22+9KcdOnMr9y2dy7T67RkBXCcIAsM7iroUcPrgvhw/um3dTgCkyNQIAgF4ShAEA6CVTIwBoNafAAdslCAPQWk6BA3bC1AgAWsspcMBOCMIAtJZT4ICdEIQBaC2nwAE7IQgD0FpOgQN2wmI5AFrLKXDATgjCALSaU+CA7TI1AgCAXlIRBoCWcYgITIYgDAAt4hARmBxTIwCgRRwiApMjCANAizhEBCZHEAaAFnGICEyOIAwALeIQEZgci+UAoEUcIgKTIwgDQMs4RAQmw9QIAAB6SRAGAKCXBGEAAHpJEAYAoJcEYQAAekkQBgCglwRhAAB6yT7CAHTWyuogx06cyn0Pnsl1+x08ATvVtfeUIAzAXEz7grqyOshLj9yde06ezrnzK9l94Sji22+9qdUXbpiXLr6nBGEAZm4WF9RjJ07lnpOnc/b8SpLk7PmV3HPydI6dOOVENtiG7b6nmlxFFoQBmLlZhNT7HjyTcxfuf8258yu5f/mMIAzbsJ33VNOryBbLAbAjK6uDHD2+nCN33Zujx5ezsjq47O9sdkGdlOv2X5ndS4sX3bZ7aTHX7rtyYo8BfbKd99Rop3eQizu9TaAiDMC2bbfas3ZBPTsShicdUg8d2JsbrrnqYW07dGDvxB4D+mQ776mmj8wIwgBs23anOMwipC7uWsjtt96UYydO5f7lM7l2X7PmJjI7TZ6j2ibbeU/NotO7E4IwANu23WrPOBfUnYSYxV0LOXxw39wrT4LY/DR9jupGmvx6Gfc91fSRGUEYgG3bSbVnKxfUNoaY9brwN7RZ23YP6drrpekjMxbLAbBta9WePUuLWUiyZwfVno0W3TV9oc1WdOFvaLNZLMycpC6+XtY6vc+5+focPrivMSE4UREGYAcmVe25VBXsix/zyEYvtNmKeS4WavIQ+6w0fY7qek1fXNY1gjAAOzKJebiXGr5+wqOvblWI2ci8gti4Q+xdDc1Nn6O6XtuCe9sJwgDM3aWqYIu7FloVYjYyryA2ztzYrs1LHdX0OarrtS24t50gDMDcXaoKdmD/Z+eWJ13fmhCzkXkFsXGG2Nu2oGxcTdk9ZCvaFtzbThAGYO42q4K1KcRcyjz+hnGG2M1LbZYuvObbQhAGYO5UwSZvnCF281LpK0F4zrq6OAFgXKpgkzVO58K8VPpKEJ6jLi9OAGD+ttq5UJGnrwThOer64gQA2kNFvlu2M+L8qYdWc+fb7817P/TxPOHRV+eWJ12fR1zR7bPXBOE5sjgBAJi07Yw4f+qh1dzyH/9X/u7sQ0mSdz3w0bzx2AO584e/vtNhuLt/WQusLU4YZXECALAT2zmm+c633/vpELzm784+lDvffu+UWztfgvAcrS1O2LO0mIUkeyxOAAB2aLMR50t574c+PtbtXWFqxBxZnMDl2FUEgHFtZzu8Jzz66rzrgY9ueHuXCcJzZnECl7KyOshtR+7O+/7q4/nUQ6t5xBW78vgvuDp32FUEgE1sZzu8W550fd547IGLpkf8wz1X5JYnXT+LJs+NIAwNdfe9y3nPBz6a1cHw6089tJr3fOCjufve5Tzxcfvn2zgAGms7I86PuGJX7vzhr7drBNAMd733rz8dgtesDpI/fu9fC8IAbGo7I86PuGJXvvvJj5tiq5pnx0G4lPI5SW5O8skkb6m1rlzmV4AtMPkBAKZry/XuUsoPlVLuLqU8cuS2r0zyviS/meTNSd5RSvnMyTcT+ufmx39eFtal4YWF4e0AwM6NM/HjWUkGtdaPjdz2U0muTvKrGQbhQ0l+cHLNg/666eC+fOljHpmlxeHbdGlxV770MY/MTRZWAkzVyuogR48v58hd9+bo8eWsrJ+nRmeMMzXi+iS/s/bFhSkRX5PkVbXW51+47e4kz0nyM5NsJPTR4q6F3PGdh22vBzBD2zmVjfYaJwg/KsnokST/5MK/vz1y2x8n+Z4dtgm4wPZ6ALM1eipbcvGpbD6Lu2ecqREfS/I5I19/TZLVJO8YuW2QZM8E2sUEGNoBgPFs51Q22mucivD7knxzKeVHk6xkOGf4WK119JXxhUkenFzz2C5DOwAwvu2cykZ7jVMR/k9JPi/JXyX5UJL9SeraN0spi0melOTPJ9lAtmd0aGeQi4d2YJKMPABdsnYq256lxSwk2bOFU9lory1XhGutbyyl/GCS51246Uit9bUjP/J1GU6L+L0Jto9t2mxoxxynZltZHeTYiVO578EzuW5/sxfIGXkAumY7p7K1SZuuMbMw1oEatdZXJnnlJb73exlupUYDGNppp7YFS4tKgC7q6kLltl1jZqHbB0j3mKGddmrblBaLSgDao23XmFm4ZEW4lPKYC//3ZK11ZeTry6q1fnDHLWNHuj6001Vtm9Ji5AGgPdp2jZmFzaZGPJDhdmiPT3J85OvLGVzmfpmRrg7tdFnbguXayMP6YTYjDwDN07ZrzCxsFlhfk2Go/cS6r4EpaVuwNPIA0B5tu8bMwsJgMPtsW0oZJEmt9XI/Cr2ztqJXsARg0rZ7jenAbhMbNtYUBmgYU1oAmJbtXGO6vNvElneNKKXcvMWfe9H2mwMAJA6roTm6vNvEOBXhPyylvKzW+m83+mYp5eokr07yTRmeQgcAbEOXK3C0T5d3mxhnH+ETSV5WSnlLKWX/6DdKKU9M8q4k35zkDRNsHwD0TpcrcF3V5Qr+2m4To7qy28Q4Qfgrk7w2yVOTvLOU8vVJUkq5Lclbk+xN8oJa67dNupEAMG1NCjLjHlbTpLb30VoF/47XvzO/9rbjueP178xLj9zdmeehy4d0bXlqRK31k0m+u5TyB0l+IcnvllLem+SLMtxn+Fm11ndPp5kAMD1Nm4owzn6vTWt7H3X9uPkub5U59hHLtdbXJHnJhd/94iR/k+SrhWCAZlAdHN80pyJs5/kYpwJnGsX8TfO4+aa8n9d2m3jOzdfn8MF9nQjByZjbp5VSdiV5eZIXJ/m7JO9O8sQkby2lPLvW+p7JNxGArVId3J5pLQba7vMxTgWuywuZ2mJaJ7Z5P0/fONunPTrJXUluS/KeJF9Va31Skh9Ncn2Su0spZSqtBGBLVAe3Z1qLgXbyfGy1AtflhUxtMa05tN7P0zfO1Ih3ZVj9/cUkh2utx5Ok1npHkicn+UiSV5RSXj/pRjZBU4YmADYzzSHaLptWkJnF89HlhUxtsVbBv+1bb8xzn3wwt33rjROp2no/T984UyN2JXlmrfVhQbfW+o5Sypcl+dUkT59U45rC0ATQFtMaou26aS0GmsXz0eWFTG0yjVNBvZ+nb5wgfGOt9YFLfbPWejrJt5RSXrDjVjVM11eDAt2xVh1c33GfdnVwZXWQYydO5b4Hz+S6/e0MYtMIMrN6PhzN3k3zej/3ycJgMPsh/lLKIElqrTN/7O04cte9+bW3Hc/of6mFJM998sE85+br59UsWq4LwYFmWnttzao6eLlRs76/1mf9fHRZH19LXj8Ts+F/tLF2jegrQxNMmuk2TNOsq4ObjZodOrC396911drJ6OvnptfPdI27fdpCkmcm+YYk1yTZvcGPDWqtXzuBtk3cdnuShiaYNNNt2ImmVcUut6DHa51J8LnJNGw5CJdSdid5c4Y7RCwkGeTiMvNg5PbG2UlP0kIEJs2+nxtrWsBroiZWxTYbNfNaZ1K8lpqtrZ/f41SEX5zkKRkeqPGfM9wu7ceTvDLDcPzvk7wjyXdNtIUTstOepKEJJsl0m4drYsBroiZWxS43ataH13pbQ0Cb9Olzs22vpzZ/fo8ThL89yf+ttf5YkqydnVFrfTDJb5RS/k+Gew3/iyQ/M+F27pieJE1ius3DNTHgNVETP8s2GzXrw2u9zSGgTfrwWkra+Xpq8+f3OEH4uiS/PPL1IMnS2he11vtLKb+T5HvSwCDcp54kzWe6zcM1MeA1UVM/yy41ataH13qbQ0Cb9OG1lLTz9dTmz+9xgvD5JGdHvv7bJJ+77mc+kORpO23UNPSlJ0l7mG5zsaYGvKZp42dZ11/rbQ4BbdP111LSztdTmz+/xwnCf5XhThFrjif5x+t+5sYkH9tpo6ahLz1JaKs2Brx58FnWPG0OATRPG19Pbf78HicI/0mSrxv5+g1J/m0p5VeSvD7DBXNfl+TXJ9a6CetDTxLaSsDbOp9lzdLmEEDztPH11ObP73GC8K8neXQp5QsvHLX8c0menuR7M5wXvJDkRJKXTLiNQE8IeLTRtENA23YQYGfaGirb+vm9oyOWSylXZBiGDyR5IMmbaq2f3MLvteqIZQCYhzbuIAANNfkjlmutDyX5rZ3cB8B6KmAw1MYdBKBNdhSEASZNBQz+Xht3EGgznfD+EYSBRlEBg7/Xxh0E2konvJ92zbsBAKM2q4BNy8rqIEePL+fIXffm6PHlrKxuf+0ETNLaDgJ7lhazkGRPC3YQaKvRTvggF3fC6S4VYaBRZl0BUwWiydq6g0AbdWkaiikeWycIA40y6z00TcWg6dq6LVXbdGUais79eARhoFFmXQHrUhUI2L42HmSxEZ378Ww5CJdS9tZaTZQBpm6WFbCuVIGAnenKNBSd+/GMUxH+UCnlDUl+qdb6h9NqEMAsdaUK1DbmMNJEXZiGonM/nnGC8PEk357kmaWU+5L8UpJX11o/OpWWAZ3TxPDTlSpQm5jDCNOjcz+esY5YLqU8McnzMgzEn5HkXJLXZ1glvmuM+3HEMvSM8MOao8eXc8fr33lRxWrP0mJu+9YbW12Jg6ZYKzro3F9k50cs11rfkeQdpZQXJXluhqH4liTPLqUcT/Jfkrym1vrxHTYW6BgLOFhjDiNMVxemeMzKtnaNqLV+IskrkrziQpX4B5J8R5KfTXJHKeW/J/n5WuufTqylXNakh52bOIxNewk/Q95X5jACzTGJ7dM+muTjSc5mOF3iERlWi7+rlPKmJN9Xa/3YBB6HTUx62NkwNpMm/HhfrdnJHEYdCWCSthWESylLSb4tyfOTfHWG8y6OJ3l5klcn+fIk/yrJ05L8QobTJ5iiSQ87G8Zm0izg8L5as90FijoSwKSNFYRLKQcynBf8PUkelWQlyRuS1FrrH4z86FuTvLWU8ptJvnEiLWVTkx52NozNpNmdwftq1HbmMOpIAJM2zoEab0nylAyrvx/OsPr7ylrrhzf5tT9L8i07aiFbMulhZ8PYTEPfF3B4X+2MjgQwaeNUhJ+a5I+S1CRvqLWuXObnk+RNGYZmpmzSw86GsWHyvK92RkcCmLQt7yNcSnlcrfX9k3hQ+whPx6T3DbQPIUye99X2mSMM7MCGHxJjHagxKYJwP1jdDUyajgSwTTs/UAO2SuUGmIa+zzMHJmvXvBtAN42u7h7k4tXdMG0rq4McPb6cI3fdm6PHl7OyOvuRLwCaT0WYqbC6m3kxGgHAVqkIMxVrq7tHWd3NLBiNAGCrBGGmYm2bqD1Li1lIssc2UczIZqMRADDK1AimoiuniNn5on3sNQvAVgnCTE3bV3eba9pODq0AYKsEYbiE0bmmycVzTdsa7vugK6MRbWc0BWgDQZhOmsRF2M4X7dX20Yi2M5oCtIUgTOdM6iI8jbmmqmT0gdEUoC0EYTpnUhfhSc81VSXbOR2JdjCaArSFIEznTOoiPOm5pqpkO6Mj0R527mA9nViaShCmcyZ5EZ7kXFNVsp3RkWgPO3cwSieWJhOEaaztVhBmcRHeTttUyXZGR6I97NzBKJ1YmkwQppF2UkGY9kV4u21TJdsZHYl2sXMHa3RiaTJBuGfaMk9rpxWEaV6Et9s2VbKd0ZGAdtKJpckE4R5p0zytJlcQdtI2VbLt05F4uLZ0bOk3nViaTBDukTbN02pyBaHJbes6HYm/16aOLf2mE0uT7Zp3A5idzSqZTbNWQdiztJiFJHsaVEFoctvoj9GO7SAXd2yhadY6sc+5+focPrhPCKYxVIR7pE2VzCZXEJrcNvqjydOHANpCEO6Rts3TavIweJPbRj+0qWML0FSCcI90pZJpgRC0r2ML0ESCcM+0vZJpgRAMzaJjq9MJdJ0gTKu0aecLmLZpdmx1OmH6dDbnTxCmVSwQgtnQ6YTp0tlsBtun0Wgrq4McPb6cI3fdm6PHl/PYfZ+V3UuLF/1MmxcIrf/7VlYH824SJGnXdovQRrZAbAYVYRpro97y4z7/s/O4a67K+zuwQEg1gCazKwVMlxHOZhCEaayNhmbf/+FP5MXP+PLsummh1TtfJIaeaTa7UsB06Ww2gyDMhpowgf9SveUHPvK3nz6dqM1UA2iyrmy3CE01TmezCdfkrhKEeZimDNl3vbfc9b+P9mv7dovQZFvtbDblmtxVFsvxME2ZwL/WW96ztJiFJHs6NjTb9b8PgM2tdTbXRjk3CrZNuSZ3lYowD9OUIfuuD812/e8DYOeack3uKkGYh2nSkH3Xh2a7/vcBsDNNuiZ3kakRPIwhewBoBtfk6VIR5mEM2QNAM7gmT5cgzIYM2QNAM7gmT4+pEQAA9JIgDABALwnCAAD0kjnCACRxjCvQP4IwAI5xBXrJ1AiALVhZHeTo8eUcueveHD2+nJXVwbybNFGOcQX6SEUY4IJLTQ3oQ7XUMa5AHwnCANl8asBotTS5uFralZDoGFegj0yNAMjmUwM2q5Z2hWNcgT5SEWburFSnCTYLu32oljrGFegjQZi52mg4+nGf/9l5xk2PzV8u/+1YwVigZic2C7tr1dL10ya6Vi11jCvQN4Iwc7XR3Mt3f/Bjed/J0zn/0OqWFyX1YTET07VZ2FUtBegmQZi52mg4ejBIPvXQapKtL0rqw2ImputyYVe1FKB7LJZjrtaGozezlUVJfVjMxPSthd3n3Hx9Dh/cp+IL0HGCMHO1fqX60uKurM8eW1mUtFGg7tpiJgBgskyNYK7WD0d/4ed+Vt5w7IG8f8xFSX1ZzAQATI4gzNytn3t508F9Yy9KspgJABiXIEzjbHdRUhcXM9kSDgCmRxCGhrIlHABMl8Vy0FCbHfkLAOycIAwNZUs4AJguQRgaypZwADBdgjA01Po9lvfYEg4ua2V1kKPHl3Pkrntz9PhyVlYH824S0GAWy0FD2RIOxmOBKTAuQZi5sC3Y1jR1SzjPH000usA0uXiBadPeQ0AzCMLMnKpNu3n+aKrNFpgKwsBGzBFm5mwL1m6eP5rKAlNgXIIwM2dbsHbz/NFUFpgC4zI1gplbq9qcHQlTqjbt4fmjqSwwZVzWOyAIM3NrVZv1c0xVbdrB80eTNXWBKc1jvQOJIMwcqNq0m+cP6AK7jJAIwsyJqk27ef6AtrPLCIkgDMA65k3SB9Y7kAjCAIwwb5K+sN6BRBAGYIR5k/SF9Q4kgjAAI8ybpE+sd8CBGgB8mtPZgD4RhJm6ldVBjh5fzpG77s3R48tZWR3Mu0nAJTidDegTUyOYKgtvoF26MG/SrhfAVgnCTJWFN9A+l5o32YaAqfMNjEMQZqosvIFuaEvA1PkGxmGOMFPVpYU35jrTZ6MBc5CLA2aTbNb5BlhPRZip6sqG5W2phsG0tGV0x2lhwDgEYaaqCwtvEsOt0JaA2ZXONzAbgjBT14UNy9tSDYNpaUvA7ErnG5gNQRi2oC3VMJiWNgXMLnS+gdkQhGEL2lINg2kSMIGuEYRhC9pUDQMAtkYQhi1SDQOAbrGPMAAAvSQIAwDQS4IwAAC9ZI4wkGR4et6xE6dy34Nnct1+iwEB6D5BGHCENAC9ZGoEcNER0oNcfIQ0AHSVIAxseoQ0AHSVIAx8+gjpUY6QBqDrBGHg00dI71lazEKSPY6QBqAHLJajNexqMD2OkAagjwThFulzELSrwfQ5QhqAvhGEW6LvQXB0V4Pk4l0NBDf6oM8dYYBpEYRbYtZBsGkX3c12NRCE6bq+d4QBpkUQbolZBsEmXnTXdjU4O/LfoI27GjStg0E7GBEBmA5BuCVmGQSbeNFd29VgfThv064GTexg0A5GRACmQxBuiVkGwSZedLuwq0ETOxi0Q1dGRACaRhBuiVkGwaZedNu+q0ETOxi0QxdGRACaSBBukVkFQRfd6WhqB4Pm68KICEATCcI8jIvudOhgsBNtHxEBaCJBmA256E6eDgYANIsgDDOkgwEAzbFr3g0AAIB5EIQBAOglQRgAgF4ShAEA6CVBGACAXhKEAQDoJUEYAIBeEoQBAOglB2qss7I6yLETp3Lfg2dy3X4nfwEAdJUgPGJldZCXHrk795w8nXPnV7J7aTE3XHNVbr/1JmEYAKBjTI0YcezEqdxz8nTOnl/JIMnZ8yu55+TpHDtxat5NAwBgwgThEfc9eCbnzq9cdNu58yu5f/nMnFoEAMC0CMIjrtt/ZXYvLV502+6lxVy778o5tQgAgGkRhEccOrA3N1xzVfYsLWYhyZ4Lc4QPHdg776YBADBhFsuNWNy1kNtvvSnHTpzK/ctncu0+u0YAAHSVILzO4q6FHD64L4cP7pt3UwAAmCJTIwAA6CVBGACAXhKEAQDoJUEYAIBeEoQBAOglQRgAgF4ShAEA6CVBGACAXhKEAQDopbmeLFdKmefDAwDQD4Na68L6G1WEAQDopYXBYDDvNgAAwMypCAMA0EuCMAAAvTTXxXIAzE4p5dVJvjvJY2utD8y3NQDzpyIMAEAvCcIA/XFbkscnOTnvhgA0gV0jAADoJXOEAXaglPKGJE9P8sJa6yvWfe/lSf51kl+ptX7/Fu7rKUluSfKkJF+QZCnJfUn+R5KfrLWeHfnZxyZ5Z5LVJDfWWj8w8r3PTPKnSQ4meWqt9W0Xbn91NpgjXEp5WpIXJXlCkkcm+WiSe5O8rtZax/jPAdAqpkYA7Mz3Jflgkp8qpdy4dmMp5WuTvDTJe5O8cIv39eIk/zTJu5L8UpJXJflUkh9P8rullMW1H6y1/mWS709ydZI7SymjhY2a5IYkP7EWgi+llPK8JP8zwxD8piQ/k+TNST4jyfdusd0AraQiDLADtdaPlVJuSfK2JK8rpXxFkn+Q5LVJziX5jlrrJ7d4dyXJX9ZaL5qzNlJZfmaS14089m+WUn4xyQ8leXmS20opz03y3CRvvXDb5Tw/w7D9ZbXWU+se93O22G6AVlIRBtihWus7kvybJNdnWMl9bZL9GU6X+Isx7uf+9SH4gp+78O83bPC9H0ny50leXEp5QYbV4I8kubXWurrFh34oyfkN2vM3W/x9gFZSEQaYjJ9M8uQkz7nw9Z211leNcwcX5va+KMm3ZDi/97OSLIz8yDXrf6fWeraU8qwM5wS/IskgyTNrrR/e4sMeyXA6xF+UUl6XYWX7T2qtHxmn7QBtpCIMMAEXKrm/PXLTz13qZzdSSllK8odJ/l2SPRlOgbgjycsu/C9Jdl/i148nefeF///eJL+/1cettRZtpd4AAAJASURBVP5shgvoPpjhXObfTrJcSvmjUspXjfM3ALSNIAwwAaWU65P8dJKPZ7iTw6tKKXvGuIunJ/lHSf5brfVLaq3Pq7X+aK31xzOcbrGZlyR5YpK/SfJFGe4XvGW11tfUWg8neVSSf5bkV5J8dZLfK6XsHee+ANpEEAbYoVLK7gwruJ+Z5NkZVnK/JONVhQ9c+Pe3Nvje12zy2E9M8hNJ3p/kiy/8+7JSypPGeOwkSa31dK31zbXWH0jy6gy3Urt53PsBaAtBGGDnfjrJjUn+Q63195P8WJI/SfL8Usp3bPE+Hrjw75NHbyylXJvh/OOHKaVcneTOJCtJnl1rXU7yrAwXv91ZSnnU5R60lPKN67ZeW7NWCd7qjhcArWOxHMAOlFKekeQFSe7OcIuz1FpXLmyp9q4kv1xK+dNa6/2Xuas3JTmR5EdKKV+S4WEZj0nyTUl+58L/X++/Xrj9hbXWd1147D8vpfzLJD+f5FeTPO0yj/sbSc6WUt6eYRhfyLAKfCjJnyV5y2V+H6C1VIQBtqmU8pgMw+gnktxSa31o7Xu11g9leNjGlUl+o5TyiM3uq9b6/5I8NcmvZzjP94VJvjTDvYC/c4PH/udJnpHkjetPtKu1/kKGi96+uZTyw5f5M16S5H8n+YoM9zH+3gxPtHtxkqfUWh+2rRpAVywMBhttWQkAAN2mIgwAQC8JwgAA9JIgDABALwnCAAD0kiAMAEAvCcIAAPSSIAwAQC8JwgAA9JIgDABALwnCAAD00v8HWxo8UXDzZuoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "width = 12\n",
    "height = 6\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "fig.set_size_inches(width,height)\n",
    "ax.plot(X[:,1], Y, \".\")\n",
    "ax.tick_params(axis='both', which='both', bottom=False, left=False, labelbottom=False, labelleft=False)\n",
    "ax.set_xlabel('x axis')\n",
    "ax.set_ylabel('y axis')\n",
    "pwk.save_fig('01-set_of_points')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Step 3 - Direct calculation of the normal equation\n",
    "\n",
    "\n",
    "We'll try to find an optimal value of $\\theta$, minimizing a cost function.  \n",
    "The cost function, classically used in the case of linear regressions, is the **root mean square error** (racine carré de l'erreur quadratique moyenne):  \n",
    "\n",
    "$RMSE(X,h_\\theta)=\\sqrt{\\frac1n\\sum_{i=1}^n\\left[h_\\theta(X^{(i)})-Y^{(i)}\\right]^2}$  \n",
    "\n",
    "With the simplified variant : $MSE(X,h_\\theta)=\\frac1n\\sum_{i=1}^n\\left[h_\\theta(X^{(i)})-Y^{(i)}\\right]^2$\n",
    "\n",
    "The optimal value of regression is : $ \\hat{ \\theta } =( X^{-T} .X)^{-1}.X^{-T}.Y$\n",
    "\n",
    "Démontstration : https://eli.thegreenplace.net/2014/derivation-of-the-normal-equation-for-linear-regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Theta :\n",
      " [[4]\n",
      " [2]] \n",
      "\n",
      "theta hat :\n",
      " [[6.57350113]\n",
      " [1.53229674]]\n"
     ]
    }
   ],
   "source": [
    "theta_hat = np.linalg.inv(X.T @ X) @ X.T @ Y\n",
    "\n",
    "print(\"Theta :\\n\",theta,\"\\n\\ntheta hat :\\n\",theta_hat)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Show it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsMAAAFrCAYAAAA0K4RcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXRc6V3m8aeqVJK8ybZsS7LkRV7acQeSTCd47DTp0Ak5Q4YlgZCQpUlYDiTwnpDMMHMmdJg5EAIdGJYDE3gZQhgymTQhM5D0kAEGhiULNG0cSKdJuo1bdpfc2r3J8qat6s4ft8qq0lq36lbVfe/9fs7xkVySrl5bSz33fX+/9015nicAAAAgidKtHgAAAADQKoRhAAAAJBZhGAAAAIlFGAYAAEBiEYYBAACQWC0Jw8YYzxjDNhYAAABoqbYWf34CMQAAABottdYbKJMAAABAYhGGAQAAkFiEYQAAACQWYRgAAACJRRgGAABAYhGGAQAAkFiEYQAAACQWYRgAAACJRRgGAABAYhGGAQAAkFitPo4ZAAAANcgXPJ0ZmtL5iRkd6evSiaM9yqTXPHUYayAMAwAAOCZf8PT+R0/r7Oi05hby6shmdHxghx556CSBOCDKJAAAABxzZmhKZ0enNbuQlydpdiGvs6PTOjM01eqhOYcwDAAA4JjzEzOaW8hXPDa3kNeFyZkWjchdhGEAAADHHOnrUkc2U/FYRzajw71dLRqRuwjDAAAAjjlxtEfHB3aoM5tRSlJnsWb4xNGeVg/NOTTQAQAAOCaTTumRh07qzNCULkzO6HAvu0nUijAMAADgoEw6pVPHenXqWG+rh+I0yiQAAACQWIRhAAAAJBZhGAAAAIlFGAYAAEBiEYYBAACQWIRhAAAAJBZhGAAAAIlFGAYAAEBiEYYBAACQWIRhAAAAJBZhGAAAAInV1uoBAAAAJEW+4OnM0JTOT8zoSF+XThztUSadavWwEo0wDAAA0AT5gqf3P3paZ0enNbeQV0c2o+MDO/TIQycJxC1EmQQAAEATnBma0tnRac0u5OVJml3I6+zotM4MTbV6aIlGGAYAAGiC8xMzmlvIVzw2t5DXhcmZFo0IEmEYAACgKY70dakjm6l4rCOb0eHerhaNCBJhGAAAoClOHO3R8YEd6sxmlJLUWawZPnG0p9VDSzQa6AAAAJogk07pkYdO6szQlC5MzuhwL7tJRAFhGAAAoEky6ZROHevVqWO9rR4KiiiTAAAAQGIRhgEAAJBYhGEAAAAkFmEYAAAAiUUYBgAAQGIRhgEAAJBYhGEAAAAkFmEYAAAAiUUYBgAAQGIRhgEAAJBYhGEAAAAkFmEYAAAAiUUYBgAAQGIRhgEAAJBYhGEAAAAkFmEYAAAAiUUYBgAAQGIRhgEAAJBYhGEAAAAkFmEYAAAAiUUYBgAAQGIRhgEAAJBYhGEAAAAkFmEYAAAAiUUYBgAAQGIRhgEAAJBYhGEAAAAkFmEYAAAAiUUYBgAAQGIRhgEAAJBYhGEAAAAkFmEYAAAAiUUYBgAAQGIRhgEAAJBYhGEAAAAkFmEYAAAAiUUYBgAAQGK1tXoAAAC4IF/wdGZoSucnZnSkr0snjvYok061elgA6kQYBgBgA/mCp/c/elpnR6c1t5BXRzaj4wM79MhDJwnEgOMokwAAYANnhqZ0dnRaswt5eZJmF/I6OzqtM0NTrR4agDoRhgEA2MD5iRnNLeQrHptbyOvC5EyLRgQgLIRhAAA2cKSvSx3ZTMVjHdmMDvd2tWhEAMJCGAYAYAMnjvbo+MAOdWYzSknqLNYMnzja0+qhAagTDXQAAGwgk07pkYdO6szQlC5MzuhwL7tJAHFBGAYAoAqZdEqnjvXq1LHeVg8FQIgokwAAAEBiEYYBAACQWJRJAABih9PiAFSLMAwAiBVOiwMQBGUSAIBY4bQ4AEEQhgEAscJpcQCCIAwDAGKF0+IABEEYBgDECqfFAQiCBjoAQKxwWhyAIAjDAIDY4bQ4ANWiTAIAAACJxcwwAAAO44ARoD6EYQAAHMUBI0D9KJMAAMBRHDAC1I8wDACAozhgBKgfYRgAAEdxwAhQP8IwAACO4oARoH400AEA4CgOGAHqRxgGAMBhHDAC1IcyCQAAACQWYRgAAACJRRgGAABAYhGGAQAAkFiEYQAAACQWYRgAAACJRRgGAABAYrHPMAAgUfIFT2eGpnR+YkZH+jikAqhWXH92CMMAgEhp5BNuvuDp/Y+e1tnRac0t5NVRPL74kYdOxuJJHWiUOP/sEIYBAJHR6CfcM0NTOjs6rdmFvCRpdiGvs6PTOjM0xQluwDpq/dlxYTaZMAwAiIxGh9XzEzOaK167ZG4hrwuTM4RhYB21/Oy4MptMAx0AoCHyBU9PnJvUo194Vk+cm1S+4G34Mes94YbhSF+XOrKZisc6shkd7u0K5fpAXNXys1N+c+up8uY2SpgZBgCErtYZodIT7mxZIA4zrJ442qPjAztWjOvE0Z5Qrg/EVS0/O66sxBCGAQChq7XcodFhNZNO6ZGHTurM0JQuTM7ocG80axgRPhdqV6Oslp+dRt/choUwDAAIXa0zQkGecGsNN5l0SqeO9bZ0Zopg1lyu1K6WRPX7I+jPjisrMYRhAEDo6pkRquYJ17VwU87lsbvKpV1E4vT94cpKDA10AIDQlWaEOrMZpSR11jEjtFojniuNOatxeeyuanRjZpji9v1Rurl92wP36NSx3sgFYYmZYQBAA4Q1I7TWLNnXH+h2ojFnNa1qKorq0nszuFK7KrnTdBYnhGEAQEOEUZu71vL2C/fvdCbcLNeKYBZ06T1uwdmV2lXJreAeF4RhAEBkrTVLlkmnnAk3y7UimAWpmY1TzWqJK7WrklvBPS4IwwCAyFprluxo33a99RX3OBFulmtFMAuy9O5Ss1kQUdhFpBouBfe4IAwDACJrvVkyV8LNapo99iBL79Sstp7L39suIgwDACKLWbJwBFl6p2YVSUMYdljcGhwAYDXMktUvyE0FNatIGsKwo+LY4AAAaJxqbyqYjUfSEIYdFdcGBwBA6zEb77ZaVo7zBU+nz03qi8+My5P0yhfu1cl7onlIRtgIw46iwQEAACxXy8pxvuDp4U88oacuXpXn+Y99/mtjetHBXfpQAlacOY7ZUaUGh3I0OAAAkGy1HOd8ZmhKT49M3w3CklTwpGdGrjl7DHQQhGFHlRocOrMZpSR10uAAAEDirbdyvN7HLOQLKx6fXyys+3FxQZmEo2hwwGrYYQQAkq2WrfGO9HUpm0mvCMTtbelErDgThh1GgwPK5QueHn70tJ4Zuab5xYLa29K6d9/ORNR7AQB8tWyNd+Joj164b0dFzXA6Jd27b2ciVpwJw0BMnH52Uv80fEWF4i+y+cWC/mn4ik4/O6n7X9DX2sEBAJqilpXjTDqlD33vqbu7SUjSA2HsJnHnljQ2LI3kpNGc/3rvPuntP1b7NRuAMAzExBeeHr8bhEsKnvTFp8cJwwCQILWsHGfSKd1/vE/3H6/h+WJ+Tpp43g+8o8NLL6+u0nw3cy349Rus7jBsjNkt6QFJtyX9hbU2v8GHAGgACiEAAA2Vz0tTY8WwW/ZnalzyVjbgrWpyVFqYl7LtjRtnQFWHYWPMj0r6fkn/2lp7tfjYyyT9X0ndxXf7kjHm1dbaW2EPFMD6Hrh3r/76a2MVW+OkUv7jAABUrVDwZ3XvzvLm/BKH8eelxYXqr5Npk/r2SQODxT8H/Zdt2YYMu1ZBZobfLMkrBeGiX5S0U9LvSuqV9G2SfkTSL4c2QgBVOXmsVy8+0K2nR6a1kC8om0nrhft26CQNlgAQWCJ25/E8aWZaGsv5wXfkOT/0jg5Lc3eqv04qJe3ZWxZ4D/kvewaktuhX5AYZ4T2S/rj0l2J5xDdJ+qi19l3Fx05LepsIw0DTlRog2G4PAOpTyylukXf7ph9yS8G3VNd783qw6+zcvRR6+wf91/fulzo6Qx9yswQJw7sklVdCf2Px5WfKHvui/FIKAC3AdnsAUL/yU9ykylPcIv/7dX5OGr9YWeIwmpOuXQ52nS3bysobBpcC8OatIQ+49YKE4auSdpf9/ZskFSQ9XvaYJ8ndWwPclYjlIQAAVrHeKW6RCcOLi9LU6Mq63qkxVTSPbKSj0w+6/Qcr63q7dvrlDwkQJAw/I+k7jDE/KSkvv4b4jLW2/Jy+QUkT4Q0PrRDL5SEAAKpUyyluDVMoSFemimE3t1TbOzEi5Rerv06mzS9nKC9x2DcodfdI6XRDhu6KIGH41yQ9JmlE0qKkzZLeV3qjMSYj6RWqnCmGg5xeHkJksdoAwBW1nOJWN8+Trl9dmuktBd+xYWlutvrrpNJSz97K0ob+Qamn34lmtlao+n/FWvtHxpgfkfTO4kOPWms/UfYur5FfIvFnIY4PLeDE8hDuciFkstoAwCW1nOIWyK0bS7s2lJc43JzZ6CMrde8pNrGVlTjsPSC1d6z67i48X7RCoFsEa+1HJH1kjbf9mfxt1uC4SC0PYV2uhExWGwC4JpSG5LnZYjNbrmzGdzh4M9vW7X5Jw9263uLrm7dUfQlXni9agflyrNCS5SHUxJWQyWoDgFhbXJQmRyq3Lht5Tro8EbCZbVPZLO/gUolD1466m9lceb5ohTXDsDHmQPHVUWttvuzvG7LWXqx7ZGiZhi8PITSuhExWGwDEQqEgXZmURnKVdb1Bm9nasn45w8CyHRy6exq2g4MrzxetsN7McE7+Vmn3SjpX9veNeBtcFw5gv1o3uBIyWW0A4JS7zWy5lSUO83PVXyeVlnr7l21dNug3s2UyjRj5mlx5vmiF9ULrx+UH2+vL/g4gIlwJmaw2AIismzeKM7y5yoa22zeDXae7p1jXO7g007t3v5RtD3nAtXHl+aIVUl6QWpaQGGM8SbLWNv1zA3FT6g4mZALAOuZmK3dwKJU4TF8Jdp1tOypLGwYO+rO+m6pvZmuVWp8vYrILxZoDppwBcBwlLQBQZnFBmhz1G9jKG9qCNrN1bl7WzDboh96uHY0ZdxPU8nyRhF0oqg7DxpgHrLVfrOL93mut/bX6hgUAAMrFZHYuPIW8dGlyZYnD5IiUz2/wwWXaslL/gWLYHSxrZtuTmOOI15OEXSiCzAz/lTHmA9ban13tjcaYnZI+Junb5Z9WBwAAQpCE2bk1eZ5fynC3mS3nB9/xi8Ga2dJpqWdg5Wzvnr1Nb2ZzSRJ2oQgShockfcAY86Ck77XWTpTeYIy5X9InJe2X9JlQRwgAQMIlYXZOkn8CW3ngLZU4BG1m29277GS2QalvX8Ob2eI4e5+EXSiChOGXSfpNSW+X9GVjzDustf/PGPOwpA9Iykt6t6UrDgAQE1EJN0Fn56Iy7jXN3pHGLvpht3zP3uvXgl2na+cqdb0H/HrfJovr7H0SdqGoOgxba29L+j5jzF9K+g1Jf2qMeVrS18nfh/jN1tqnGjNMAACaK0rhJsjsXJTGrYV5/0CKseHKGd/LExt9ZKVNWypDb/9B/+/botPMFtfZ+yRsjRl4Nwlr7ceNMdskfVjS10u6JOmV1tpLYQ8OABC+yM8aRkQjw03Qr0GQ2bmWhLJCXro0sbKud3LEP7WtWtn24slsg5Xhd+fuyDezNaq2Ngo/r3HftShQGDbGpCV9UNL7JN2U9JSk+yV9zhjzFmvtP4U/RABAWCI1axhxjQw3Qb8GQWbnGtrw5HnStcsrT2Ybv+jPAlcrnfZreJfX9e7pk9JuNrM1oraWn9fmCLK12n75TXL3S/qK/LKIc2U1w6eNMf+emmEAiK64LuU2QqMah2r9GlQ7OxfauG9cXzqCuDz83rkV7Dq7+1aWODShma3ZGlFby89rcwSZGX5S0k75TXQ/bq2dkyRr7YeMMZ+XH5Q/bIx5jbX2DeEP1W1RWOYAgCRskxSWRjUONfprEHjcs7fLTmUbXgq9MwGb2bZ3V5Y39A8Wm9k21fXvcUUjamv5eW2OIGE4LemN1tpPL3+DtfZxY8xLJP2upNeHNbi4YJkDQFQkYZuksDSqcajRX4M1x51fkEZHVm5ddnky2CfYvLV4BPFgWfA9KG3bHsr4XRZ2bS0/r80RJAzfZ63NrfVGa+20pO8yxry77lHFDMscAKKiFdskubwy1ojGoYZ/DfJ5ZS6N6dSNYZ26lpO+mpM+PixNjQZrZmvv8Gd2+wcrZ3x37Ip8M1tcJGFbsyhIeUHO6Q6JMcaTpKSUFz/6hWf1Pz5/TuX/0ylJ73jwmN72wD2tGhYc43KgQLSUvpeasU3SRitjSf2+DuVr4HnS1UvLDqjI+fv3Li5Uf51MRurdt7LEYU9vZJvZkvR908yf15hb8z8t8NZqCI5lDtSLUhuEqZnbJK23MnbiaE9iv68Dfw1mppfKG8rremdvB/vEe/au3Lasd0Bqywa7Tgsl7fdh3Lc1i4KgW6ulJL1R0rdIGpDUscq7edbabw5hbJFT650oyxyoF6U2CCJKs2brNQBJ4vt6uTu3imG3LPCO5qQb08Gus2NXWV3vYPH1g1JHZ/hjbjJ+HyJsQbZW65D0J5IelD/V7Klyytkrezx26rkTTcLpLWgsOoorRSnsRU3UZs3WWxlL9Pf1wrw0/rw0+lxl8L06Few6m7f6YXff4FLw7T8obd0W+pCjItHfNxEUh9/HQWaG3yfpVfIP3fgv8k+e+2lJH5EfkH9e0uOS3h7qCCOi3jtRljlQD0ptlkQt7EVN1GbNNloZi/P3db7g6cy5cV169rxeoOs6mr+m9Hgx+E6OSV7QZraDK0sctncnrpktCb8PXQmYcfl9HCQMv0nSP1prf0qSjDGSJGvthKTfN8b8vfy9iP+NpF8OeZwtx50oWolSmyVRC3tRE7XfVeutjMXq+9rz/Fnd0Zw0klNhNKfJrz6tl926rKzyG374XZmM1Le/GHgPLQXfXb3+qW2I1/fNKlwKmHH5fRwkDB+R9Ntlf/ck3a24t9ZeMMb8saTvVwzDcBLuRBFdlNosiVrYi5oo/q5aa2XMye9rz/Ob2cZyZeUNOX8Hh7JmtrSk/vWuk0pVNrOV9ux1rJmtFZz8vgnApYAZl9/HQcLwgqTZsr/fkLRn2fsMS3pdvYOKorjfiSL6KLXxRTHsRYlrv6si/X19+9ayo4hzfgC+eT3QZS5ltmg4u0u59l0aeNHX6eWvOint3R+LZrZWifT3TZ1cCphx+X0cJAyPyN9BouScpJcve5/7JF2td1BRFPc7UcAVroW9ZuN3VQ3m56Txi5WNbGM5fw/fILZsu1vLe6GtWx95+raeTe3QzYwfejuzGT38jfdJg9EKNIgWlwJmXH4fBwnDfyvpNWV/f0zSzxpjfkfSp+U30b1G0u+FNrqIifOdKOAKwt7G+F21hsVFaWpsqcRh5Dl/5ndqPFgzW0dnsZmtWN6wb9APwV077zazHSx48h49rcXRaaUcDgloPpcCZlx+HwcJw78nab8xZrB4LPOvSnq9pB+QXyeckjQk6SdCHiMAVCDsYV2FgnSl2MxWXts7MRLwZLY2v5yhoq53UNrVs2EzWyNDgis7DaA2rgXMOPw+rus4ZmNMm/xAfFRSTtJnrbUbHoeTtOOYAQAN4HnSzLXKwylKJ7TNzW7wwWVSKamnv2zrsuKfnn6pLVoHtbq00wAQMY05jtlauyjpD+u5BgDUihmyBLl1Y9nJbDn/7zdngl2ne8/Szg2lGd8+d5rZXNppAHBFtG55AaBKzJDF1Nxs8WS2nF/iMJLzQ++1y8Gus7WrbJa3bOuyzVvDHnFTubTTgKu4yU4ewjAAJzFD5rjFRWlyZGnnhtKM76Vxv/yhWh2bKk9kKwXfrh2xPJnNpZ0GXMRNdjIRhgE4qdkzZMwW1ahQkK5MrqzrnRiR8ovVX6ct6zez9S8Lvt0bN7PFiUs7DbiIm+xkIgwDcFIzZ8iYLaqC50nXr1YeTlGq652fq/46qbTU2++H3fLg29PvH1WccK7tNOCauJShcPMeDGEYgJOaOUPGbNEyt26Uhd3c0uu3bgS7TndPZSPbwKDfzNbeEfqQ4yQOW1lFVRzKULh5D44wDMBJzZwhi8tsUWBzs5U7OJSC7/SVYNfZtl0aOFQZfPcelDZvacCggdrFoQyFm/fgqg7Dxpgea+1UIwcDAEE0a4YsDrNF61pckCZHV5Y4XJ4I1szWuXkp8JaXOHTtaMSogdDFoQwlsTfvdQgyM/y8MeYxSb9lrf2rRg0IAKImDrNFkvxmtssTxbD73FLonRyR8vkNP/yutqy090Ax+JbN+HbvqXsHB2od0Wqul6HE/ua9AYKE4XOS3iTpjcaY85J+S9LHrLUB18sAYH1RC0TOzRZ5nl/KUNHIlpPGLtbQzDawcuuyPY1pZqPWEahfbG7emyjQcczGmPslvVN+KN4kaU7Sp+XPFn8hwHU4jhnAqghEAd2cKdb15ooHVOT8AHz7ZrDr7OqpPIp4YFDq2ydl20Me8NqeODepD336yxUzWp3ZjB5+w33OztIBrVCaUHDi5r15wjmO2Vr7uKTHjTHvlfQO+cH4rZLeYow5J+m/Svq4tfZaHYMFkGA0f6xh9o4/szuWq5zxvX412HW6dq6s6+0/IG1qfTMbtY5AOFwv9Wi2mnaTsNZel/RhSR8uzhb/sKTvkfQrkj5kjPmfkn7dWvul0EaKhgp7WTpqy9xwR9IDUX5+Xl8985RuDD2rQwtX1H/7klJjw36tbxCbNi8dQVxe4rAtus1s1DoCaIUwtla7IumapFn5pRPt8meN326M+aykH7TWBpy6QDOFvSzNMjfqkZhAVMhLlyYqdnDwiiezvcQrVH+dbHuxmW2wMvju3O3cccS11jpy8w2gHjWFYWNMVtJ3S3qXpFfKr8M4J+mDkj4m6V9I+g+SXifpN+SXUiCiwl6WZpkb9Yhd84fnSdcuL9X1lmp7xy9KC/MV75qStGZbWjot9e6rLHHYd0ja0yel43EyWy2Nitx8A6hXoDBsjDkqv074+yXtkpSX9Jgka639y7J3/Zykzxlj/kDSa0MZKRom7GXppC9zoz7O7dxQ7sb1pSOIy/fsvXMr0GXG27qUy+7ScLZbufZdevHLX6pv/db7m9rM1ipBax25+QZQryCHbvyFpFfJn7wYkz8L/BFr7dg6H/YPkr6rrhGi4cJelk7MMjcaJvLNH7O3/Wa28ka20Zw0E7B3ePvOYl3voDRwUF/Nd+mDj1/WdH5pprczm9GD996biCBcC26+AdQryMzwqyX9tSQr6TFrbTU7tH9WfnBGhIW9LB27ZW4k18K8NDFSOcs7lpMuTwa7zqYt0r7Byoa2/oP+McVl7i14Gnx+5ZI/Pztr4+YbQL2q3mfYGPMCa+0/h/FJ2Wc4esLek5A9DuGUQl6aGq8scRjJSVOj/qlt1WrvKDuZbXDpz45dVTez8bMTDDXDAKq05i+EQIduhIUwjPXQGY6G8Tzp6qXKWd7RnDT+/IpmtnVlMsVmtsGymd5BaU9vbJrZXMINBIAqhHPoBtBozPIgNDemK+t5S+H3zu1g19mz1w+85SUOvQPU8EZI5GvMAUQaYRiRQmc4Artzq1jaMFzZ0HZjOtBlvO3dSt2t6x30g+/eA1LnpvDHDACIDMIwIoXOcKxpYd4vZ1he4nBlKth1Nm+VNzCoJ2516isL2zSU3qmJzXs0cGAvKxAAkECEYUQKneFQPi9dGvMb2Mob2ibHpCAns7V3SP0HpIFDlQ1t27t1+tkp/fynv7z0fVaQbrACAQCJRBhGpLAtW4J4nnR1atlevcP+yWyLC9VfJ5OR+vaX1fUO+q/v7vNPbVsFKxAAgBLCMCLF6dPHitgNYxUz02WNbMU/Yxf9wyuqlUr5AffuDg7FGd/eAaktG2g4rEAAAEoIw4gclzvDE78bxu1blUcRl16/cT3YdXbu9g+lKN+6bO8BqaMzlGGyAgEAKCEMAyFKzG4Y83PSxPNLh1OUGtquXgp2nc1bpX2HKmd6+w9KW7Y1YNBL4rAC4SpWTgBEDWEYKArjSTp2taj5vDQ5Wty5oWzGd2o8WDNbR2dxpndZXe/27qpPZgubyysQrkr8ygmASCIMAwrvSTrsWtSmzaIVCsVmtuHKut6JkYDNbG1S377Ko4gHBqVdPWs2syE5ErNyAsAphGFA4T1Jh1mL2pBZNM+TZq5V7uBQOrBi7k7110mlpJ7+srreQX+mt2dAamvtrxWW4aMrdisnAGKBMAwovCfpMGtR6w7ot28u1fKW79l7cybYQHbuXgq7/YPSvkF/K7OQmtnCxDJ8tLGLByRuWBE9hGFA4T5Jh1WLWnVAn5tdOpmtvLb32uVgn3BrV+Usb3/x5eatdf07moll+GhjFw9ww4ooIgwjdmqZdWjGk3TQcS0P6Bkvr8O6oW+49oz02N8tzfROjfnlD9Xq6PQDb8XWZYekrh0ta2YLC8vw0cYuHuCGFVFEGEas1Drr0Ogn6UDjKhSkK5M6cTOnH118SpuujOrA3GXtW7imrArSxSo/aVtW2rt/ZV1vd3yb2ViGjz528Ug2blgRRYRhVM2FOq96Zh0a+SS96rhGrunJJ8/pZR23KkscxoaluVllJL22moun0lLP3pUlDr0D/lHFCcIyPBBt3LAiigjDqIordV5RnXV4fnhCR29c1ODCVR2cv6LBhSsaXLiqLjsb7ELdPUsnspVe9u2X2jsaMWznsAxfyYUbWCQLN6yIIsIwquJKnVfLZx3mZqXxiyu2LnvTtct6U5DrbN3u79pQXuLQf1DavKUBg44XluF9rtzAIlm4YUUUEYZRlajOuC7XtFmHxUVpcqTsgIpi8L08EaiZzevYpNTAQf9I4vLg27Uj3PEicVy5gUXycMOKqCEMoyotn3GtUuizDoWCH3DvzvLm/NcnRqT8YvXXacvK27tfl7f1aaRztzYdOqJ7vuElyuzudX4HB0STKzewANBqhGFUxaU6r5pmHTxPun7VD7x3D6jISWMXpfm56q+TSku9/cuOIz4o7elXKpPRHkl7qr8aUDNXbmABoNUIw6iK63Ve5Y1Ex7an9dKOW8qMDy/N+I7m/BPbgtjVU6zlHVxqZtu7X8q2hz5+ICiXbmABoJUIw6iac3VecyTSjoQAAA9nSURBVLPS2LAKzz+nx//yCW29OqbXzl/RrvytYNfZtsMPuuUNbf0HpE00syG6mrF3NjtVAIgDwjDct7jg1/CWGtlKdb2XxiVJaUkPVHOdTZsrZ3lLOzjQzAZHNeoGlp0qgHBwUxkNhGG4o5CXLk0Ww27Or+0dy0mTo1I+v/7HlplXRhfbu5XZN6hD973Yn/EdGJR27qaZDagCO1UA9eOmMjoIw4gez5OuXfZPYivfumw8WDObl0prpqtH17b36fGbnRrK7FQuu0vjbdvV3p7Vw2+4T4cce+JmFgFRwE4VQP24qYwOwjBa6+bMsr16n/Nf3glY17u7926JQ6H/oH7tyRv6m+k23VpMqb0treyetBbzntONRMwiICrYqQKoHzeV0UEYRnPM3va3Kas4mS0nXb8W7DrbdxZD7+BSbW//Aalz8913+ftzk/rc9S9rdtH/JTO3WJAkfffLDyubSTu3E0YJswiICnaqAOrHTWV0EIYRroV55cdHdOEfn9Jc7rwG7lzSjulxpa5MBrvOpi2VjWylZrZt2zf80NXutucXC8pm0nrbA/cEG0eEMIuAqHB9q0UgCoLcVFIi11iEYdSmkJemxoszvMN3Z3y9yRFlCgVVHTmz7cXtypbt4FBHM1tc77bj+u+Cm5zbahGImGpvKimRazzCMNZXamZbXtc7/ry0ML/i3df6sfRSaaX27lu5ddmePimdCXXIcV3Cjeu/CwCSqpqbSkrkGo8wjCU3pitPZCvt2XvndqDLjLV1aTi7S7nsLuXad2k4260Hv/mE3vLgCxsx6hXiuoQb138XAGBtlMg1HmE4iWZvlzWxLZU4aCZoM1v3ska2Qf397U793P95pmIpvzOb0WD/rhD/ARuL6xJuXP9dAIDVUSLXeIThOFuY98sZKk5my0lXpoJdZ/PWska2g0t1vVtX/iC+rODp+JfHWcoHACAElMg1HmE4DvJ56dLYyhKHqVGpUKj+Ou0d/jZly7cu27Gr6mY2lvIBAAgPz6uNRxh2iedJVy8tHUxRfjLb4kL118lkpN59lduWDRz0D64IoZmNpXwAAMLD82pjEYajamZ6aZa3vK53NkAzWyol7e6rnOUdGJR6B6S2bCNGDQAA4BTCcKvdueWH3ZFcZfC9cT3YdXbsKgbeQ0vBd+8BqaMz/DEDAADEBGG4WebnpImyZrbSy6s1NLPtKwbeUm1v/0Fp67bwxwwAABBzhOGw5fPS1FhZI1vxz9S45AVtZju4sq53e3fNJ7MBQK04DhZAXBGGa1Uo+LO65Ts4jA37W5kFamZrk/r2rdy6bFevlE43ZOgAEATHwQKIM8LwRjzPb2Yby/nBd+S5Yl3vsDR3p/rrpFLSnr1lgbdY6tAzILXxZQDiIK6zpxwHCyDOSGHlbt8sO5yirK73ZsBmtp27l0Jvqa53736a2YAYWCvwxnn2lONgAcRZMsPw/Jy/N2/FIRU56drlYNfZsm1lTe/AoN/kBiB21gu8cZ495ThYAHGWnDC8MC995Of9EoepMb/8oVodnUu7NpSH3q6dNLMBCbJe4I3z7CnHwQKIs+SE4Wy7NPS19ffvzbT55QzlJQ77BqXuHprZIiiu9ZmIrvUCb5xnTzkOFkCcJScMS37IPfsVKZWWevZWljf0D0o9/TSzOWL5cnV7W1r93Vv0jcf7dM/e7VU/UROoEcR6gTfus6ccBwsgrpKV/N7wg1Im7Z/M1t7R6tGgDsuXq+cWC3pu6oaem7qhziobl+Lc8ITGWC/wMnsKAG5KVhg+/IJWjwAhWW25uqTaxqU4NzyhMTYKvMyeAoB7KISFk0rL1Wsp1XGuZ736T2AtpcD7tgfu0aljvcz8AoDjCMNwUmm5unONQFxN49JqgTouDU8AAKA6ySqTQGyUL1cPTVzX3zwzofFrtwM1LsW94QkAAGws5QXZbzckxhhPkqy1Tf/ciKfSrhBBG5dq/TgAAOCUNZ/cmRlGLNTauBSnhie2iQMAIDjCMBADbBMHAEBtaKADYqB8mzhPldvEAQCAtRGGgRhgmzgAAGpDGAZigG3iAACoDWEYiIHyfZdT0t0jqdkmDvBr6p84N6lHv/Csnjg3qXyh+bsoAYguGuiAGNjomGAgqWguBbARwjCcwdZh64viNnF8zdBq5c2lUmVzaZR+VgC0DmEYTmB2xz18zRAF6zWXEoYBSNQMwxFsHeYevmaIAppLAWyEMAwnsHWYe/iaIQpoLgWwEcok4ITS7M5sWbhidifa+JohCmguRTXob0g2wjCcUJrdWV5/yuxOdPE1Q1REsbkU0UF/AwjDcAKzO+7hawbABew4AsIwnMHsjnv4mgGIOnYcAWEYAFA1aisRN/Q3gDAMAKgKtZWII/obQBgGAFSF2krEEf0NIAwDAKpCbSXiiv6GZOPQDQBAVTjNDUAcEYYRWfmCpyfOTerRLzyrJ85NKl/wWj0kINE4zQ1AHFEmgUiiUQeIHtdrK9kJA8BqCMOIJBp1gGhaq7Yy6kGTG2wAayEMI5Jo1AHc4ULQ5AYbwFqoGUYkxaVRh7pnJEF50PRUGTSjYr0bbADJxswwIikOm6C7MFsGhMGFlRxOGQOwFsIwIsn1Rh2JZVkkhwtBMw432AAagzCMyHJ9E3QXZsuAMLgQNONwgw2gMQjDQIO4MFsGhMGVoOn6DTaAxiAMAw3iwmwZEBaCJgBXEYaBBnFltgwAgCQjDAMNxGwZAADRxj7DAAAASCzCMAAAABKLMAwAAIDEomYYQGD5gqczQ1M6PzGjI300BgIA3EUYBhAIx0wDAOKEMgkAgZQfM+2p8phpAABcQxgGEMh6x0wDAOAawjCAQErHTJfjmGkAgKsIwwACKR0z3ZnNKCWpk2OmAQAOo4EOscauB+HjmGkAQJwQhhMiiaGQXQ8ah2OmAQBxQRhOgKSGwvJdD6TKXQ8IcYiTJN7sAkBYCMMJ0OxQGJUn5vV2PSAMIy6SerMLAGEhDCdAM0NhlJ6YS7sezJb9213b9SAqNxaILlZAAKA+hOEEaGYojNITc2nXg+XB3JVdD6J0Y4HoYgUEAOpDGE6AZobCKD0xu77rQZRuLBBdcVgBAYBWIgwnQDNDYdSemF3e9SBKNxaILtdXQACg1QjDCdGsUMgTc3iidmOBaHJ9BQQAWo0wjFDxxBwebixQLZdXQACg1QjDCB1PzOHgxgIAgMYjDAMRxo0FAACNlW71AAAAAIBWIQwDAAAgsQjDAAAASCzCMAAAABKLMAwAAIDEIgwDAAAgsQjDAAAASCzCMAAAABKLQzdCli94OjM0pfMTMzrSx4lhAAAAUUYYDlG+4On9j57W2dFpzS3k1ZHN6PjADj3y0EkCMQAAQARRJhGiM0NTOjs6rdmFvDxJswt5nR2d1pmhqVYPDQAAAKsgDIfo/MSM5hbyFY/NLeR1YXKmRSMCAADAegjDITrS16WObKbisY5sRod7u1o0IgAAAKyHMByiE0d7dHxghzqzGaUkdRZrhk8c7Wn10AAAALAKGuhClEmn9MhDJ3VmaEoXJmd0uJfdJAAAAKKMMByyTDqlU8d6depYb6uHAgAAgA1QJgEAAIDEIgwDAAAgsQjDAAAASCzCMAAAABKLMAwAAIDEIgwDAAAgsQjDAAAASCzCMAAAABKLMAwAAIDEaukJdMaYVn56AAAAJINnrU2t9gZmhgEAAJBYKc/zWj0GAAAAoCWYGQYAAEBiEYYBAACQWC1toAMANI8x5mOSvk/SIWttrrWjAYBoYGYYAAAAiUUYBoDkeFjSvZJGWz0QAIgKdpMAAABAYlEzDAB1MMY8Jun1kt5jrf3wsrd9UNJ/lPQ71tofquJar5L0VkmvkLRPUlbSeUn/S9IvWGtny973kKQvSypIus9aO1z2ti2SviTpmKRXW2s/X3z8Y1qlZtgY8zpJ75X0Qkndkq5IelbSp6y1NsB/BwA4hzIJAKjPD0q6KOkXjTH3lR40xnyzpPdLelrSe6q81vsk/StJT0r6LUkflTQv6acl/akxJlN6R2vtc5J+SNJOSZ80xpRPblhJxyX9TCkIr8UY805J/1t+EP6spF+W9CeSNkn6gSrHDQDOYmYYAOpgrb1qjHmrpM9L+pQx5qWSNkv6hKQ5Sd9jrb1d5eWMpOestRX1a2UzzG+U9Kmyz/0HxpjflPSjkj4o6WFjzDskvUPS54qPbeRd8gP3S6y1U8s+7+4qxw0AzmJmGADqZK19XNJ/knSP/BndT0jqk1868bUA17mwPAgX/Wrx5bes8rYfl/QVSe8zxrxb/qzwJUkPWWsLVX7qRUkLq4zncpUfDwDOYmYYAMLxC5IelPS24t8/aa39aJALFGt93yvpu+TX+26TlCp7l4HlH2OtnTXGvFl+jfCHJXmS3mitHavy0z4qvzTia8aYT8mf4f5ba+2lIGMHAFcxMwwAISjO6H6m7KFfXet9V2OMyUr6K0k/J6lTfjnEhyR9oPhHkjrW+PBzkp4qvv60pD+v9vNaa39FflPdRfm1zZ+RNGmM+WtjzDcE+TcAgIsIwwAQAmPMPZJ+SdI1+Ts8fNQY0xngEq+X9C8l/Xdr7Yuste+01v6ktfan5ZderOcnJN0v6bKkr5O/n3DVrLUft9aekrRL0rdJ+h1Jr5T0Z8aYniDXAgDXEIYBoE7GmA75M7lbJL1F/ozuixRsdvho8eUfrvK2b1rnc98v6Wck/bOkry++/IAx5hUBPrckyVo7ba39E2vtD0v6mPxt1h4Ieh0AcAlhGADq90uS7pP0n621fy7ppyT9raR3GWO+p8pr5IovHyx/0BhzWH498grGmJ2SPikpL+kt1tpJSW+W3xD3SWPMro0+qTHmtcu2ZSspzQhXuxMGADiJBjoAqIMx5jslvVvSafnbn8lamy9ut/akpN82xnzJWnthg0t9VtKQpB83xrxI/oEaByR9u6Q/Lr6+3H8rPv4ea+2Txc/9FWPMv5P065J+V9LrNvi8vy9p1hjzN/IDeUr+bPAJSf8g6S82+HgAcBozwwBQI2PMAfmB9Lqkt1prF0tvs9Y+L/9Aji5Jv2+MaV/vWtbaW5JeLen35Nf9vkfSi+XvFfy9q3zuH5P0nZL+aPnJd9ba35DfCPcdxph/u8E/4yck/Z2kl8rf5/gH5J989z5Jr7LWrthyDQDiJOV5q21pCQAAAMQfM8MAAABILMIwAAAAEoswDAAAgMQiDAMAACCxCMMAAABILMIwAAAAEoswDAAAgMQiDAMAACCxCMMAAABILMIwAAAAEuv/A+mf2ndG7C4SAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 864x432 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "Xd = np.array([[1,xmin], [1,xmax]])\n",
    "Yd = Xd @ theta_hat\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "fig.set_size_inches(width,height)\n",
    "ax.plot(X[:,1], Y, \".\")\n",
    "ax.plot(Xd[:,1], Yd, \"-\")\n",
    "ax.tick_params(axis='both', which='both', bottom=False, left=False, labelbottom=False, labelleft=False)\n",
    "ax.set_xlabel('x axis')\n",
    "ax.set_ylabel('y axis')\n",
    "pwk.save_fig('02-regression-line')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "End time is : Thursday 17 December 2020, 16:30:38\n",
      "Duration is : 00:00:13 289ms\n",
      "This notebook ends here\n"
     ]
    }
   ],
   "source": [
    "pwk.end()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "<img width=\"80px\" src=\"../fidle/img/00-Fidle-logo-01.svg\"></img>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}