N3_a_Random_Forest_Regression.ipynb 64 KB
Newer Older
Olivier Michel's avatar
Olivier Michel committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This notebook can be run on mybinder: [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/git/https%3A%2F%2Fgricad-gitlab.univ-grenoble-alpes.fr%2Fchatelaf%2Fml-sicom3a/master?urlpath=lab/tree/notebooks/8_Trees_Boosting/N3_a_Random_Forest_Regression.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## RANDOM FORESTS regressors "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Consider first the  same example as in notebook `N2_Regression_tree.ipynb`\n",
    "This is a regression problem. Rather than evaluating the optimal tree structure of a single tree, random forest is considered. \n",
    "See\n",
    "https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The number of point in the set is 629\n"
     ]
    },
    {
     "data": {
41
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEGCAYAAAB7DNKzAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/d3fzzAAAACXBIWXMAAAsTAAALEwEAmpwYAAArNklEQVR4nO3df3Ccx3kf8O8j0JcKgGMSJAyLIiiIIEUGzqiwfaYVyjJNEXRNhUO1mYJje1oznE5ot5bKspmkZsMmbYYz0jRlWcRxZLKyVHriRBWdKOawkh0SYShRHNYEVUWxIYHEkZQBUYaOgJQYgMZnQNs/7vbl3nv7vvfe4e7e9737fmY4AO4XFsBxn3f32X1WlFIgIiLyckvYDSAiomhjoCAiIl8MFERE5IuBgoiIfDFQEBGRr0VhN6Aali1bprq6usJuBhFRbFy8ePGGUqrddl9dBoquri4MDQ2F3QwiotgQkde97uPUExER+WKgICIiXwwURETki4GCiIh8MVAQEZEvBgoiIvLFQEFERL4YKMjT1EwGh8+kMDWTqcnziCiaGCjI07GhMTzy3Gs4NjRWk+cRUTTV5c5sqoz+ZGfex2o/j4iiSerxhLtkMqlYwoOIKDgRuaiUStru49QTERH5YqAgIiJfDBREROSLgYKIiHwxUBARkS8GCiIi8sVA0QDisFM6Dm0kalQMFA0gDjul49BGokbFndkNIA47pePQRqJGxRFFA2hrSeBLG7vR1pKo+GtXasqomm0kooVhoKBAvAJCtaaMmLMgig5OPVEgOiCcvzKJgzt6nSv/ak0Z6e8HAF/a2F3R1yai0nBEQQBuXsGn0tPWK/n+ZCc+uXopTo+kcfTcNed295RRpUYC/clO7Nu6jjkLogjgiIIA5I8YTo+kAWSv5KdmMjg2NIb+ZCc+dscSnB2dBOBdcVi/zmxmDs2JRehPdpaVd9ABiIjCx0DRoMwA0NaScK7c+3o6cM+qCedrcwpo54Y7c88WTM1krAFAP282M2+dOnJ/X782EVE0MFA0KHcOwLyC797Y6kwh9fV0AIDTeTcnFuWep6wjBv06UzMZNCeaCqaO/HIPzEsQRRMDRZ0xr8oBeF6hmyOIw2dSBY85eu4qBgZHMZuZw94tawue5zVi0MyAoQPOqeGJvMCj26rv+3hXGzatbUdfTwdHF0QREmqgEJEnAGwD8JZS6pct9wuAAQAPAJgF8OtKqZdq28p4Ma/KAXh25rojP3wm5fEYcX3Mf142Wa0wm5n3nIYy26NzH+aqKf299X2b1rbj9Ega96ya8G07EdVW2COK/wXgjwB8y+P+rQDW5P59AsBjuY/kwb1cdTYz53TmQOEIw2t5684NXXlTR+4rfHMaqjnR5NmZ9/V04PyVSTx8/xoAyK2auormxCJndKHzIu78iK1dRFR7oQYKpdTzItLl85AHAXxLZQ/2Pi8ii0XkNqXUm7VpYfy4VwuZnflsZg4Dg6OYnP4Zlrb+gtPp68e7g4H5Orb8QX+yE7OZOUxOZ3Do5CXs3NCVN7KYmsngwInh3ChhKQ7u6MWxoTHrtFX3xta8j+Z9RBSusEcUxdwOwNzyO567rSBQiMhuALsBYOXKlTVpXByYIwa9/2H4zZ/i7OhVAPlLYGcz8xgYvGxd2mobeehRxcBgttN3jyyODY05U0rm87b3LrcmuokomqIeKMRym3URv1LqCIAjAJBMJr0X+jcYc2Sgp5N08ti9BHbP5tXYt3Wd9Yrfa+ShRxWAFHT8ZsLcDETm6xJR9EU9UIwDMHufFQCuh9SWWLGtGspPRN9kjhbaWhIFS1vdr+WehjJXRdkcf/k6BgYvO4HItuLJbCdXPBFFS9QDxXEAD4nIU8gmsf+e+Qk7d8frd/Vu20PRn+wMnJ8IWt/JPVIxO373iiezndxPQRQtYS+P/TMAnwawTETGAfwegPcBgFLqGwCeRXZp7Ciyy2N3hdPS6HMvQzU7Z80MJkB+R+/XOdv2XJh7JLyu/N0jFbMds5k57Nm8Btt7l+etdEqlp/H8pTR2f2pVoBwGRx9E1Rf2qqfPF7lfAfhKjZoTa7YSHO6OM0gwsHXOXnsuzNdzj0jM57kdGxrDwOAo9m1dh+721ryVTgdODOPF1CQSi24pCC62gMDRB1H1RX3qiQJyl+BwM6/i/YKBH3cwyV9Rld3J/cLlNP7w8x/1vbr3C0r7t/UAGM59vMkrIPBkPKLqY5nxmCm3jLe+im9ONDkJ60MnR3Do5KWC0uJe38MMJofPpADczHHoBWpnRycLDjFyv57faXbd7a04uKMXp4Yn8r6/V9lxnoxHVH0cUcTI1EwGv/n0ywXJX/e0jG2axn3lrQMHALwy/k7eaxYrFW67ut+5oQvZlcuFy2SLTVG52V6fZceJwsNAESPmBjYzsezuWIN0tOb+B51Q1q+pk91ehf/0c806T20tCWeZrB6tAIKdG7ryHn/03LWieymKTScFCYxEVDkMFDFidqDuq3S/jyazUzX3P3RvbC1IVnuVCi9W58kcrej79eP1aiwzKLn3UfiNHmyjKia0iaqLgSImvA4asu17sO2i9tpf4d5lrV/T/TrudtiW2Gq23dru23Tn/vTQGFLpGac9xdjKgpgfOdogqjwGipjwO2goyPP0/ord993pTF3ZXrfYawa5ejenoczbzFFIf7Izr7x4sVVLtgBlG4H4LeHlaIOoPAwUMeE1neR1xezuWPX+itnMvHPmQ/fG1pKXl9oeH/Sq3T0K0tVk/Z7nLlj4/KU0kl1tnt/XvTnQb+RDRMEwUMSE3+Y124oi25W0PuJUX9GXMy1ja0fQq/YgU2ReK6zu7V6KT65ehrOjN/BiahKvjL+Du1csdqrdZmWT5/4HMhFRqRgoYi5IgtvrnAlbZ1pO8LCV+Cg1H+C3oe6Fy2mcHZ3E7vtW4WN3LMbF17PLeTNz72HP5uyBSDrvopPn3IhHVDkMFDHiVxEWKJza0cHDq0CgrTN1d9hBAkfxY1WL8+rY21oS+NgdS3B2dBK3Jm7B3i1r81Y+feound/I38PhPrObyWyi8jFQxEjQ2kq6c9QBwlYg0Hy8ybYxL2jnb9tfEZTX1Jrenb1n85rcpj5Y8xtepc6LbR4kouIYKGrMdoVeajK4r6fDukNb8yvvXYxtY575sdhzg5yjXQqzgGA5Hbxut948yIBBVDoGihqzXaGXmgw+fCZVsKzUaz/EQjtD21kVfiqdG/B6Pf0706ug3Od1A/m/EwC5c8PnfUdlRFSIgaLGbB2fLRkMwHoCnPssB69VTpVc6VPK9FM5NZn8RlRer2fuw3gxNemMYMxlwQdODOeNuty7zbnHgigYBooas3V8tmQwAOsJcOZUzKnhCc8yHpVU7RVE5XTYOk9x9NxV6CS2meT22sznlfwnIm8MFBFi67g+nttc5t44dnPuPZs8Bqp3VVztyq3ldtjuQoQ6SGxa247923o8D3Ayn8+RBFFxPI8iQsyzFfTnF65N4fRIGqeGJ6yPaU4swsDgZRwbGiv7rIqweZ0pkUpPY9eTP0AqPV30NcwaUAd39KK7vdWZXorb74MoajiiiBi/4n825pLUICW84+RmjmEYT+5a7/tYWwKfOQiiymCgiAh3TSPAXvzPHUjMJale+yXiyutYVBvb6izmIIgqg4EiAsz5dbOzt60Gsl0lV3I5bJR0t7fmjSSK7Tcpt8IuEfljoIgAc359e+/tTj7CthPbVg21UTrEYlNJHEEQVQcDRQj88hBmyYl3M/P45Opl6Ovp8Awa7tPh6lmxQNAoAZOo1hgoQuBXs8ksOXHkhasA4AQD8/G2PRbkj6fdEZWHgSIEfiuVzMJ+ZkVU20YxfRhRo0y1BF3F5BUQjp67ioHBUcxm5pwqtAwcRMUxUIQgyEolczNZKj2N33z6Zezf1oPu9ta8oNG9sbXm7Q9L0ByEd0ARAMC7mffyqusWPo6ITAwUIbGtVPK6wi1lP0E9C5qD8AooOzd05RUGdAdpjjCI7BgoQlLKkaKl7Ccg74BiTuvpwoC60KJthMHAQZTFEh5VVGpJjf5kp3UaSu8n6G5vnGmmSnL/HczNeXpaLxugVd7vXwfuY0NjIbaeKHwcUVSR37GiAJzKp/osBS7vrA6/M0DMKrM7N9yZN8Kw7VkhakShBgoR+SyAAQBNAB5XSj3quv/TAL4L4Grupr9QSv1+Ldu4EH7HigLAwOAoAOSdj8Bpjsrr6+nA+SuTTscP2FeOsUYUkV1ogUJEmgB8HcAWAOMALojIcaXUsOuhLyilttW8gRVQ7FjRyemfYfjNnxZsqGPnVFmnhidweiSNe1ZNOKvEvFaOmQdDcSRBlBVmjmI9gFGl1BWlVAbAUwAeDLE9FWXLT7hLhC9t/QWcHb3hbKgzazzFsVx4VPUnO7Fn82pMTmdw6OQlz9+rrrk1MDiK5kQTR3ZEOWEGitsBmFnC8dxtbr8iIn8rIs+JyIe9XkxEdovIkIgMpdPpSre1ZEESoWZwMIMIk6gLY0teNycW4cgLV5yzO2zMmlvu0QSDNzWyMHMUYrlNub5+CcAdSqlpEXkAwF8CWGN7MaXUEQBHACCZTLpfp+aCbA7zOw+62HPJm1eF3dnMHPROd81cYOBXhVfvoJ/NzGPvlrtq84MQRUSYgWIcgNkTrgBw3XyAUuofjM+fFZE/FpFlSqkbNWpjYLZzIvqTnQUrm4Lg6qeFsQVavdPd/XdyBxXv37tyfSRqHGEGigsA1ojInQDeAPA5AF8wHyAiHwIwoZRSIrIe2amyyZq3NADbUlh9xgSQXdmkOyH3Mln3aidu9FoYv0BrLos9uKM30OhNTzft2bwGOzd0Vby9RFEXWqBQSs2JyEMAvo/s8tgnlFI/EpEv5+7/BoB/DuBfi8gcgHcBfE4pFclLOttS2NMjadzbvRQfXv4BzGbmkUpP49TwRN4OYACea/zN26gy+pOdzt6JY0NjzihC5yBswfnY0BgGBkexb+s6Bm5qSBLRfndBksmkGhoaCrUN5qhAd/yb1rY7p9g1JxZxRBES2+/38JkUHnnuNezbuq4gOPPvQY1ARC4qpZLW+xgoKsOvM9H3NdpBQ3Fimw7k34saiV+gYAmPCnHPfZsdS1tLAn09HThwIlvYj51OtEzNZJxFB2/PZpxqvTwYiiiLgaJCbHPfJpYKjy6dgwCAV8bfcXJLaz74fty9YjGXKVPDY6AokdcUU1tLwikHbtYU0o9/+P7s9g+WCo8ec4/F9t7luGfVBCanMzjywhV88Z6VeYdGaUGmGjllRfWCZcZLpKeYjp67WrBTV9cUOjU8UfD4C9em8OSu9VjSnOAO3wgwd1rrPRZ7t9yF7vZWfGljN25NNAEAnr98A6dH0jhwYjjvOX6757mznuoNRxQBmQlpAM4paUD+7l/zo/5cn49tdjDm86j2iv0d9Gl4H+9qw9f++jL2b+vJO3N754Y7Adj3X3BnPdUbBoqAbBvqdHlwzTwQx9yhrc/HNh/PTiRcxf4O5qa9gzt6cWxoDO/+/L3cveK7qY8766necHlsQEHnnW3r8TlnHW/6b2ruf2FeguoNl8dWgO0q0bY/wqvOEINGfPkVC9Q4pUj1jIFiAdzHaQLZTsLvtDp2KPFjm1J045Qi1TMGigWwHafpLgboDgbsUOKpWIBnXoLqGQPFAtiO0zx8JuV5+I37ORQftgDPvRTUKBgoKizIfDbFjzvAmyNHW9kWTjFSPWGgWKBUehq/+5c/xIdv/wC+nDvKlB1D/dJ1oS6+/g7Ojt5Ad3uLtWwLpxipnjBQFOGuKuo+se7AiWG8mJrEi6lJLGWQqHtmXahNa9uxf1uPs+LNxAsGqicMFEWYK5vuXrHY6ST0iXX7t/UgM5cdUfDqsf6ZdaH0xYLOTxHVK264K8Kci96zeXXuVnGOxGTCsnGZ5cnN94O5r0bfxvcIRR033C1AW0vCKeHgPshG79gFmLBsROY0VHOuiKB7X42+DeB7hOKLgSIAPd/sDgxeCUsujWwM5jSU+R4w99WYjyWKK049lWAh9Z6ofvBCgOoRp54qJOhKFi6NrF/Fdt4T1SMeXFQG8wAbGx1QeLVZf44NjRXsvE+lp7HryR/gpdff5qFUVJc4ovDgN73AXbeNy7bzXp+H/uOpWaTSMwD4vqD6wkDhwS8Y9PV04PyVybyzsakx2KYf9VnpD9+/BheuTXFxA9UdBgoPfnkGfTb2PasmuNmK0N3eiid3rQcAfPSOJc7tOkDMZuYxMHgZAEcaFE/MUbjo/AMA5wyCVHo6b+65P9mJfVvXMVlNvm6OShX2bV2Hvp4O531ULM9FFCUcUbiYU05A4QYqnaTmlSGZ3NNLUzMZzGbmsGfzGqfUh7kPB+BGPIoPBgoXvYlqNjOP7b3LAdg3UBGZ3DktvWt7z+bVTgBxT2fq99nUTIa5C4o0BgqXtpYEmhOL8MhzrzmF/4CbBxPZzsnmf3JyB4GbwWAejzz3Gianf4bLb01j/7Ye5/1ie58RRREDhYVfItvrnGxqbHo6Uuce+pOdztfNiSa8cPkGzo7eADDsJL65MZMqpdor64oGChF5CMC3lVJvV/qbi8hnAQwAaALwuFLqUdf9krv/AQCzAH5dKfVSpdvh5peDsJ2TTaSZU1B6MUR/shN9PR04cGI4t5Q2i7kuqpRq7+0KMqL4EIALIvISgCcAfF9VoECUiDQB+DqALQDGc9/juFJq2HjYVgBrcv8+AeCx3MeaMyO2ezqKSDP32Lj/8+qRBFGlVXt0WjRQKKX2i8h/AvAZALsA/JGIPA3gm0qp1AK+93oAo0qpKwAgIk8BeBCAGSgeBPCtXGA6LyKLReQ2pdSbC/i+JeFaeCqFuceGU0tUK20tibwRbKWnnwLto8h11D/J/ZsDsATAd0Tkvy7ge98OYMz4ejx3W6mPqSr3Wnj+pyc/5h6bhdb84l4L8mJ7b+i+6tjQmM8zyxMkR/FvAewEcAPA4wB+Syn1cxG5BcBlAL9d5vcWy23uKa0gj9Ht3A1gNwCsXLmyzCbdpE8vezfzXt5aeCI/QfIOQROPrClGXmzvjWqOYIPkKJYB+DWl1OvmjUqp90Rk2wK+9zgA8ydaAeB6GY/R7TkC4AiQPY+i3EblTzVlTy/bt3UdgwRVTNAAwKkr8mJ7b1RzcUSQHMXv+tz36gK+9wUAa0TkTgBvAPgcgC+4HnMcwEO5/MUnAPx9NfMT5lkD93Yvxe77VuHWRBP/o9KCuEcQQQMAV0WRTRhFJkOr9aSUmgPwEIDvA3gVwNNKqR+JyJdF5Mu5hz0L4AqAUQD/E8C/qWab9FkD3e0teDE1iaWtCezdchdHE7Qg7rljM3dhzjUzJ0FBmO+nWr1nQt1wp5R6FtlgYN72DeNzBeArtWqPuUdC77omWqggGzg15iSoGPP9VKs8Fs/MJqoxc+oAQN7nR89dBSBcPEGBVHIaimdmE4XMrBGmT8Q7f2USB3f05l0Jsv4TlaJWeSyeR0FUA3qKQAeJ7vYWnB5JF6x5D3rWCfMZpNXivcARhYFHVlK16I7/411tAGA9NrWU9x/3WDQOv/eFuVITqN57gSMKQzV3NlJj01MEF65N4fRIGheuTRXs2ra9/7yuFnnKYuNwvy/M94ReqblpbXveCYqVxhGFgRucqNr83mO2+7xGDtxj0TjM94U5gpjNzAEA9mxeg+29y51pTaDyIwuueiKKME6Hkkkfp3tv91KICM6O3sC+resAZJdWb1rbjoM7est6r/iteuLUE1HIUulp7HryB0ilpwvuM6uCMnFNesox2dWGs6M3nCknfT57uUGiGAYKopDpKYMDJ4at9wfNnXElVLzZdumn0tPWv+n23uXYt3UdDu7oxanhCQwMjqI50VS1USdzFEQh27+tB5m5H2JNx/sxNZNxSnuYp+Ppw5D8cCVUvB09dw0Dg5cxm5lHc6LJeuSy7QRF/b6oZm6VgYIoZN3trfjUXe145LnXsDSXpHaX9tCHIfmdqsjFGHGnnI9mOaG7V7yB2cw8pmYyoZTvABgoiCLB7BgOnRxxzkExO31Wm61PevS4vfd2NCcW5R16BRTu1q/F+RNuzFEQ1YhfDkF3DHq++cgLV5w5Zya065seGZwanijYWzM1k3ES1e6AsNATFEvBEQVRjQSZKuhPdubWx0vR/RRcOlsfilUXHhgcDf3wNAYKohpxdwi2jr6tJYGdG+7M24XrlbBk8ro+mNNM+ghmXUE4KnknBgqiGnHnELw6en37+SuTuHvFYgwMXnYeo6ev+pOdkelEqHL0CAJAQU4iTAwURCHx6uj7k53Ossi7V3zAqelkKwAXhU6EKsc99WiOKPVhamFMQTFQEIXEPeVgTkMd3NFbMC11+EzKKQDHUUT8eU097t2y1rlfXxi491PUGgMFUQS4p6Hc01Tm6heeflcfvKYedZ7i4uvvOGU6Hr5/DQAU3XRZLQwURBFQLN8QldUvVDlef3MzT6GL/Oly4j+eGsKRLybR3e698bIaWD2WKERBl7h6nbPNoBFt5Sxhdq980iVd+r9xDqn0DDatbceTu9ZXvK2sHksUUUEL/pmbq/yew8KA0VLOYWg6T7F3y115uYsjX0xi09p27N/WU63meuLUE1GI/KacvK5G/YoEmktrq1VymoKr5BLm7vbWqowkguCIgihEfmUYvI7APP7ydZweSePU8ETBc/qTndi0th2nR9I80jcCallmo5oYKIgiyn0u9s1VMsrzvGy9tJbnaUeX30FVpihNI3LqiSii3EtkzWkMvytUWxVZ1oWKjptnWw/7TiVFqUQLRxREMWFOY5R6tVlOUpUqx/x77d/Wk5eU9vpbukeUYeKIgiiG3CeduZdTmvxKVVNleY3c3KMDcyThNXKI0vkiDBREMeKuJtvX05FX/0kXkjM7LG7Wqx2vTt9v9VMcijsyUBDFiLsj0vWf1nctQWJRkxNA3CMO8yNVR5BDhvQ0U19PB46//AbMcuJRziExUBDFiLvT1x9nM/MYGLyMU8MTWJJM5HVYUZrCqGfukZttGsrc52KOAgFEJnFtE0qgEJE2AP8bQBeAawB2KKXetjzuGoCfApgHMOe1vZyoUbg7ffNKtTnRVDDVBMA5vyKKV6r1xB28bdNQ5tnod6/IjijM0UdfT0ck/15hjSi+CmBQKfWoiHw19/V/8HjsJqXUjdo1jSh+zABidlhRWmJZ7/yWM7sfkw3si/ICgp5KjOLfK6xA8SCAT+c+Pwrgb+AdKIgaXin7ILyCRjmvReUzg8KhkyMwV6XpAD6bmcsLGFHNJ4UVKDqUUm8CgFLqTRH5oMfjFIC/EhEF4LBS6ojXC4rIbgC7AWDlypWVbi9RqModGdjyExxl1JbteFMzt+R3DklUVC1QiMgpAB+y3PU7JbzMvUqp67lAclJEXlNKPW97YC6IHAGyZcZLbjBRhHldaZZTpjyqV631xFzGPJuZw+77VuHWRFNeDsLMLUU1N6FVLVAopfq87hORCRG5LTeauA3AWx6vcT338S0ReQbAegDWQEFUz7yuNIOMDnjWdvUU22CnVzfpHdbuv4P+u0Y1N6GFNfV0HMBOAI/mPn7X/QARaQFwi1Lqp7nPPwPg92vaSqKICzI60KejmWdt2w7HodL5bbCbzczh3cx7WPPB92M2M4+j565Z/w7mBsqojvLCChSPAnhaRP4VgB8D6AcAEVkO4HGl1AMAOgA8IyK6nX+qlPpeSO0liiSdAPWbfrIVEzTnzQFg75a7atPgOqMDwmxmHlMzmbyDhpoTizAw+JpT9n3P5tXOyMK9rwKI5khCCyVQKKUmAWy23H4dwAO5z68A+Mc1bhpR7BTrbGzTVv3JTjx/KY0XU5MAFFdClUkHhEeeey1vH4uZC+rr6cA9qybQ19NRcIZIXPJF3JlNFHPldDZtLQl87QsfzasHFYcr2ygyTxx0/x51whoAjr98HQODl537gGgV/vPDMuNEMWd2Nu5y1X7lyM2y5VEqaR03p4YnnBMHbb/HIAdORR1HFER1wn01a1vt5CUuV7ZRZOYp9NfmNF7QA6eijIGCqE64E6u21U5UPncex/zazFMAiMUmulIwUBDFlLvjsiVWgdKuZP06w7heDVeKe8RWrJR7PQVnBgqimCpWndTWwfsFAgAFU1WNmuS2BUhzxJZKT2Ny+mf45Opl6OvpKBg11NvvioGCKKb8qpN67fR1d/xHz13DwOBlzGbm0ZxoKpiqMlf02NTriMMWIM0R24lXriOVngGQTWZ3b2wNra21wEBBFFN+c99eVWPNq+As5Xy0TVXpFT33rLJ3hvU44vA7qa4/2emU5bi3eymSXW11NcXkhYGCqA65g4h7BdSBE8M4uKMXOzfcmXuEACjs7N0Bxz2CiMuGsSD0z5Y9LdD7pLqDO3qt5U/c03j1NNJioCBqAHoF1L3dSyEiOD2SxrGhMXxpY7d1Z7H7TGevkhP1sKJH0z+bWWrDvP38lUkc3NGbNwUFKOc8CfN3A0T7aNNSMVAQ1Sm/0uJHz111ltHadhY/PTTmzMHrMxTM50dlBFHJHInXKjFzukkHV9t5EvW88omBgqhO2cpJaO51/zoPUWwOPmojiErmSLx+Nj3dZE4r2c4qr+eVTwwURHXK7+rf6+rX3SlGfX693BFOKSMRsxS4/qjLdUQtcFaLKFV/h8Elk0k1NDQUdjOIYmUh0zhxWyarlw/rXIRfEvrQyREMDI7ik6uX4uzopFM2fN/WdU6QiNvPbyMiF5VSSdt9HFEQEYCFTeN4PTeqHag5EimehM6uCOu57QO4b027UzbcXvivvqacNAYKIgLgP41TrMP3em5UOlBbuRO/BL35+c4NXXl5CAAFe0qiluSvNE49EVHRQKCnajatbXeWiJbzurbvU4tRhznVFDRgRXU0VC1+U088j4KoQfidTaGv/I8NjVkf39fTge72FmeJaFDmmRde38d2W6WVet6G3qBYTrv8fs9xxaknogbhNw1k24Ft7uQGgFR6xrNkedCRgt80TzWnbczlrIfPpIoWS1xIifaoTLdVEgMFUYPw65Ddyzy9OkqvqrRBD02yLSet5RJTr07cfXuxKrx+6jFfwUBB1CBK6ZBtu5T7k52eJ+a5O8eoHprk1Ym7b9d1rIKeEGiqx70VzFEQUQF3bgHIrxc1OZ3BoZOXnHl49+N1TqCUxLebOddfbN4/aF7A3U79PACeP2/Ugl0YOKIgokDM+kYDg5cBAK+Mv2MNBpW4qi6lyF6QvIAtZ3L03FUMDI5iNjOHvVvW5j3ePcqw5TYaBQMFEfkyO1idfwAULr7+Tl6hvFJeJ0hnW0qRvSB7QMwA595Md/PjTWaw8zoIqlEwUBCRL1tp8b1b1hacv6B5BYRSVwOZHXXQKSW/9u/ZvBp7Nq92qua2tSSczXR9PR04dHIEgGB77/K8Wk5A/hGzjTiyYKAgojy2w4n0WdG6gwW8O2d9vOoLl2/gDz//Ebw9m8GBE8N4+P41Je1lMC1kyanZyR84MYzTI2mndLp5dOzA4CgA5B1zataB8jtitt4xUBBRHtsIwixLXryDzFZ7ODt6A8eGxpyy5QDw5K71ZbWplCWnXuU6Dp9JOcl4d9DTwfDi6+/g7OgNJ4HttWy20ZLbDBRElMe2+c7rDGkb83jVvp4OvPH2LLqWNuPh+9eU3aZSkuO2PR26PDgATE5ncrkK5SSwbdNptqNe63HpaxAMFESUx7b5zjxDOsjzdadr7kO4cG0KXctaKlLKHIBzbrU7p2Db02EGjkMnL+VeUQpe1/2zN2pgcGOgICJf5VaVNfdd6JPyysk12FYtAXByCq+Mv+MEo/5kpxNA3p7NH0no9psJ7MNnUh6rocgUSqAQkX4A/xnALwFYr5SylnoVkc8CGADQBOBxpdSjNWskEQEItqLo/JXJgv0UXru7gfzVQ/p1vA4OMlctmcnw2cwc9IhCnw+hRz9AfgCxjRIOnbyEgcHL2H3fqrKT7I0irBHFDwH8GoDDXg8QkSYAXwewBcA4gAsiclwpNVybJhKRF3PeXyer3fspvOo6uUtjANnNdE8PjWHzL3XgyPNXANzs3G0BB8jmQo4NjWFJc/7ZErYAYmv/0LUpAMCtiVs4kigilEChlHoVAEQKN7kY1gMYVUpdyT32KQAPAmCgIAqRO/egz9j222NgTlHZSmM8PTSGVHoG8+/9JC9pXmxqyzaN1ZxYVHCutduxoTG8mMoea3oz+U5eopyjuB2AWQh+HMAnvB4sIrsB7AaAlStXVrdlRA3M3dGby0+98g9mp65HIfu39Tid+JEvJrH7W0NIpWfQnGgqmHbSr2mOZGwrscypsNMjacxm5qyBw2uUQnZVCxQicgrAhyx3/Y5S6rtBXsJym+dxfEqpIwCOANkT7gI1kohK5tXJ+iW9+3o68PylNCZnMjj+8nWcHknj7hVv5O4V7NzQhSNfTOLAiWF8vKvNGZl4rWDSgWDP5tV5+Q0dPPS002xmPu/xAPI22lEwVQsUSqm+Bb7EOADzHbcCwPUFviYRLZBXJ+vX+Z4ansCLqUm8mJp0SmnozW0A0JxoAgCnMzdXMZnMZLgZCDS9jLe7vRVLkgkcPXc1L3AwYV2eKE89XQCwRkTuBPAGgM8B+EK4TSIityDF/swk884NXTg2NIazozfQueRWLF98K/p6OrCkOftcHQRsy2nNYNS9sdUpUDibmcf23uXO9wLy9390t7eie2NrdX8RdSys5bH/DMDXALQD+D8i8rJS6p+IyHJkl8E+oJSaE5GHAHwf2eWxTyilfhRGe4nIW5C9EXoTntaf7HSmg8befhenhifwpY3dzvOXJBPWPRBAYWDyKi/SqOU2qiGsVU/PAHjGcvt1AA8YXz8L4NkaNo2ISmR2yEFLibe1JHBwR6+zOc4dCPxOlgtaf4l5iMqJ8tQTEUWY+5wKoLRzG9yjDP2aOkh4nSzndWxpuaVBqDgGCiIqi23KaaHTPWbZj7tXLC64P+hZF+YyWr/9FBQMAwURlcUWFBY63eM+btWdd/DKhxRbRmsrMULBMVAQUVncQaHUo079XnNqJoPmRFNB3sMvH2FOP5nLaIHhko5spUIMFERUEX4FAktlO69a77L2K8thjjb083WJEa5+Kh8DBRFVhLnktZJX7+Z0lF+inKufqkeUqr9qF8lkUg0NWSuXE1EVVWL6KYzXJkBELiqlktb7GCiIiMgvUNxS68YQEVG8MFAQUSRNzWRw+EwqV8+p+O1UPQwURBRJehXTsaGxQLdT9XDVExFFku18bfehQ1QbHFEQUSTpZa2nhifyRhD6dq58qh2OKIgo0jiCCB8DBRFFGjfMhY9TT0RE5IuBgoiIfDFQEBGRLwYKIiLyxUBBRES+GCiIiMgXAwUREfmqyzLjIpIG8HqZT18G4EYFm1NrcW8/EP+fIe7tB+L/M8S9/UDtf4Y7lFLttjvqMlAshIgMedVkj4O4tx+I/88Q9/YD8f8Z4t5+IFo/A6eeiIjIFwMFERH5YqAodCTsBixQ3NsPxP9niHv7gfj/DHFvPxChn4E5CiIi8sURBRER+WKgICIiXwwUOSLyWREZEZFREflq2O0plYg8ISJvicgPw25LOUSkU0ROi8irIvIjEdkTdptKJSL/SER+ICJ/m/sZ/kvYbSqHiDSJyP8TkRNht6UcInJNRP5ORF4WkaGw21MqEVksIt8Rkddy/x9+JfQ2MUeR/Y8B4BKALQDGAVwA8Hml1HCoDSuBiHwKwDSAbymlfjns9pRKRG4DcJtS6iUReT+AiwD+acz+BgKgRSk1LSLvA3AWwB6l1PmQm1YSEfn3AJIAflEptS3s9pRKRK4BSCqlYrnhTkSOAnhBKfW4iCQANCul3gmzTRxRZK0HMKqUuqKUygB4CsCDIbepJEqp5wFMhd2Ociml3lRKvZT7/KcAXgVwe7itKo3Kms59+b7cv1hdiYnICgC/CuDxsNvSiETkFwF8CsA3AUAplQk7SAAMFNrtAMaMr8cRs06qnohIF4CPAPi/ITelZLlpm5cBvAXgpFIqbj/D/wDw2wDeC7kdC6EA/JWIXBSR3WE3pkSrAKQBPJmb/ntcRFrCbhQDRZZYbovVlWC9EJFWAH8O4N8ppf4h7PaUSik1r5TqBbACwHoRic00oIhsA/CWUupi2G1ZoHuVUh8FsBXAV3LTsnGxCMBHATymlPoIgBkAoedMGSiyxgF0Gl+vAHA9pLY0rNy8/p8D+LZS6i/Cbs9C5KYL/gbAZ8NtSUnuBbA9N8f/FID7ReRPwm1S6ZRS13Mf3wLwDLJTy3ExDmDcGIl+B9nAESoGiqwLANaIyJ255NHnABwPuU0NJZcI/iaAV5VS/z3s9pRDRNpFZHHu81sB9AF4LdRGlUAptU8ptUIp1YXs/4G/Vkr9i5CbVRIRackthkBuyuYzAGKzElAp9RMAYyKyNnfTZgChL+hYFHYDokApNSciDwH4PoAmAE8opX4UcrNKIiJ/BuDTAJaJyDiA31NKfTPcVpXkXgD/EsDf5eb4AeA/KqWeDa9JJbsNwNHcKrpbADytlIrlEtMY6wDwTPa6A4sA/KlS6nvhNqlkDwP4du6i9QqAXSG3h8tjiYjIH6eeiIjIFwMFERH5YqAgIiJfDBREROSLgYKIiHwxUBBVWa4y7lURact9vST39R1ht40oCAYKoipTSo0BeAzAo7mbHgVwRCn1enitIgqO+yiIaiBXnuQigCcA/AaAj+QqFRNFHndmE9WAUurnIvJbAL4H4DMMEhQnnHoiqp2tAN4EEJuKskQAAwVRTYhIL7InKN4DYG/uRD+iWGCgIKqyXGXcx5A9Y+PHAP4AwH8Lt1VEwTFQEFXfbwD4sVLqZO7rPwawTkQ2htgmosC46omIiHxxREFERL4YKIiIyBcDBRER+WKgICIiXwwURETki4GCiIh8MVAQEZGv/w8suE5o6k3sNwAAAABJRU5ErkJggg==\n",
Olivier Michel's avatar
Olivier Michel committed
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Create a new realization od the signal\n",
    "import numpy as np\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "noise_std=.1\n",
    "X=np.arange(0,2*np.pi,.01)[:,np.newaxis]\n",
    "nx=np.random.randn(X.shape[0],1)*noise_std\n",
    "y=np.sin(X)+np.random.randn(X.shape[0],1)*noise_std\n",
    "print(\"The number of point in the set is {}\".format(len(X)))\n",
    "\n",
    "plt.figure()\n",
    "plt.scatter(X,y,s=1)\n",
    "plt.xlabel('X')\n",
    "plt.ylabel('y');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In **random forest**, each tree in the ensemble is built from a sample drawn with replacement (i.e., a bootstrap sample) from the training set.\n",
    "\n",
    "Furthermore, when splitting each node during the construction of a tree, the best split is found either from all input features or a random subset of size max_features.\n",
    "\n",
    "(Note that on thour example, we only deal with a single feature.)\n",
    "\n",
    "The purpose of these two sources of randomness is to decrease the variance of the forest estimator. Indeed, individual decision trees typically exhibit high variance and tend to overfit. The injected randomness in forests yield decision trees with somewhat decoupled prediction errors. By taking an average of those predictions, some errors can cancel out. Random forests achieve a reduced variance by combining diverse trees, sometimes at the cost of a slight increase in bias. In practice the variance reduction is often significant hence yielding an overall better model.\n",
    "\n",
    "*(extracted from https://scikit-learn.org/stable/modules/ensemble.html#forest)*\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
94
      "MSE = 0.001900837145744446\n"
Olivier Michel's avatar
Olivier Michel committed
95
96
97
98
     ]
    },
    {
     "data": {
99
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXyU1bnA8d9JwgBJ2CaEACEQSCAYFlECIouIBAXEpUooai1aLVi1otXbKy3Xtuot3rZUudpboSiCGwVXRBAJIiCIEgRRAoEkbAEMIQEkCTAkOfePyTvMmnUyk5l5vp8PnyQz78x7svC85z3nOc9RWmuEEEIEvzB/N0AIIYRvSMAXQogQIQFfCCFChAR8IYQIERLwhRAiRET4uwE16dixo05MTPR3M4QQImBs3779pNY61t1zzTrgJyYmkpWV5e9mCCFEwFBKHfL0nAzpCCFEiJCAL4QQIUICvhBChAgJ+EIIESIk4AshRIiQgC+EECFCAr4QQoQICfjCRUmZhfkb8igps/i7KUIIL5KAL1wszzrCnNV7WZ51xN9NEUJ4UbNeaSv8IyMtweGjECI4SMAXLsxRJmaMTvJ3M4QQXiZDOkIIESIk4AshRIiQgC+EECFCAr4QQoQICfii0SRvX4jAIAE/yPgj+ErevhCBQdIyg4wRfAGfpVZK3r4QgUECfpCpLfiWlFlYnnWEjLQEzFEmr5xT8vaFCAwS8INMbcHXuAMot1QQaYqoMfA3xcVBCOE/EvBDjNHzL7dUOgz9GME9PTWOzOxCMtIS/DI8JIRoOhLwg0B9euLGHYB1UldTbqm0vX7O6r1szS9mfU4RYL04lFsqbMdIL1+IwCYBP4AZgbq4zMKCjfmUWyp5bFyfOr3WHGUi0hTBnNV7iTSF23r+6alxDOtVaLt4GMeArnUISAjRvEnAb6Zq67WXlFl4fNlO1ucUMSIppvpRXa/3Tk+NA7Cdwxi2SRodbTvW3RCQMdyTnhrHip1HAcW04YkAMuYvRDMmAb+Z8jR+bgTrcksl63OKGJMSy+xJqbZx98a8tzvmKBMZaQks3nKAmWN7V39+kHnr9rNp/0m+yD0JQKQpHKBeY/4yKSyEb3kl4CulXgUmASe01v3dPK+AecBEoBy4R2v9jTfOHazsh1jmb8izBUUjWM8cm8ysCX1tj9v3ymuTnhrH1vxiWw+/tsC7POsI89blMmtC3+rnrXcSqV3aMLhHe0A5XGya4sIjhGg8b/XwXwNeApZ4eH4C0Lv631XAP6s/Cg+MIZb5G/IcgqJ9nn1D0ykzswtZn1PEsF6FJI2Odgm8zq93zu2fNrynx/H8+gRuWbAlhG95JeBrrTcqpRJrOOQWYInWWgNblVLtlVJdtNbHvXH+YOYuU8ZdULUP0vYB3PjaPjg7v6dz4LXP2Jk7ZZDLOY2LQGOHY2TBlhC+5ataOvGAfaGVgurHRA2MIA6Keev2szzriMdaOfb1bDLSEpg1oS/pqXE8vmyn7XHjtQCRpgjbexqB91S5hbv+tZWjp88xMrkj63OKPNbHaUj9HCmyJoR/+WrSVrl5zG1KiVJqOjAdoHv37k3ZpmbPCKrTR/VkTEos6alxHse9nYd6jOEgY2LXXc8fLs0RpKfGMX1JFnlFZWzOK2b6qJ60CFe2cX6DcREakmi2tammISRPdx7SsxfC93wV8AsA+4HabsAxdwdqrRcACwDS0tLqlmcYpOxTIq2LobKZPSnV9pxzoHUOovZB3VMapjFHsDW/mLyiMhJjIrm+X2datwhnfU4RA7sddRivN4L2mJRY2zwA4DIHsHjLAYzr/Lx1+9m4r4h+XdvZMn2EEL7nq4C/AnhYKbUU62TtGRm/d88liI9MpORMOdkHTrBh9zE+6xZNxuAE3t2SyzlLFS+tzyXMcoFfXpPk8j7vbj9iPXb7Yeau2ceF0nJam8LhwgWIsF5LMwZ0Isxygesui2N4t2huH2w9b0mZhWgqOGe5yNw1ewizXOD2wQlcKC3nN6N6cOPlXazHD+gEYH1+QCe4eNGW1QMwIimGkckd+SL3JJvziu0yfYQQvqas86iNfBOl3gauBToChcAfgBYAWuuXq9MyXwLGY03LvFdrnVXb+6alpemsrFoPCyrzN+TxPx/vZkHrg6QX7YXXX4fSUn83q16qOsZS1D6Ws5Vw5kIVUSlJnI9oyenyi1yR3Imq2bNZdrRS8u+FaAJKqe1a6zR3z3krS+eOWp7XwEPeOFewyxjUhUmP3kX8zq+gZUu4/Xbob13a8NWBYjbkFDE8KYYWEWEMiG/Pd0dPsyGniNEpsQC2z43n7I8xXpfcKZrcE6Ukd4pm/d4T5BeVMTollqt6xrhtk3HeIYlmissuMKZvJ2KiWro9trzsPIe+zSGp4kc6VFZSVHyWzgd2c67sPD+er6Dd5pN8EdaWOZ3GAjKWL4QvyUpbH6hXcbPDebDzK/jTn+B3v6PkQpXttR3LLexZmc3tk1JJirUutOpdZmFn1hF6V4+LG59HRplsCx2MY1Krz2+M249JimV9RRFjbozl/imDwEPbjNfvtFQyb91+Zo3r6xKo7VcAz6vYz6wJ1mOMVCxLmYXfL9vJfzw5lQ5bNjLzxQdcFpUJIZqWbHHoA3VJYTRSFs/us6ZNcv31EBFhe+3iLQd4dmU263OKyMwudEixnDE6CXOUyTYRax887evmLM86Ql5RKeWWCmaO7c3sSanMmtDXlmvvyaXFV5qZY3vbArV9euWlDBxtWwHs/B5zpwzi8BVXk5y7izb6IpnZhbI1ohA+JD18H6jLilIjYKZadjEKoEcPh9fY187xlOLofCdhX2DNKHtsfJw1oS9JsdF1LslgX17BCNT2567LCmBzlIkRM35KyzVvcc/v7+XC3dNQ149jsg935xIilEnA94G6rCg1AmbaRxugRQuIi3N4bUmZxaGMsdFLN752Du5zpwxiedYRhwJrw3oVOpQ/ro19wHV30bIfkqmrNhOuhylTiNizh4iZDzP9r3+F655wOZ99CijIWL8Q3iABv5mwXRTmF1DZrRsLNx1w6NnaXzTmb8hzKmaGLbgnxUaxPqeoOg8eZo7tzbThiQ4F1urTq7cPuPZB17nOD+CydaLxHvbfR0llGMsf/DMZg7thvmUivPACzJwJLVq4nM9TATkhRMNIwG9uDh/mh3adXGrZ2HOudgmOi6wyswutk6fr9jcq791dr762Xr9RN7/cUsGugjO23bOMi4VDUP/Nb2DSJPjwQ5g82WPdIOcCckKIhpGA7yPuxteNr8GuJ3z4MG2vHmnrqS/POuIS5JyrXYLjHUDS6GiXIaD6tM3gbiiqtl6/sXXi9kOn+SL3pG3OweBwkWiVaB2+ysqCyZNdduFyNz8ghGg4Cfg+YgTKcksFgC0gGuas3ouqqGD60aPsa2kmr6jMJVga6hIAnYN1TROg9Rkrr+3cRtA2gr3zHYpzuyp79uLQlzvRRaVkZhc6lH/w9BohRMNIwPcRYxjm3MUqFmzMB3AJ6HfmboKqKlKGDWBWv74ex6wbEgBrCur16UHXZwLa+Ohu/N2WLmruysXsvfxPdcqp0T4j7VTG7YXwHgn4PmIMwwzs1o6ZY5Mx9oE1gtmML5fDrFkwciTRP7uDGW3bejXo1RTUvdWDtr+LsJ9gdnehMS5Ag7slMuibLcye2JdhvWJcavI7v04I0XAS8H3EU556SZmFLS8vZdKsWTB1KixaBK1aAd4Ner4YFnHXXk8XGuPrHtsGEfHOImJOnWDG6GSX52XcXgjvkZW2PuJuFSxYg2S7F5+nLKYTvPYaJZVhtlWsxkYm9Q16/tpoxF17PX3fxuNbI8wAfLHqS7fPA7JpihBeIj38JmBfD37a8ETMkS3g228p3fEdX+eeYGivGKJbRsD589yT+RktD+2kePafWLDxoMNkrnMWjKdzOU/G+nI4pLaa/LUZeeMIeAzGFuW4fb4u34usyBWibiTgNwH7evBxxce48+kHYc8eooHrnI41RUbCE0/w3jWTba/xlJ3j6Vx1HUZpCo29uHRIToRbb6X1X5+Dndth9mwYMcL2vLs1B95ugxChQgJ+EzAWEIFi4rv/S+X+XM7933wq04by7u4TGDtBvbr1MD/PGMnkYT0p3XKA6aN60doU7jCZW5dz2X8E36YxNvTi4tArX7IEHngA1q2D22/n1JdZLDt8gYy0BLdrDrzVBiFCjVc2QGkqgbABSo3DCZWVlHWOZ0v7RPIXvumwatSaqQP22wAaJYVDgfFzcPied+2CIUM4FxnNlpgkOl1+GT0euIe3Td1luEaIOqppAxSZtG2kGksfv/8+UScLsWRMsZULMCY2pw3vSaQpgnnr9mOUFHZXdjgYOE8il5RZ3BZ/m38qijOfbSR8+NUMoJSUVe/Q9vrrmPHybMxc9Oe3IERQkCGdRnI3nHBmaxbnf3E/nfL3oi6/nOPX3mAL7PaFxYygZwzhBGvNGOcxdvtSyy4TzRP6MuPjlby/IY/nV+zk9dNfMGTRPIiJgZdecnhfmawVon4k4DeSu/Hy7CXvcvWeb8kZfzspr7/Mba3bUtmyla2wmME56AXrWLTz9+U8EWvf4zfucqzPDSIp7SYoPUbVsuUsvO0RJl+VCHBpd611+4HgukAK0VQk4DeBK8LKsLSOImLxIubvPkFGWluPNe3tK0MGa80Y5+/LeSK2ts1VuO02wpYtY+0r71NWeautCueIpBiHYSEhRM0k4NdTXYYRWhUehx4JZO454RC8nAOfu8qQocC5x+/uzsYhiE+ciG7ZkjdW/JnyDS/zfueBHJ/8EJvzirmmT6wM5whRRxLw66lOOd8FBRAfX+sQTbAO4dTG+cLn/LXLz7VNG0r/tYiDb79Py+KT/OLrFfz88FZ2j7mJnmcnQ2kcRNdtUxchQpmkZdaTux6+y2MJCTB2LLz2mn8bG+Dsf672F9oXO5/ipo3vwUcfQUUF9O4NGzdC585+brEQ/ldTWqb08Oup1k1BRibC8ePQrZsfWhdc7H+uxmK2c5Yqck3hlPxyCubzZzm76lNaTb8P/csZmD76UDJ3hKiBBHwvcBiaKSyEykpbwJcAVH+2Wvl2m6GYo0w8Ni7FlrpqzHss7HwlXfqM4pYNGygpvcDjy7912VZRCGElC6+8wKEiZEGB9cHqgF/jwiz8V9myOTN+ZpnZhQ6VNt0t2ALFri69aX32DJ98tIX1OUX1qkUkRCiRgN8ANQZpI+DHx3sIUI5quyCEIk9loS8VpdMs3nKA59fu4+ZBXRl82zgAris9zJiUWGZPSpW7KSHckCGdBqgxU+e770ApSEx0u6LUWahm6tTE03qES+sXLi24ijSFM+OeCfBrE6c++Jgv+0aT2SvGY6E1IUKZBPwGyEhLoLj0Apv2n2RIopltB0sujdF/9BEMGwYdOpCRFmU73pNgXWzVELXNdxg/K+udlQaU9WdrMkFaGpetWs627Zs5NeVj22rdzOxCmT8RopoE/AYwtwyjVeandN53iDfXteUYLel4XT/S+3Sk3fbtlP/xGSKRYF5fnu6cnC8ExgSugzfegI0bafPEE7S6djjxiWmsHzGGOR2HuryfEKFKAn5DPPkkv3lhruNjSy99urLHYKb4tkUBzV1Wjr06XQh69oSePeGaa6j67ZOMW7WKcyV5nH3jLhkuE6KaVwK+Umo8MA8IBxZqrZ9zev5a4EPgQPVD72mtn/bGuX3u/Hl49VW49VZOPzOHtV/lckN8K7Zsz2Xtln0k9uzMXRlja3wLSdV0VNvqZed5DuPn57Z4Ws+etFz+b76681cMXroAVXlRfsZCVGt0wFdKhQP/AMYBBcA2pdQKrXW206GbtNaTGns+v1uxAk6dgl/9ivb9+5LRvy8AQ0ddw6GRl4J4TUFdtuRzVNvEtfPQmPHzmzk22baPwPNrc7DtIRxl4sfO3YjQVUSfPAH088F3IUTz540e/lAgV2udD6CUWgrcAjgH/IBlvyn5g+++T8u4OEqGjWL5hjxbkKrPRuKSmeOovnMd9j8/Yx8BYz9gY0HWVWOugOdhSsfKJmmzEIHIGwE/HrBPIi8ArnJz3NVKqW+BY8ATWuvd7t5MKTUdmA7QvXt3LzSv8Wz531pzz5pMuH4sy3ccc6htP2f13up9bAEUNw/qCrgP6jKZ2zjOPz/7PYQz0hIoKbOw5seW3AG0PXEMkGE0IcA7AV+5ecy5Its3QA+tdalSaiLwAdDb3ZtprRcAC8BaPM0L7Ws0I6Ac++pbOpw5ycbuA9z20l3ywyWo+4R91k5JmYXHl+1ky+4fuQPg4EHg0h1XuaXCtuuYx+J3QgQpbwT8AsC+G9sNay/eRmv9o93nq5RS/6eU6qi1PumF83udpzTAsu/XADDw7ltp76akb0mZhXOWCrKPn7VlnAjfWp51xFpeoV88VZ27cDHvAK/ZdtDCYdcx4/cncyoiVHgj4G8DeiulegJHganAnfYHKKU6A4Vaa62UGoq1pEOxF87dJNwGgG++IepPT0H//rQfmOr2deYoEzHRLfki9wCZ2YWy2tOH3KV2hi3owcnvcxx+l867jhnH2n8UIlg1OuBrrSuUUg8Da7CmZb6qtd6tlHqg+vmXgcnAr5RSFcA5YKpuxoX4XQJAVRXcd591k42PP6ak/KItuDiv5JTg4R9uL9J9+tDl41X8blwyY6v3ys1IS3DpxcuciggVXsnD11qvAlY5Pfay3ecvAS9541y+4BIAli2DnTutqzm7d2d5dYnerfnFLqV4JXj4h7sL7dnrJ9JmyRKmnj/I29kRtt/Z3CmDZKxehCSpllkDoyqmZfES6yrOO+5wqIA5e1KqraqjlDn2L4cS1dWWdexHeYuWHPy/V6ksLuaOsEJ2fbOfxZvz/dhSIfxHSivUYHnWEZ5blc09X2yGqVMgLMyhAmZSbLRtnN7YmANk4q+5+MnIPhy5biIDVy2jz6cf0KrCwhzg+NqBsPdbfzdPCJ+TgF+D9NQ4CjZspWXpj3wWm8KgMovHMXoZu29+zFEmzO+9Qflzf+XgzhzMt95I+eLX6bVxDZw7B61b+7uJQvhUyAf8mnKw1313jPZrVwPwVEkHerz1DWmJZtvyfXsydt9MRUYS+fQfMPKqSvVF2LiG03v20/7KgX5tmhC+FvJj+Iu3HGTO6r0s3nLQ8YmLF7nnuV/z+BdvcrFHT3oP7c/mvGLmrdsvu1MFsI/PtgTgs0++9nNLhPC9kO/hG4uCtx8qoaTMcqnn/tRTmNZ+Cs89R4tf/IK5ke1s9XRk2CZwnYmz7jXcrvCon1sihO+FfMCfNrwnuwrOsD6niOVZR6zDMqdPo198kdyxNxHz8GPWsWBw3XRDBJzJk4ZQ2aIFw8PP+rspQvhcyA/pmKNMzJ0yyFZmd/6GPMpfXoAqK+PRbmOre/UiUDmny5rbtIIePTi2c49LCm1JmYXn1+bw/Np9kl4rglLIB3y4NOGamV3InNV7KV38Jsf7Xs7uzsmcKrvIvYu+Jq+o1N/NFA1grMC1n3c51j6Os3td52IWbznAvHW5Mk8jglZIB3z73p+xoOqRQWZic76j5U03MmtCX/JPlrE+p4hnVwZNef+QkpGWYFscZ4gd0JeUspNkDO7mdLS18OvI5I4yTyOCUkiO4dtvkfftgre4su0ZLraOpDD3R0aF/YjSmo09BjFjdBLpqXE8uzKb2ZPcF0wTzZs5ykRGWoJD6m2rIYNh0Su0LiyA6EuptNOGJxJpCic9NU7KJYugFJIB37jNf/vYJzz2zqUSP8Ptjhlw2zgAkmKjWXTvUB+3UHiTS2G1UaOsT2zaBEmXAr4xtCerpkWwCsmAn5GWAFozdPI0Dg27ljYffYA5QnP6h5N8+tQ8ss6GkbSvmKQu7f3dVOEFLqugU1Op6tCBfcs/plPGnZiXvAJvvgmvvw7HjpGRNsTxeCGCREgGfHOUiRldKqG4mH8MGUjS7pPMGJ3Ev78tZk6v8YxJieVJ+c8eNFxWQYeFcfiyK+j82SccHnYtHXZvRWkNvXtDZSXm++9nxuDBcM0M/zVaiCYQspO2pes+ByB18nhbtUujCqaUzw1+MY8+xJlO8eiTJ9l61Q2cfPZ/ONUtkQu3Z8DChfCrX/H0HxZLdpYIKiEV8O2zcgpWZlLSui0Xkqxb6z6+bCfz1uUSaQqXYB8C2mTcRkVWFr95YgF3jH6Yn7YayhVT5/Har+fA4cNUhoXTIfMTpi/JIq+oVEpfi6AQlEM6ngqiGZN3EefPce+2DRwccjUZQ7pf2gc1JVbGbUNASZmFxVsOsP3QafKKykiKjSKvqOzS7z/KhGXY1dyY/zVzi+7m2ZXZLhvdCBGIgjLge9qU2gjmU7/6gLCTRXw3+V5G4TipJ7374GfsaQAwJiWW2ZNSXbaqbH37T+j1+OM8k9ae4aNTGdjtKOWWSsd6S0IEmKAM+J5q09sm72a+zg/9r2Tm0WhmVdfPkZ5b6MhIS6DcUgEoW6lrYyMbo/cfr+OYAtwdfRZio4k0WbdIjDSFy9+KCFhBGfBrrE2vNezbR7v7p7uswBShwRxl8lgIz+j9tz8XxhSA3bth/HjZ4EYEhaAM+DU6fhzOnaN13z7SUxMu8z32vf+qpXGE7d4NyAY3IjiEXsDPy7N+TJL/vMJ1vseh99+/n7WHL0SQCJm0TCMls3S39T/328UtJM1O2IqrGaWxHf4m+vWD7GzrMKAT57LLQgSCkAn4Rk9u3+adVIWF8187zkoJXOFSGtvhb2LgQCgthQ8/dHmdu7LLQjR3ITOkY0y29XtpCbpHdx66vq+k2Qkbt5Oyd9wB8+fDXXfBvn0QH1/z8UI0c8HXw6+shDlz4NNPHR42R5mYsegZTB99yLGYroDyuNGF3K6HHqOn73Dxj4qCpUvh3DlYsAC49LcBuB4vRDMXfAE/PBz+8heH2/CSMguvv7sFFi/mh979+c+kCYD2mJYpt+vCJikJxo+Hf/0LLl6Uvw0R0IJzSCcxEQ4eBCCvqJT7XtvGyLXLAWj92iuMLo2ucVWt3K4LQ0mZha9G/YQJq1dz9t/vUh53BTPH9pa/DeFVnsrBeFtwBvyePWGvNRtn/ZN/4b/XfUjfEwcpTuhFzODLqa3oreRcC8PiLQd48VQcO+K6cmbuPObdMJuZY5NlKEd4ladyMN4WfEM6YA34Bw/ChQv84s2/kFxWzNnel9HyD//l75aJgKOoCgvnu4lT6b5zK9O/epfkVe+y8NPdMscjvMIozT59VC9bIklT8UoPXyk1HpgHhAMLtdbPOT2vqp+fCJQD92itv/HGud1KTLROtH36KWEXLhD3xguUTLiZN7OOkCFZOaIejH1uU1OGcfHADn73+SL4HL565y123XEX1/75PyAiOG+UhW8Y5TzGpMSyPqeoSes1NfovVSkVDvwDGAcUANuUUiu01tl2h00Aelf/uwr4Z/XHptGzp/Xj229bPw4bZrtl2ppfLBuciDpzGN5b+ynvLVzB1g838PS6+bT66+9gXBolw0fLpueiwYz5oPTUOIb1KmzS+SFvDOkMBXK11vlaawuwFLjF6ZhbgCXaaivQXinVxQvnds8+4MfHQ7duZKQl2K6gkmEhGiQigmvvnkTRlLsYc//L1sdycyVzRzSK0alIio1u8lRfb9yLxgP2f+kFuPbe3R0TDxx3fjOl1HRgOkD37t0b1qLExEufDxsGWH+oc6cMsvXEhGgI299RYgf0qy1RBw6Q8XPJ6hKBwRs9fOXmMefiI3U5xvqg1gu01mla67TY2NiGtSgqCsaN4+Kwq1k+8V7bJIhRDXF51hGZcBO1sl+AZ/+5OcrEjDG9UT16wIED7hdtCdEMeaOHXwDYd226AccacIxXlby/kseX7WR9TpFtkxPwXfqTCHzG30q5pYJdBWdYn1NEuaWCSFME6alxhLWNI3LnHlpIIoBoAF/l3tvzRg9/G9BbKdVTKWUCpgIrnI5ZAfxcWQ0DzmitXYZzvMnYpzYpNor01Djb40Z1RLn9FrUx/lZA2fY8BsWc1Xt5dmU2X1S2oWXBIZexeynNIZy5+5twmPvZtg0++qjJ29HoHr7WukIp9TCwBmta5qta691KqQeqn38ZWIU1JTMXa1rmvY09b20y0hLYml/M+pwiMrMLbVvYyaIqUVfG30pJmYVIU7itkxBpCic9NY6jW/rRfscqro9vyfwNebbnjTtLkLtIYeVuZMHYbKf8/EUqf/kzwo8dhZMnoVWrJmuHVxKItdarsAZ1+8detvtcAw9541x1YdwqzZ6U2uRpTiL4OXcSjM+Txl8FC2HHb59lYdcRwHDKLZWszyliZHJH+bsTNu7KtZi/zeK6ha+wO+cY4fv3WR9ct468oaN5dmU2syelkhQb7dV2BOWKERmnFz4xZAhl5lhue38+N0W8Svldm3nV0haAwT3ay7i+sHE7sjB7NgM3buTyykosXeMJKyoiYtIkotvH8dsWrXmWf7Po3qFebUdQBnwpfiZ8ont3ftiTx+J/fsjTf7ybdp9nMu2Rx4k0RcjfnqhVZX4+Xw9J57Er7uDq5I5c/doLTPkuk68796GsUxdmT0r1+jmDspaOpF8KX8ncc4Il5zpwpmt32L7dJUVTJnCFs5IyCy98shuOFLBdteOyIZfx8N2j+fFvL/D3pVv49S3/yfHZz3h9OAeCtIcPMqwjmp5R9Grm2N60yr0KvvoS/vlP2LULZs+G+Hj5OxQulmcdYfmHX/NoVSVRKcm2Ui9J1/ejpMxCVOemWxwatAFfhnVEUzOKXs2a0JeWw4bCu8vhwQetTw4YAA8+KH+HwkVGWgIdsqyh9/bbR9LWbq6nqbMIgzbgS/qlaGoOwbxqsPXB1FTYvx+OWHPza/o79MfCG+E/9r/vKR2rAGjbt7dP2xCUY/hC+IL9eH3J4GFseeh3nFqdaS3Yd/hwreP3UnQttDj8vg8eBKUgwbd3fhLwhfCC5TuOcWf0cJYdKIfu3eHIEYf/4O6Cv6z6Di3pqXGMSYm1rvw/eBC6doWWLX3ahqAd0hHCl9JT49iaX2z9z9y9O2za5DDk427yVoYdQ0tmdiHrc4oY1quQpJ07oU8fn7dBAr4QXuDwn7l7dygowNwq3FaawYcPsdUAABseSURBVMjmkd586DHG7o2aXj/tDOzcCX/5i8/bIkM6QniBw/BM9+5QWQk//EBJmYXHl+1k3rpcQLtdGyK5+sHF+fdp3N2VPfQIM1a+TPt1a6wH3nSTw/F5RaVN/ncgPXwhvMB+sd+dsZ1pA1y882ecyz1Exch7GTP+BoxKm+CYky+5+sHB6MmXWyqZt26/YyntiosMuHW5da/tlBRITrZ+5NLv3yj2CE33dyABXwgvMf7jdkgxMQVosfFz2ppa8+zWN2nzym9BKYeqmwbJ1Q8Oxu9/+qiejEmJ5dzFKuatswbyF3paUKWl1gOzs+Gtt6xZOvh2T1sJ+EJ4iTFxO3hiX7Y+8J/8uTyOW8JLuG/Rs3DXFJg7lxmjk2238MZ/bMnFDw7G79OomDqwWzvbPtqfrlpNBsCjj0K7dnDHHbbX2U/eG2Xcm4oEfCG85NLEbQwZf3uaG7OO8JOBcRB2HJYuhSeegA8+cBjCAWQ4J0g4758w7Z3/JWLD51zIO0Crc2WUJCZjfv55wH+L7iTgC+El9kMzDimXCxdCly7w3/8NeXmXNr6wVHLzoK4Or7UnK3EDk1lVMCP8OLz0vxRedjmtJmew9/R5ut9xq+0Yf83bSMAXwktqzKt/8EH0n//M9mfm0esffyPSFMGc1XuJNIXbXuMc4GUyN0CNHAk7dnC+bXuuu+H3PHLrlS6/YyNF09fzNhLwhfCFLl04mdSX859vtAV1cPwP7xzgZTI3AFVWwo4dAHz24O+5a2R/yi2VlFRvdO/vi7gEfCG8rKTMwuItBwDFtOGJtuGYNteN5qrFi0kd1MXt3YBzgJeVuIHD6LlP6aLoAGz8zTM8qC9jzImz1amWmkhTBEMSzYxIiqG4zGK7CPiSBHwhvMwomwzYhmxKyix80ymF9PPlmHP3wJVXwhtvwIgR0KsX4JjLL+P2gcXoucd1L+dW4IoRA5gV09eWalluqWTO6r2MSYllc14xm/OKifHDBV0CvhBeZkzKgnJIvXztZDvSgUMPP0Hnrh1p+d47MG4cfPqp7bX+vuUX9eM8Jp9+cCsAbfokMaP/pVRLI3MnPTWOgd2OYv+34UsS8IXwMnOUicfGpdi+NmrpTPnJcDaUPsLQt+cTrqpg8GBYuxY2bbJO9Ckl4/YBxuUCve196xMeyh53iHT82/A1qaUjRBMzhngiTeF8c/eDDJr5Ni+t2AFr1kBUFFxzDfzHfwC47IkrmjeXEteHD0PbtpREtHZbT8ffex9ID1+IJubca480RVg/jzJZMzp+/3t44QXKo9ux9WJrBj16P+bY9v5ssqD2dRD2z3PuHJ/+bQHXrf+ciATXctjN5c5Naa392oCapKWl6aysLH83Q4imdeKEtZjW2bMA5Iy/nZTV7/i5UWL+hjzmrN7LrAl9XeZTjCqo63OKeGaomVsemkLbHwoAKO7Zh9NfbSczu9Avk+9Kqe1a6zR3z0kPXwh/69QJdu7k9LmLHHv0P0na+AklJWcxm9v4u2UhzX5FtH0KpRHsq1Z/woc5a7hsUT5Vp0+zcuYzTJr3X7zbsR86u7BZTrrLGL4QPuSp9n1JXDf+fTKCfddMoGV5GV++Ij18fzNHmYg0RTBv3X6HsfflWUc4/dkmFr3zR/qfKmB3197c95PZvHv59ZQcK4I5c/w+dOOJ9PCF8CH72udzpwwCYPGWA2w/dJovck/ym1FXc7FVJOnfbwTu9W9jhdux94zB3Zh830KqunRlyfwVDErtjumz/cyelIo5NprpXTr6q7m1kh6+ED6UkZZgK5m7POuILYPni9yTJMVGcePQXlROnUr4229xKt/aq5QdsfzHfrWz8TswH9xPTN5etkx9gKc3FPDiZ/uZO2UQSbFNW9rYGyTgC+FD5igTc6cMsqXyZaQlMHNsMiOTO5JXVEZmdiEfXDeVsIsXyf+Tdc/T5pLSF8ocfgfr1gEw8JdTHS7egaBRWTpKKTPwbyAROAhM0VqfcnPcQeAsUAlUeJpBdiZZOiJUOKT4AaUjrqH12dP8uDWLFTutKzPt6/II33GujRSd8RPKv8umam8OgMPvrTmUxagpS6exPfwngXVa697AuuqvPRmjtR5U12AvRLCzH6qxX3BljjLxQ9oIYvNzeH7pl7ZFWxLs/eNSbSTNu1/mweefs7LjZSzPOuLwewuEO7HGBvxbgMXVny8Gbq3hWCGEnZoCRN87bgZgdvQJx5WcwqvqMj9irKYFxeb5/8Z0rhzTzTfZ0jWdj2vOv6vGBvw4rfVxgOqPnTwcp4FPlVLblVLTa3pDpdR0pVSWUiqrqKiokc0TovmqKUC0vWY4REXR+ZMVzBjVE3OUSSZvm4Cni667u69pwxO573gWP5oiWRs/wCVdMxDKYtSalqmUygQ6u3nq9/U4zwit9TGlVCdgrVJqr9Z6o7sDtdYLgAVgHcOvxzmECCg11bsvsWgO33Qng5b+C0v5ORY/8hylWHPCQSppeounkgfGhaDLzq+4+a+/pUpDWFgLRp44xv5rJ/LkTwaRVr2SNpDUGvC11umenlNKFSqlumitjyulugAnPLzHseqPJ5RS7wNDAbcBX4hQUFudluVZR5jT/Wbe/HVnRrz4DKZjVfDcXLd3BLL3bcN5uuimp8Zx4MsdTHryPujRg/09+7P30Ekuu+xy+sx9BmKjSRrd/NMwnTV24dUKYBrwXPXHD50PUEpFAWFa67PVn18PPN3I8woR0Gqre28E9c6p15L/9UZuLNxP2PCeDgHdCPTllkrp+XtZZnYhIxb/L5Uowj77jNh2Hfk86wj9U+OYn11Ihh92q/KGxo7hPweMU0rtB8ZVf41SqqtSalX1MXHAF0qpb4GvgY+11p808rxCBLTaJviMnmdmdiHrqjrQ7vhhzK0d+2eXLhq62U8WNnfO8yN3lOZy095NVDzyKHTt6vD7qCkTp7nPszSqh6+1LgbGunn8GDCx+vN84PLGnEeIYFPX/Woz0hLYPfIKWmz7gNff+5IbJwyx9Sztx58DsbfZnDjccQ3pQtsZ90HfvrR+ynGqsrYyx819xzKppSNEM+M8Jj9q4nB4Hla+t4ny2M62QCKbnHuPfSA/s20b7Y4e5ewbS2kTFeXy+6jpZ95c6t57IgFfiGbGpZeYnAzAg/FVDGimgSTQ2QfyNZt2cQPw4Y8myjbk1WuOpLlfhKWWjhDNjMv4fkICtGjB6PAfHWqy248VN/ex40AystV5AEo7dnGYI0lPjQv4n7H08IVoZlx6ieHh0KsXfPed7SHnMsvNfew4kEQV/QAmE1MmXMG59gcB6+bywfAzloAvRDNlP3bc+sabaP33v/HDwiV82HsE6alxbM0vtlVq9DR2LDn6DXD0KMTHY45uSaQpgjmr9xJpCrf9zNNT4/zdwgaTgC9EM2Xfowyb8AuufvN9Ojz1B+b8/CUAW88+PTXO4aN95UbJ0b+krpuS/+LQEVrExwOOk7DLs46wPqeIYb0KA3LRFUjAF6LZcu61f3/TbfRf+Fd+N6gtk+0yRozNto0ev2HO6r3MHJssOfrVnIfB7OdD7Hcdm5p/kHbXDAcch9eaewZOXUjAF6KZch7LLxo6Ahb+lYQdW1nepautp2oEoPTUOIb1cqzvIkM5l2SkJTgMgxk/20vlj2FMn460LTkB3bq5vL65Z+DUhQR8IQLEmKk3cO7xdiS/s4Tvtn7Pxg4tGTdmIOarhzIjKgwqIkmyC0iBHpy8zRxlYvakVCDbYRw+PTWOjfuK6Bffjl/1b486fx6qh3SCjQR8IQKEuU0rdk76KZcte5Xf5n9vfdCuelVleAQX77mXVn95Dsxmt+/RkEncYJr4zcwudBiHLzl+kj2/fILrj5/kiqROdDh73HrgZZf5t6FNRAK+EAGk+79e5LVfPkF6bzOZ+0v4afvztD96iE++O0bhOyv42aJX4ZV/cXHEKFp84VqQtiGphYGSjujpwmT/eEZaAuWWCsotleQVlXL4zulM/PwDyqLbEr2/CsLCKP/zc7zeMilgC6TVRAK+EAHEHGVixpjeACR1i7E+eMUAho6x8HjnQSzddANPbHydsZs3UXLiFOZOHVwCHtRv4rGur/H3nYCnC5PD493DGPfP/+a7vELOhJ9nzI6NfHLDnQx9dxGnsMts+iQHlGrWF7iGkIAvRBAwR5mYO2UQi7u1I7vFGcbmbWPdx1+Sce9El+yU+gaxuk5W+vtOwNOFyfj6p52q4Kqr6Xf2LD2j22GKimRXxr0M/cffMUeZbNlOwZzZJAFfiADjqSdtjjLx2LgUzrROhwX/xfhWZYBjdsriLQeINEU0SS+8pjsBX/T+nS9M9uecMToJFi6EoiJ+3LiFpVWdyEhLYKBdW0Kh+qjU0hEiwNS0+TlAVc9eAIQfyAMu9f6Njbhr28O1oWra09Vo8+ItB3xWA8j553Qu/xBaKV4rbcuc1Xv59Vvf8PzafbZzm6NMpKfG8fiyneQVlXq9Pc2B9PCFCCAlZRbKLRXMHNubjLQEl55zSZmFxz89yNzWbflh6y5SubSwCBQ3D+pKpCnc4x6u0DTDMcb5yi2VDufx9nmdv1f7cx/6di8dojpQFRHBmJRY1ucUsTmvmEhTuO3cz67Mrl68ls2ie4c2uj3NjQR8IQKIsUho1oS+DuPOcCmArs8p4lTnBHqXFjq8BnAIbsbFIj01zuEi0hSM3n9JmcXhglPfSeS67AXs7nsFSLpwmlPdujFteE8A24XB/txGnr71Y/CRgC9EAPEUKJ0/djswgBYb1sOePWSkJVFuqcA5uNlP5q7PKbJdROqqIePyzuPs9V296umOwPni5fy9ArQ4foxOqX2huq2PjUtxef+k2Oig7NkbJOALEUBqC5i2r0/9FD54D668EvOXX/LYuEEu71VTSQZDTUHdH1k5nu4I7NsybXhP27i9ffs7HDnC95cNIT4I8+vrSgK+EEGoZNxEPn5rPXc9MoWw4cOhbVvrzlkffQQdOgCOFwtP1R/tA6lRMdJdDZ/5G/K8kt1S212DpzsC56qWRpvBWkSuRdlZfnH2LB8VhxFT/f7Bsnq4PiTgCxHAPAXI5VlHmLPtFK2eXUBG1sdgsaBfe4382+6iw0fvY45uWaf39BRIjWwc+2qdxuONUd+7BmOS9pylitamcFtbjY+nyi3WGvZtLgIwdNTlXBkkm5k0hAR8IQKYp8BllBAoQFEy5XnMUSY2V7Vl5Kt/Z8Gvn2HyS0/ZsnqcLxjO72kEe6PgmKeFTd6Y8K3Pe5WUWXh82U6HktC7Ck47LC4zJrEnnzxBdyD9+sFgd3cSjIuraiIBX4gA5ilwmaNMDrs1zRidRFbG/YRlZnLnW3/n3z/9KfeNH+D2guH8nsYxy7KOsODnaS5DIN4sG1yf9zKC+YikGPp1bUf28R9dSh8b38N1Kz63bhWZkuL1NgcSCfhCBLCaApdz4P75qCQ2/WIG0X98iKicbEpGpbi9YNi/p5H3nxgTSV5RGc+ubPr8dPu7DsDjWLvzyljn19m+7tOO1q8vhptugtjYJm17cycBX4gg5S6D55ZpE+GP8N2qjZwZlMaM0UkuefkOi7iqh0ymj+rJ/hOlHvPTvVk6wd2kqzXVEkAxbXgi5ihTrRlLK9d8Q8wfZmP+fp31gRkzGtWuYCABX4hQ0qMHVe3acWer03Rx6gk7739rDJmMSYnlgWuTawzkdZkEtV8FawRtd+yzf1bsPMrMsdbqoEbbnBdUcf48vPgiLF0KBQW2h+8+dQpdWcX5Bx6k1ZDBcMMNdfgBBTcJ+EKEEqUIGziQficPglMvfkRSjMNq2/TUOLbmFzN7Umqtvfa6TILar4IFbSviBjhk2kwbnmjL/jFWFaenxpF1sIR+8e1cz3H//fDmmzBiBNxyi3WsHlDt26Puu49WycnWi9rG/JBLw3QmAV+IEHM+tT9hSxbz3gtLqWjZihNfH+aGDq0p2JJHgkokPOo072T/wPmLVZz4+jA7Kn8gaXACZ85dZG32D4xL7Uy71i0c3tMMzGgL7Nvt8bxTW1wkqocFUJC7h7e/PkzM/kQAMjcftB2XcCiRyYMTmNriIjHJlYxrUczaD3dzestB+oxIxLzvvPXAY8fgk0+swf6pp+BPf3J73pIyC4+8/Q1f5BZTbqnksXF9GvqjC3hKa+3vNniUlpams7Ky/N0MIYLKJ3MWcP3vHiCM5vt/v85atMAy4UYWPzKH24dZq4QaKaSZ2YUu6wdmjk12W1IhmCiltmut09w9Jz18IULM0Efu4V9J/ei89zuuS+lIm1aOvfWz5y+yOfckI5Idn7N/fHPuSd795igD4tty74ieLu/h/JoLFVWs3HW81uNraoP9YwCrC85R2GcgFa1aM2/tfqoirMfNWb2XjfuslTDLLRVMG97TVl9n2vDEBv7UgoMEfCFCjDnKxIwpIygpG8JbbjJr2gBDyywuz7UBxlcf07uolPwlWawtKqNl5762SpjOmTpvbchjzo69jEiKQcUr/p570u3x4Jh+abxuVvWx9u8105zMroIzrC8qgqICtztUFZdZ2JxXDCjbxjCikQFfKZUB/BG4DBiqtXY7/qKUGg/MA8KBhVrr5xpzXiFE7WrLiqkps6a2rJvM7ELyisoYkxLrskDL/jX2u23NHJvMqN4dXY4vt1RYA3j1illjda+x2XhJdbGz9NQ4Nu4rYmt+CV8dKGFEUgxpiWaX7824mMTYragVVo3t4X8P3AbM93SAUioc+AcwDigAtimlVmitsxt5biFEDWqqDQ81Z9Z4ei6vqJRnV2bz6+t623rVRrD1tIhr7pRBLvn98zfk2Uo1lFsqbemfxmvdrRTOzC6s7rVDUmwUT9/an6RY90XfQnUlbW0aFfC11nsAlFI1HTYUyNVa51cfuxS4BZCAL0QTMnrJ7mrDg/ug6LIPrNNz05dkkVdk3SvXecVtrXvKVrPv2UeaIhx24bLvqTtX4zRq3W8/dJovck+SmV1IhzRTSFa9bChfjOHHA/YbaBYAV3k6WCk1HZgO0L1796ZtmRBBrCFj17UN8+QVlZEYE0nvuDa2oZb6vpen7Q7t2V8s7N/nsXEpHp+THn3tag34SqlMoLObp36vtf6wDudw1/33mA+mtV4ALABrWmYd3l8I4SV1GeYxVuTG1DJsUlNhN3fbHcKleQejF2+UVLBfEGZ/JxGqVS8bqtaAr7VOb+Q5CgD730Y34Fgj31MI4WV13XzEXaD29D7uLgg1PW8/7zAmJRZQzFu33+P2izJWXz++GNLZBvRWSvUEjgJTgTt9cF4hhAc11cHfml/M3CmDPA7X1BRknWvUuzuupvPYzztcypnXDtk6ouHCGvNipdRPlFIFwNXAx0qpNdWPd1VKrQLQWlcADwNrgD3AMq215/XXQogmZwRdY+9XsAbbMSmxtpry9WFk3izectAl48ZZTecx5h0eG9fHVhEz0hTBvHX7690m4UpKKwgRgjwN3xiP25cmqKlX7Vxpc+bYZFtRNE87atV0/vq0VbhXU2kFCfhCCBfGPrWzJvStcYzcOM450Nf3fYT3SC0dIUS91DX7xXnXKYP95uL2GTY1kZ5805OAL4RwUdfsF0/H2WfbeMqwcfcayalvWhLwhRAN5qlXXtsqX3ckp77pScAXQjSYp165u1W+dc3zF01HAr4Qol7sA3ddeuWe9swVvteoPHwhROixz+E3euVGj72kzMLza3N4fu0+SsosDseDdqlbL3xLevhCiHqpqVdvP1m7q+A0c6cM8pjJI3xPAr4Qol5qGms3Jmu3HzptW0k7Y3SSDOE0ExLwhRBeY0zWOm9fKJoHGcMXQjSKUUfHGLMHXMb2RfMgAV8I0SjuCrGJ5kmGdIQQjdKYBVNSTsG3pIcvhGiUxgzfNPTuwN0wkqid9PCFEH7T0LsDqbvTMBLwhRB+09ByClJ3p2Ek4AshAo7U3WkYGcMXQogQIQFfCCFChAR8IYQIERLwhRAiREjAF0KIECEBXwghQoQEfCGECBFKa+3vNniklCoCDjXw5R2Bk15sjq8Fevsh8L+HQG8/BP73EOjtB99/Dz201rHunmjWAb8xlFJZWus0f7ejoQK9/RD430Ogtx8C/3sI9PZD8/oeZEhHCCFChAR8IYQIEcEc8Bf4uwGNFOjth8D/HgK9/RD430Ogtx+a0fcQtGP4QgghHAVzD18IIYQdCfhCCBEigi7gK6XGK6VylFK5Sqkn/d2e+lJKvaqUOqGU+t7fbWkIpVSCUmq9UmqPUmq3Umqmv9tUX0qpVkqpr5VS31Z/D3/yd5saQikVrpTaoZRa6e+2NIRS6qBS6jul1E6lVJa/21NfSqn2Sql3lFJ7q/8/XO33NgXTGL5SKhzYB4wDCoBtwB1a62y/NqwelFLXAKXAEq11f3+3p76UUl2ALlrrb5RSbYDtwK0B9jtQQJTWulQp1QL4Apiptd7q56bVi1LqN0Aa0FZrPcnf7akvpdRBIE1rHZALr5RSi4FNWuuFSikTEKm1Pu3PNgVbD38okKu1ztdaW4ClwC1+blO9aK03AiX+bkdDaa2Pa62/qf78LLAHiPdvq+pHW5VWf9mi+l9A9YyUUt2AG4GF/m5LKFJKtQWuAV4B0Fpb/B3sIfgCfjxwxO7rAgIs2AQTpVQicAXwlX9bUn/VwyE7gRPAWq11oH0PLwC/Bar83ZBG0MCnSqntSqnp/m5MPfUCioBF1cNqC5VSUf5uVLAFfOXmsYDqmQULpVQ08C7wqNb6R3+3p7601pVa60FAN2CoUipghteUUpOAE1rr7f5uSyON0FpfCUwAHqoe7gwUEcCVwD+11lcAZYDf5xSDLeAXAPbb2HcDjvmpLSGretz7XeBNrfV7/m5PY1Tfhn8OjPdzU+pjBHBz9Rj4UuA6pdQb/m1S/Wmtj1V/PAG8j3XINlAUAAV2d4bvYL0A+FWwBfxtQG+lVM/qSZKpwAo/tymkVE94vgLs0Vr/3d/taQilVKxSqn31562BdGCvf1tVd1rrWVrrblrrRKz/Bz7TWv/Mz82qF6VUVPWkP9VDIdcDAZO5prX+ATiilEqpfmgs4PfEhQh/N8CbtNYVSqmHgTVAOPCq1nq3n5tVL0qpt4FrgY5KqQLgD1rrV/zbqnoZAdwNfFc9Bg7wO631Kj+2qb66AIurs77CgGVa64BMbQxgccD71v4DEcBbWutP/Nukevs18GZ15zMfuNfP7QmutEwhhBCeBduQjhBCCA8k4AshRIiQgC+EECFCAr4QQoQICfhCCBEiJOALIUSIkIAvhBAh4v8BAwL6j8Y2xBgAAAAASUVORK5CYII=\n",
Olivier Michel's avatar
Olivier Michel committed
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# random forest estimator : \n",
    "from sklearn.ensemble import RandomForestRegressor\n",
    "clf=RandomForestRegressor(n_estimators=100, \\\n",
    "                           max_depth=5,\\\n",
    "                           random_state=None, \\\n",
    "                           criterion='mse')\n",
    "clf=clf.fit(X,y.ravel())\n",
    "\n",
    "Ntest=300\n",
    "XX = np.linspace(X.min(),X.max(),Ntest)\n",
    "y_ref=np.sin(XX)\n",
    "XX = XX.reshape(len(XX),1)\n",
    "yp = clf.predict(XX)\n",
    "\n",
    "plt.scatter(X,y,s=1)\n",
    "plt.plot(XX,yp, color='red')\n",
    "\n",
    "\n",
    "error=np.square(y_ref-yp).sum()/Ntest\n",
    "print('MSE = {}'.format(error))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
137
    "## Exercize 11\n",
Olivier Michel's avatar
Olivier Michel committed
138
139
140
    "\n",
    "- Change the parameter max_depth in the code above. \n",
    "- Compare the behaviour of the random forest regressor with the behaviour of the tree regressor of notebook N2_a_Regression_tree, when max_depth is changed.\n",
141
    "- Explain your findings. "
Olivier Michel's avatar
Olivier Michel committed
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In **extremely randomized trees**, randomness goes one step further in the way splits are computed. \n",
    "\n",
    "As in random forests, a random subset of candidate features is used (*again, here we have a single feature, so this does not apply*) but instead of looking for the most discriminative thresholds, **thresholds are drawn at random** for each candidate feature and the best of these randomly-generated thresholds is picked as the splitting rule. \n",
    "\n",
    "This usually allows to reduce the variance of the model a bit more, at the expense of a slightly greater increase in bias."
   ]
  },
  {
   "cell_type": "code",
157
   "execution_count": 4,
Olivier Michel's avatar
Olivier Michel committed
158
159
160
161
162
163
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
164
      "MSE = 0.000706083859664731\n"
Olivier Michel's avatar
Olivier Michel committed
165
166
167
168
     ]
    },
    {
     "data": {
169
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deVzU1frA8c8BHBBQEUXcUBQQxSVLUnPJVCwt1DbMFrNuXVuu5a1+t7JsX+x2s67duqm3NC3TtG7mTc2V3E1xyRRFATVxQQQ3QBmW8/sDZhyGAUQGhpl53q+XL3Dmy3zPsDzf833Oc85RWmuEEEK4Pg9HN0AIIUTtkIAvhBBuQgK+EEK4CQn4QgjhJiTgCyGEm/BydAMq0rRpUx0aGuroZgghhNPYvn37aa11kK3n6nTADw0NJSEhwdHNEEIIp6GUOlLec5LSEUIINyEBXwgh3IQEfCGEcBMS8IUQwk1IwBdCCDchAV8IIdyEBHwhhHATEvBFGVk5RqavTSErx+jopggh7EgCvihjYcJRJi/bz8KEo45uihDCjur0TFvhGHHRIaU+CiFcgwR8UUagn4HHBoQ5uhlCCDuTlI4QQrgJCfhCCOEmJOALIYSbkIAvhBBuQgK+qDap2xfCOUjAdzGOCL5Sty+Ec5CyTBdjCr5ArZVWSt2+EM5BAr6LqSz4ZuUYWZhwlLjoEAL9DHY5p9TtC+EcJOC7mMqCr+kOINdYgK/Bq8LAXxMXByGE40jAdzOmnn+usbBU6scU3GOiglmVmE5cdIhD0kNCiJojAd8FVKUnbroDKB7U1eQaC81fP3nZfrakZhKflAEUXxxyjQXmY6SXL4Rzk4DvxEyBOjPHyIx1qeQaC3lmSIcr+tpAPwO+Bi8mL9uPr8HT3POPiQqmd/t088XDdAzoSlNAQoi6TQJ+HVVZrz0rx8hzC3YRn5RB37AmJY/qKr12TFQwgPkcprRN2AB/87G2UkCmdE9MVDCLdx0DFGP7hAJIzl+IOkwCfh1VXv7cFKxzjYXEJ2UwMDKISbFR5rx7dV7blkA/A3HRIczedIgJgyNKPj/M1NUHWX/wNBuSTwPga/AEqFLOXwaFhahddgn4SqmZQCxwSmvdxcbzCpgK3ArkAg9prXfY49yuyjLFMn1tijkomoL1hMHhTBzW0fy4Za+8MjFRwWxJzTT38CsLvAsTjjJ1dTITh3Useb74TiKqRQN6tA0AVKmLTU1ceIQQ1WevHv6XwCfAnHKeHwZElPzrBXxW8lGUw5Rimb42pVRQtKyzv9pyylWJ6cQnZdC7fTphA/xLBd57ejXhv3uXcuC4N88PGm7u4ZvOCTC2T7ty8/lVCdwyYUuI2mWXgK+1XqeUCq3gkJHAHK21BrYopQKUUi201ifscX5XZqtSxlZQtQzylgHc9H/L4BwXHcKGY4tY9UcCcTkTiYsOoUgX0anNGfrOHEFiRiJoxfajH/Ht2CfKnNN0EahuOkYmbAlRu2orh98KsFxoJa3kMQn4FTAFcVBMXX3QXE1jK9BaB3koTt2YBnZNj9/77VPsyvyRU7npANT7UTH33lfYdm4yT86ZhScG7g79iFVpn7Lm1CT+vb4bL90ygLyCPOrXq2/zfFcatCVnL4Rj1dbiacrGYzZLSpRS45RSCUqphIyMjBpuVt1mCqoXjQUMjAwiJiq43IXK4qJDSuX0HxsQZk7dDIwMIi46hC83/86KP2bh7dGE1298l06N+zLv4Gu0/CCCWbtm0dIrluBLn7JtXwSPdPoEL08PPto9ihZTWtDyw5aknU8zL852fWiguU0VLdhm+ZwssiaEY9VWDz8NsEzUtgaO2zpQaz0DmAEQHR19ZXWGLsqyJLK4l57IpNgo83OmILrj/Ef0bdOTxwY8aPPrTReKs2oNqHy+vnM2N7aL5qUb/48/f/dP5iZ+TCNDKJ4XHqZ9k8bc3Lk59et50vjiyzQLWIVvPV9+y1zOcyueo15+N+YnfUSvJo+RfKIevfYGoini7z8fBC7P2p296RCm6/zU1QdZdyCDzi0bmSt9hBC1r7YC/mJgvFJqPsWDteckf2+bddrDFEB3p50lPimDjq128/AN3fl222H+s/dZUk8EcK7eXOb+/iWD2w0mOSuZ3Pxcrg3uz48702nadD//t+wzNu9vQn7Df9OuwTV0adYNgHqe9fgwdgK9g+9gUKcg4vefNp83K8eIr+Euco23M3X1QWI6tWXB3o+BBdTz9GFT1rvgDRsyj/Djge8Z3eNt4qKHAJeregD6hjWhX3hTNiSfZmNKpkWljxCittmrLHMecBPQVCmVBrwG1APQWk8DllJckplMcVnmw/Y4ryuyzo3nFeRxPOcg797ZibdW7uPlLXcwa18vMk51J8uwpOS7DPlF+XT6NIoLxvMAtG3QldNnmpLjFV98gDeQB17n/8zChKOl8u5KKZr61y8zMGu5BEORDsNLNWB58gpev2EqfTsWMXbRWObu+RIvDy9m7nuG8f0HEOh3jXmgefuRs2xIPs2EweFEtWhA4okL5hSQ5PKFqH32qtK5t5LnNfAXe5zL1VmWKl7Iu0DXz7py5NwR2gW0I+38cShqQNLZDWDYQOsGbUi78AeGokie6v46G08sZH9aQ27v2pm5B16isJ7m+d6TaKpieXX9E0Q2C+bxLveZa/tjooJ5+6dE86CurcFXy+UVxvW/j4DCEdx1bRRhQf5MuXkKTy17inl3zeOWr29hws8T8DcE0KfJszzerzdj+1CqcmhD8iHe/imRbq0DmLr6YLnnFELUDJlpWwuuZnEzgO8T/8eRc0cY2uZxTuRt5bbw3hhyxjDkmhxO5yVzV6e7+GLHV1zMDuOlmDuBO83nGX9uID5ePnRu1pmsHCMNDQvN5zfV9psWSjMN6pbHeizBVL8/suNIRkSOQCnF6M5j+GLXNABW7T9AQ8N3jB8YZX4vcdEh5vMZC4qYMDiizKQyIUTNki0Oa8GVVKfYqnT56eBP+Ho1IjFpGGPDviHg0l/5NbmIwtxrGXfts6zZ48HzfSYx9Y6xBPoZzBeLQD8DPVr2MAd707o3CxOOkpKRTa6xgAmDI5gUG8XEYR2ZMqp7hQH38uQrXSpQZ+UYKZ5EDRH1x9CgIJYRoS+Q57mP6fvuJfVMKr+n/47WmkA/A1NGdWdgZBAbUzLxNXiyKjFdqnaEqEXSw68FVzKj1Dp3f6ngEksOLOHWiGHEBHcutXZOeWvVW99JWC6wZupdmz5OHNaRsCD/K16SwXJ5BVOgtjz3n/tGE+D9T+KiQ9h0rB9jfhhDh391oFAXMiN2BsuSl9GzVU8m3zWeH3eWXvenNnfnEsKdScCvBVcyo9TyolBQVMDt82/ndO5pHr3uIW4JDyupmvG0SK8UlCpxtA7uU0Z1Z2HC0VILrPVun15q+ePKWAZcWxcty5SMpdgOsax7aB2T4iex+ehmxi8bj7HQyA/7f8BYaOTVAa+aj7W1MJz1ukHWxwkhro4E/DrC8qLw0qo3WJ6ynA+HfMot4beUeX762hSrxcwwB/ewID/ikzJK6uBhwuAIxvYJLbXAWlV69ZYB1zLoWq/zA1htnRjJrOELuffbp1hxdAa3hN2CwdPAR1v+ScOCETzYO6pMr936fOUtICeEuDoS8OuYo+eO8v6md/AtGMDupGiyrim705T1apdQepLVqsR0co2FTF19sFp177Z69ZX1+k3r5ucaC9iddo49B6/Dx8+fif0mUr9efXp93ou31nxBfa+/lum1l7dukPUCckKIqyMBv5bYyq+b/g+Xyxfn/j6XQp3PNQ3HEZ+UUaZmHsqudgml7wDCBviXSQFVpW0mtlJRlfX6TXX7phr8myOvZcqoTAL9DGitad2gDU29D9hsl/UuXJYVPpYfhRBXRwJ+LTEFylxjAYA5IJpMXrYfrTVfJX5FWMMeHE9vVG655JUEQOtgXdEAaFVy5ZWd2xS0NySfZmBkUKkKIKUUt0YMZd6eefx2aiMBPgEE1W/Hkt8yzXcmlrtwlfdehBBXRwJ+LTGlYS7mFzFjXSpAmYDeLOggiWsT+XDIp/jmdyw3Z301AbCioF6VHnRVB6CBUvn3oeFDmbFjBoPmDAJA4UHTvJfZkhpbagKYqUxV8vZC2I8E/FpiSsN0a92ICYPDMe0Da0p1tG11kOdXPk/bRm15ouef8PHysWvQqyio26sHbXkXYTnAbHmhGRI2hJGRI/EpvIbVe41k+35Kx9CD5ioiyxSX5O2FsC8J+LWkvJ2qsnKMvLvqe6bsug9P5cm3d3+Lj5cPYN+gVxtpEVvttb7Q+Bv8WTR6UfHFofVRvkr+lcNnd9PYt3T7JG8vhP3JTNtaYjkL1tLChKNMT5iDt4cvac+mMbDtcPMsVss17quiovXpa5Kt9pb3vk2PN/DoyNEL+5n760GbzwMOeS9CuCIJ+DUgK8fIRyuT+GjlgVKByhSI95w8RMycGDb+sZHQlsco8tnM0PARzNt8jqfn7TQvN1BesLQ+l3VArM2NRizPfyXttXb/dYNAFdG+VTrTEqZx6MyhUs9f7bIUQoiyJKVTAyzXgzeVF247to331vyXX/d2pE9aMqsPrWb1odUAGDwNRPjdYf6ayhYzsz5XZWmUmlTdtNPg9n3xUB7c98MdnM87T1RQFFsf3YqfwQ+wPefA3m0Qwl1IwK8BpglEoIiJCubhbz/gy/1/A8Dg58v6463xVPXwye/L3V1u5eVB9/PTrguM619EfYOneTD3Ss9l+RFqt4zxai8ulgO834/6nn9u+Sfdm3fn418/5tX4t4nwfoRR17exOefAXm0Qwt2o4qXq66bo6GidkJDg6GZUqLIFvt5fsYWJm2Jo16gj39/7BTd83p+LhRfoHTyKUWFvYrkN4MRhHd2mh2qq3rF+z3d8ewcrk9dyyehF60Yt+M+If5N6rLmUZwpxhZRS27XW0baekxx+NVWWY86tt4YilcPd7d8ipEEn/tr7OQD+fP1IfA1eJRuBaCYO61hq2WFXYp1jz8ox2lz8bfraFMZ2e5ycgjN4eeVwLu8Uo/87kiHdlAR7IexAAn412apMsQxwaw7/TIh/J+ZvLmL2pkO08LibKUM+4daIEeagN7ZPOx4bEOay68NbXxRNYxy+Bs9Si79NXrafk+lhvND3BR7v8hm+59/gUn4+jyx+BFt3ojJYK0TVSA6/mipabyYn/ywbj27kmV4vEOHTkVxjIVNWHGbisKH8uDO9zIqXrpqLtn5f1gOxlj3+IZ2bsyrxz8T0C6ZTYDoXvN7mb6ueZs6uhVw636PUxCzTAnEgg7VCXAkJ+DXAvGa911qKdBE3h93KzeWsaW+5MqSrrhlj/b6sB2Ir2lyloOgJZu76jJdWv4LX6SnmVTjjkzLoG9akVFpICFExCfhVdCW7MAX6GXi0fyg3TH8GD92IlLRgCC8b+GytDOkOrHv8tu5sTJ97eXjxYr8XGbtoLE9efwLoYF73f2NKJjd2CJL8vhBXSKp0qqi86hJLPx34ieHzhuPj5cO1TW/jpwe+sRmUZAu/K5NfmE/YxxEUFHjzRKfv8Pf2ZkT3lqxKTJfvnRBWpErHjiobpAX4Yd8PAFwquMRz/e+t8E6gqjNT3Ynp+3rhkubWkBc4kXuAD7a8Sf16HoQF+cv3TogqkoBfRbaCtHUVSuLpRHy8fLir010MDR/qqKY6Pcvv6ztD/8T1QXdwvt53fHPgzVIlntYln1K5I4RtksO3A8scdH5hPrtO7uLJ6CeZcssUsnKMfL1Z1nWvClOqy3IzlEA/A78+8T0DPx/D2uNf8crP/fjkzj8ze9Mhpq5OJtdYwNg+7cwbuYNU7ghhTXr4dmDZ69+bsZdLBZe4vtX1QOUTs6RHWpbpe7YqMb3U3dSZ3HyGhjxHU5+2/HvPYwyfN5y8woslX6XMG7lXZS0iIdyJ9PCvQkWDrZuPbgYgumW0zRml1mThr7LKm4+wMOEo0345zjMDvmNz+rcsOfghyz2W063NEO6OnouxoB5bUjOZFBsld1NC2CAB/ypYBulxN7bnXN45AnwCAFiUtIjwwHDCGocxY11qmclV1lx1slV1lDcf4fL8hUJ+338TzTwC6dR+H+uOz+X5NU9yU9O3K11oTQh3JgH/KsRFh5CZncf6g6dJvPA1n+2czOJ7ltMrpBtrDq3h2d7PopS6qs3G3VllZaqm71Vx+ksDEYztE8praxvx+c7Pef/JfwOX1yQybYwu4ydCFJOAfxUC/QwcPJXN+uQMTh6bTj553PPd3TzX5zkKigoY3G64+TgJ5leuvPSW9YUg0M/AM0Mizc/f0ekOPtn2CdvT1/LYgNvNcyW2pGbKAK4QFiTgX6Wbr8ti6fEpGAtO0KnBPey/sIDX1k7EUBRBalpzCHd0C52HraocS5VdCG6/thcBPgGMXTSW2A6xfHzzTHKNBVw0FtGtdYCky4QoYZeAr5QaCkwFPIHPtdbvWT1/E/AjYNq/7r9a6zftcW5H2HZsG+OWxpFTkIOfV2OWPfwp93ybx6/pixjY/K+Mur5NhV8vM2xLq2zg2jo1Zvr+WS6e9uqNr/Ldvu/45vdvuLHNjcBNzFifyoTB4fI9FqJEtcsylVKewKfAMCAKuFcpFWXj0PVa6+4l/5w22G8+uplBcwbR1LcpfzzzBxnPH6Nt4yYsvGcm/9d9Pt88+CiBfoYKyy1rc89ZZ1DZZu3Wk90uXyAu7yNA9q3c2foLbmwzkOdXPc8FY2bJV6vaeRNCOAF79PB7Asla61QApdR8YCSQaIfXrhOycozM3nQIUKw69R7+Bn9+Gh3Pkp15xEV7ctFoZOnuLCbG3GEjKJXttUplTmlVHeuw/P4F+hmYvjbFvB/wwwMmsjntNvbkfsqLQ9+o9G5LCHdij4DfCrDsqqYBvWwcd4NS6jfgOPB/Wuu9tl5MKTUOGAfQpk3d+GM1Ld+ryee47yoe6HY/G/YXmgM6wORl+0v2sQVQjOjeErAd1GUwt3qsv3+WewiP7RNKjufz/GPzO0QP7kmg3wRA0mhCgH0Cvq17ZuslOHcAbbXW2UqpW4FFQIStF9NazwBmQPFqmXZoX7WZAsqSpJX8kZmLb2G0zV66ZU7Z3ZY8diTLqp2sHCOn0mLxKfyZ1355mcevf5iG3g3Nd1y5xgJ8DV6lAr9cDIS7sEfATwMsu7GtKe7Fm2mtz1t8vlQp9W+lVFOt9Wk7nN/uyisD3HL2Lbyy6vHioLgyvUxTffhFYwGJJy6YK05E7VqYcJRfDmQyOPQvLEn/E//e+gWNi0aYfx65xsIyqTaZ7SzchT0C/jYgQinVDjgGjAbuszxAKdUcSNdaa6VUT4oHizPLvFIdYSsArEhZwYLEubzY90VCGje2+XWBfgaa+HuzIfkQqxJltmdtKlvaOYTb5s9gyqaP8D3TDrh8Ubbcdaz4WBlTEe6h2gFfa12glBoPLKe4LHOm1nqvUurxkuenAXcDTyilCoCLwGhdh3desRUA3ln/Du0C2vHaTa+VCi7WMzkleDiGrYv0Kze+wm3f3MbobruIiYph+triVUute/EypiLchV3q8LXWS4GlVo9Ns/j8E+ATe5yrNlgHgJSsFNYdWce7g97Fx8uH2RvLn8kpwcMxbF1oe7UYTESj64k/PoPle0bz/vIUtqRmMmVUd8nVC7ckyyNXwFRLP23bTDyUB2OuGVNqBcxJsVHm+nFZ5tixbG1M8932NM6eGkZ6zgm2nlxOr3aBxCdllJTYCuF+JOBXYGHCUd5dlsjMnV8ypP0QWjdsbS7R9DV4ltpmTyZT1T1x0SG8cfP9BHq34qukV9l1YQoajUzGEu5K1tKpQMtm6TRoNp8/LhynteF5snKM5eboJXdf9wT6GXjipg74B3zEW7/8gwPnvmdM9x6M7XOzo5smhEOoOjx2SnR0tE5ISKjRc1RUg912Slf+yN6DwaMhwTmz6B/WkujQQMb2CZUcsJPRWjNoziB2nNjJy9ct49G+PeRnKFySUmq71jra1nNun9KZvekwk5ftZ/amw6Ue33liJ39k7+GWkHEsHrWawZGt2ZiSydTVByVt44SUUsyInUGu8SJvbPib5PGFW5KUTsmk4O1HssjKMZp7fdO3T8fb05t5975H4/qNub715fV0JG3jnCKaRHBjy4dYk/YfsvPPOLo5QtQ6t+/hj+3TjoGRQWxIzjT33FPPpDJz50x6NrsDXeQHXJ6+/8yQDpIKcGIv3vQgKE3zZgcd3RQhap3bB/xAPwNTRnU3L7M7fW0KL6ychMaD1NRhcuvv5KzLZQe1701g/SbMTFhUpoQ2K8fIRyuT+GjlASmvFS7J7QP+zhM7+eNCIo8NCGPl3pO8/vNivt83nxtbPIgXTTiTk8/Ds7aSkpHt6KaKq2BdLuvp4UlYgz5sPbmG+VtTSx07e9Mhpq5OlnEa4bLcOoefmZ3HzXNG4umZz5Y/7ebj3x7lpM9a6ns25B+3vMK2CCPrD55mQ/JpIJFZD/d0dJNFFdkql32u3yOM/u//8GqwFehocXRxfX6/8KYyTiNckluWZZpKMfdl7mTq73EABHq3IivvGF0aPEBmRm9eGzqUxwaEkZKRzds/JTIpNoqwIFkMzRlZl94W6SKiPo3C3+DP1j9vxUN5lDrO1hpJQjgLKcu0YrrN/y1zKZ6qHre0vxU/by/uCX+V2Xd9zC0drjevuhgW5M+sh3tKsHdi1mkdD+XBxH4T2X5iO59t+4wDmQf4ZOsn5uUZViWmy6xp4ZLcMqVjul3/4LfNRAb05ps7fzDvQ/vcgl3EJ2XQu70sb+wqbKV1YsNH0zlwJn9b+TfabG1DUmYSg9oNIiooSmZNC5fllj38QD8D998QTMqZJI6dCjH35BYmHCU+KYOBkUHyx+5CyltY7dyxP6O0P0mZScWPJX5X7vFCuAK3DPgAaw9tRaOJ6zrAvNqlaRVMWT7X9cVFhxAT2YFGOa/Tu/HfiG5xAx9s/Ji7vr2H/MJ8UjKypTpLuBy3CviWNdlfblsFQETjbgA8t2CXeRVMCfauL9DPwKTYKDo1jeLE8QHkZA0g23iB/+5fwNKDS3n7p0TikzIYNyeBlIxsWfpauASXzOGXtyCa5a5Iul4qjb2b82jf6ySV42aycoqXydh+5CwpGTmEBfmRktGPMR2G83PmnczaNYspsV/zR1YCKRk55uAPsuetcG4uGfDL25TaFMyHdw/i5emrad+wZ6nHpQzPPZj2NAAYGBnEpNgocxlm8KYH+XDzhwzv8C0LHx9jLtPs1voYucbCUustCeFsXDLgl1dlYRqMm7lzJpkXT+N5ti8LE47y2IAw6bm5kbjoEHKNBYAyL3Vtqsh6/Lrn+HHvRh7936N0CerJYwO6AuBr8GLysv34Gjzld0U4LZcL+Fprpu+YQnTraAL9bP9hztg+g05NO/N051GSwnFDpoXwbFm5J5uck+PAZzPvrJnJ4rEfAbLBjXANLjdoq5Ri8obJLDm4xObz2cZsEo4ncFenO3j8pnC5PXdz1ourxUWH8Nzg3rRv1IPk7JXm46RUU7gClwv4AI3rN+bMJdvrnf+a9iuFupC+bfrWcqtEXWQ9C9fU+3+270PsO72XDX9scHALhbAflwz4gfUDOXOxdMA39eRWpaxDodh/pLmU2QniokNKLY1t+p14+NqHadmgJc+vfB5b601Z3xkI4QxcMuA39inbw1+YcJSJK99lWsJ0WvpF8s+Vx2StFFHu+jm+9Xx5a+BbbE7bzOc7Pi/zddZ3BkI4A5cbtIXilM7+0/tLPdatXTZn682itU8bnun1FOSES5mdMLM1KPtQ94eY+/tcnl3xLLEdYmnRoEWFxwtR17luD98ipVNQVMA/Nr+Bbz1fVj+wCd/8WwBV7kYXcrvufmwNynooD2bEzuBi/kXe3/g+cPl3A5BBXOF0XDLgB9YPNKd0tNbcNncEP+z/gQm9nmdN4sWSSVmaicM62uyhye26MAkLDGPMNWOYtn0aJ7NPyu+GcGqumdLxacylgktczL/ItK3zWJG6jID8P9G06J4rmlUrt+vCJCvHSDvDAxgL5/DO2vdpb3iMCYMj5HdD2FV5y8HYm0v28BvXbwzAmUtneO2X1zAUhdGw4HZAXVE9tdRcC5PZmw4xc+0lrm0ay4wd0/hw9TZAy++GsKvaunN0zYDvUxzwU8+kcqEgjaiAoTx2Yzhj+4Q6tmHCCRXvczuk9ZMU6HzO1pvLRWORjPEIuzEtzT6uf3tzIUlNsUtKRyk1FJgKeAKfa63fs3pelTx/K5ALPKS13mGPc9ti6uGvO7IOgPeGD+f6FmG1csskXMvYPqH4GjyJiw7hjHqcGTs+5dPfH8Hj7HhAVs8U1WdazG9gZBDxSRk1ul5TtQO+UsoT+BQYAqQB25RSi7XWiRaHDQMiSv71Aj4r+VgjAusHArD2yFoAugZ3Nd8ybUnNlA1OxBUzpfcAPrhlMoczjCz/4wtaNfiWmKgRQO3lX4VrMo0HxUQF07t9eo2OD9kjpdMTSNZap2qtjcB8YKTVMSOBObrYFiBAKdXC+oXsxZTSWXt4LU3qN6GFfwviokPMV1CpsBBXw9/gzzf3/IuODe7kWP5P/GfLCkCqukT1mDoVYUH+NT52aI+A3wqw/E1PK3msqscAoJQap5RKUEolZGRkXFWDTCmdvMI8ugZ3Raniwdopo7qXW4opxJUI9DPw09iPaVK/JbMPPMXp3NPm5Rnk90rUdfYI+MrGY9aLj1zJMcUPaj1Dax2ttY4OCgq6qgY18m5k/tynqKN5ECTQz0BcdPGm5TLgJipjOQHP8vOwps1ZPmYRJ7OP89VvX0lVl3Aa9hi0TQMsuzatgeNXcYzdeHp40rZRKOdzvUhMutm8yQmUvxuWENZMvyu5xgJ2p50jPimDXGMBvgYvYqIiCfHrwocbv2Bst79IsBdVZjn2o9UFso3ZtA1oW6PntEcPfxsQoZRqp5QyAKOBxVbHLAYeVMV6A+e01ifscO5y/V+3JTQ4/xHhQY2JiQo2Py633+JKmX5XQJn3PAbF5GX7efunRLLP3EBazl6mrl1T6utkaQ5hzdbvhOXYz/hl47n565trvB3V7uFrrQuUUuOB5RSXZc7UWu9VSgaDpFgAACAASURBVD1e8vw0YCnFJZnJFJdlPlzd81bmvl7hbD+STXxSBqsS081b2FlWXQhREdPvSlaO0VyaCeBr8CQmKph2Wx/irR1z2J89n+lrL8++fW7BLtn0XJRiK7Ng2moz11jI2sPrOZF9jPN552no3bDG2mGXOnyt9VKKg7rlY9MsPtfAX+xxrithulWaFBtV42VOwvVZdxJMn796Wx+OFDzEl7u+YOOOgUA/co2FxCdl0C+8qfzeCTNby7UE+hnwNXjx1rINnKh/DIDEjETq6Za8viSef951F2FB/nZth0vOtDVdTVclpstgmqhR7b3vpUgr6jX5uuSPubgWoUfbAPm9E2blDexHhRhp1nyL+f8/Jq5m4JdDWHryaV5bvM3u7XDJxdNk8TNRW0b36MH3+x9m57kZ7D39K2P79MLX4CW/e6JSK5M3EjvvVoxF5wFQ2pv3Nk8C7UUXnzd4Y8T1dj+nS/bwpfxS1JZViemcPnkz3h6+fL376zI9ORnAFdaycox8tDKJB797hsJCL9rUH8DIDqPRKg+AgIL7+FOPu+2ezgEXDfggsx9FzTMtevXM4K7EdhjO9/u+p6CooNQx8nsorJnWzsm4lEKo/w3s/MsKFt07j2d6P4NfPX/eGvRCjS306LIBX8ovRU0z/eH6Gjx54JrRZF7MJPDvgby97m2KdBEgv4eirLjoEMYNCKZQZXH/dX3Md4NTbp7CmReyGD8oqsbGf1w24MvsR1HTLIN5bIdY3hv8HgPbDeSV+FeY9/s8oOLfQ0n3uBfL7TEHdy0E4LqWnc3PK6Wo51mvRtvgsgFfiJpmGczPXywioPBuZsYuINgvmCUHl1Qa0CXd414sf977TxfX5HcK6lSrbXDJKh0hapvlxJohYUNYnrycb7cd4e8/HwAwFxFYLqEs1WTuJSYqmC2pmcREBfOf3/ZRz6Me7Ru3r9U2SA9fCDuIiQpmYGQQMVHBDGk/hIzcDMJbnzKnfGz15iXt6F5WJaabZ/4nZiQSHhiOl0ft9rkl4AthB5Z/zLeE3UIDQwNGfR+LT6MNaK3JNRbI5uduypTaC2pylPv7FXLndS1Z/8d6+oT0qfW2SMAXwg4sB3CD/YPZPm47XZt15aEfHyJ25mtMXZ0MaJtzQ2Tw1rVY/zxNd3d/WfYoS4+9xqHzv3P20lli2seUOj4lI7vGfw8khy+EHVhO9ouLDiGiSQT/jVtB1L96kpD5Jfd1GIlppU0ovaiaLNntGkxreOUaC5m6+qDFUtrB5OSf5bmNKaTnKhbsXQDA4HaDgcs//y2pmTW+6J4EfCHsxDJwx0WH8LeFu+HCcAq836Wg0XRuv+59fA1la/Jl8NY1mH7+4/q3Y2BkEBfzi5i6ujiQD40uXg1eo/l026d0b96dIL/iDZ5qc09bCfhC2IllFcbChKPEJ2UwrEMsnoGnmbP7C3y8PPli5BfmW3jTH7ZsgO4aTD9P04qp3Vo3YmBkEP9LXsD3x7/FQ3ng7enNxYKLvNTvJfPXWa7GalrGvaZIwBfCTkwDt5a9tOJAfgP+3t58lvAZrwx4heW/FZrvBABJ57gIW/snbDq6ni//+BAKNI0MwYzoeDPn884zqO0I80W/Ni/0EvCFsJPSQb70GvrP932ezxI+Y3rCdP52wxvmjS9GdG9Z6mstWW6BJ71/52H62e88lsQDP9xPm4bt6BM8hruv6cldnW8FYPraFIdc6CXgC2EnFe2m1rpha25o3Y8vdy7kbze8ga/Bi8nL9uNr8DR/jXWAl8Fc56W1ZuT8O7iQl8vT3Wbx5m3DgMs/Y9O2q7U9biMBX4ha0sLQn/W5b/HJuvWMv7E/UPoP3jrAy2Cu81qRvJ6j2fu4PfRNGhuK0zx14SIuAV8IO8vKMTJ70yFAMbZPqDkd89LAMSxIfouMoiUE+g0u8wdvHeBl/2XnYXl39uraZ5n/+48oXR+/wv5MXX0Q0PgavLg+NJC+YU3IzDGaLwK1SQK+EHZmWjYZMKdssnKMbDnowZiuf+KThI+4tmUX/rPjP4zpNoYnr38SKFvLL3l752HquecX5fHptk8BiAn5E2+MiGZVYjq5xuKB+oGRQWxMyWRjSiZNHHBBl4AvhJ3FRYeQaywAVKnSy8nL9vPETU8T2mA7jyx+BIDfTv5GbIdY2jRqU+o4kLy9M7DOyXdqcwbWwfy75nNPl3uA4lJLU+VOTFQw3Vofw/J3ozZJwBfCzgL9DDwzJNL8f9POWBMGR3Axv5DCUxO5NuRrxlw3iJfXvEyvz3ux4O4F9G/bX/L2TsZ0gS7SRWzMeoMf07IA6BbczebxjX1L/27UNllLR4gaZrkzVv16Hnjgz5gOH/DMDc/wy0O/4Kk8eWPtG4CsoOlsTGso5XmvYu7vc1mWvAxvT2+a+LS1uZ6Oo/c+kB6+EDXMutfua/Ayf96zVU8e6PYAUzZP4VDmKVbsuSD5+zqisnkQpudHdG/GNTNex9erEbkF5+jSrAs/7DhRJyuuJOALUcOsq22sc/MjIkfw941/5/WVX7J2V2ebx4jaV9F4SlaOkecW7CI+KYNt6f8jI/cUQXmvoPz+hZ+KLFNnX1cqriTgC+FgvVr1ol1AO+YkvUC/dg+QkzfJISV7ojTT4HuusbDUz8My2LdqtZlVxxbg59mSp/veg1EP4ssNxfsi1IUAb01y+ELUIltr33t6eLLk3l+4seV9bDj5NW+vneLwXK8o7pX7GryYuvpgqZ+HaWG88DZJbMp6hxPZ6XhfvIuUUzn8dWAfXh52ncNTN+WRHr4Qtchy7fMpo7oDMHvTIbYfOcvhlHtpH3yA7PyV3N3jXw5uqQDbS1fHRYeQm3+eKbufpLlvGIvu2si0tUeYFBtVZ1I35ZEevhC1KC46hIGRQcQnZbAw4ai5gmdD8mnCg/wZ2/1+Tl08zMfr1pjvAmRHLMexDOCmn0Ggn4G1p9/gRPZxOPM409YeYcqo7oQF1ezSxvYgPXwhalGgn4Epo7qbqz8Aco0FbD9ylg3Jp7klvxcKxdRNX9HSrwOPDQiTyVh1gOXPYEDnfH5M+pGX+r3B8T9uNF+8neFnU62Ar5QKBL4FQoHDwCit9Rkbxx0GLgCFQIHWOro65xXCmVnf9j8zJLJUCeCa9GGsOfwDqWfvJyUjWDZAdzDLiXNx0SH889d/ADA66n5a9b68FIYzLGdd3ZTOi8BqrXUEsLrk/+UZqLXuLsFeiGKWqRrLCVdf3vEFPp5+fLTjSV5bvN08aauuBhFXd3ltpOJN6GfvXIChKIJNB0pPlKsrk6sqUt2APxKYXfL5bOD2ar6eEG6jvADR3L85X97+FQUeR8nz/5yJw8rugyvs40rGR0yzaUHx5s9r+CN7NzeF3GYu17Q+ri7/rKob8IO11icASj42K+c4DaxQSm1XSo2r6AWVUuOUUglKqYSMjIxqNk+IuquiAHFn1DBe7v8y3+3/Gt+ATQT6GWTwtgaUd9G1/F438FEcL5pDz4gcGgetBe1BgI4pU67pDMtiVJrDV0qtAprbeOrlKpynr9b6uFKqGbBSKbVfa73O1oFa6xnADIDo6GhdhXMI4VQqKuHLyjHSQj1A71bxPLXsaU6eisSLgJK11WXw1l7KW/LAcpC2oP7PvLnuTaZumUYReVwbNIR3bx/AqsT0Ot2bt6XSHr7WOkZr3cXGvx+BdKVUC4CSj6fKeY3jJR9PAT8APe33FoRwPpX11hcmHOX95cnc0vIVso05vLnuVUDbvCOQnv/VK69XHhMVzMDIIK5r58nra18nxL8T5/NyaOodyrx7phIW5F/ne/O2VLcsczEwFniv5OOP1gcopfwAD631hZLPbwberOZ5hXBqlZVamoJ6TFQwPx+5nZ2nlxB3fTCtAwLMx5iqQnKNhdLzt7NViemsSUpne86LnM87z38f+JnfD/lzS5dWrNqbTlC0cy59Ud0c/nvAEKXUQWBIyf9RSrVUSi0tOSYY2KCU+g3YCizRWv9czfMK4dQqG+Az9TxXJaZz+Oi1GIty2Zi2vNQxly8atnv+4spZ3yXFRYcQ2eF//J4ZzwdDPqB/aA+eHBjJqsT0Citx6vrdVrV6+FrrTGCwjcePA7eWfJ4KXFOd8wjhaq50Cn5cdAhF+i7eSPiEl1b+nQFtbqN5Q3/zc6aPztjbrEus77iWp37PyqP/4S/X/4XxPcebj6tsmeO6PklOaV13x0Wjo6N1QkKCo5shRK2yNYHnTws+ZNa+5xjYaixrHv3SsQ10QZbf8zMXs+gyLYJuwV3Z+Mg6zl8suuIJVXVh8pVSant5851kaQUh6hhbvcQPbhtP0pnNbElfwIW8f9HAu4Ejm+hyLO+4Xvn5f1wqzCa68RN8sf5IlcZIZPE0IUSV2MrvB/oZ+PuwCVwsuMii/YvK5Irreu7Ymfg3+AOFByH+XUuNkcREBTv991h6+ELUMeX1EvuG9CU0IJRZu2aRe7ZPqWWW63ru2JnsOb2Dzs2iGNc/Cm/P+oAiLjrEJb7HEvCFqKMs88FQnOoZ3elR3ts8if4hi/BvtoPFyYH0TmhS7mBiXcgpOxOtNQnHE7g14lbzBiiTl+3H1+BJTFQwW1IzzdsXOiMJ+ELUUZY9SoDJy/bTP6I/HjqAN9f/HwAGn/r06/Aq3247ws2dW5S5QEiN/mW2Ln5aa345/As+Xj68suZ1OjaI41TOKaJbFI95Wl5ITTtd9W6fTtiAur/2vS0S8IWoo6x77bnGAi4ai3gy4FMiW+fSqmET7lxwJ8O+GcDJ8zncmTyNLclF5q+fvGw/EwaHS41+CevdxgL9DCxPWc6wucPMx6zWa/D0qMfwyOFA6fRaZSWZzkACvhB1lHUuv3h/1f1MGNyDepe86N2xGf4Gf45m7wMPSCp4jReGzi2zHZ+kcorFRYewJTWz1IYlq1NXA+BXMAhFPbK9lnN/1z/RplGbMl9f1ytwroQEfCGcRFx0SKndsbakZjIsfAQJxzfzfN/neWLJEzTut51AvwhAUjjWAv0MTIqNAhLNefhfjvxCz5b9COcdwpt7kU4Er9400bENrUFSlimEkzANIm5IPk1YkB/xSRnc2ORldj2+i7s7PkQrv468vHoShUWF5b7G1ZRvulLJ56rEdOKTMliVmM65S+fYcWIHedkd2ZiSScuGjZk2/F+0bNDS0c2sMRLwhXAiphr9GQ9GM3FYR+7rFU5D74Z8v/0YeVl3kHo2hUe+/bTCVTiruiuTM+zkBOVfmCwfj4sOYcLgcHKNhczc/j1FuoiTGeEMjAwyp8Jc6QJnTVI6QjgRyzyyZaVIXHQIm1KG882RL/lm/4cYczrxyb0DzBunmKpTrmbg8Uq/xtEloOXVyVs+/mCflhw4H8+Sbc3IazQVr6IWDA0fwJRR3QGYvjbFpSubpIcvhAsI9DPw0T09GBv1CoUex/nu6Di+3XYEuBzwnluwC6DK67hf6U5Ojr4TKG8FUtPjt13ThNu+uY1pe54gs8HDnDLu5vaIh/nwnutK7UnryquPSg9fCCdTXk860M/A5/c8wTVbvHh6+ThaBR8GIkpVp8zedAhfg1eN9MIruhOojd6/dRWN5TkfGxDG5zs+J/5wPM/fMIn1h/bRuUUzPrzlRRp4G8q031Urm6SHL4STqawnfUfHOHw8/flmzxygOBBOGdXdvBF3ZXu4Xq2K7gRMbZ696VCtrQFk/X1anryGhvWaEswDHD/0MLnpD/L5uhPmcwf6GYiJCua5BbtIyci2e3vqAunhC+FEsnKM5BoLmDC4uOdu3XPOyjHyyg8H8Mrrz4LEb+i2vhMD297G+78soF/zB7j92hB8DZ4V7uFaE3nry5PHCkudx97nzcoxMnvTIUAxontL87m11qxOjafgYkeUgoGRQcQnZbAxJRNfg6f53G//lEh8UgaQyKyHXW8nVgn4QjiRhQlHmbo6mYnDOhLoZ2D62pQyATQ+KYOREX/jkn9DXl7zMj6e73KpMIcVB9dTv95/eHJgJHA55RETFVzqIlITTL3/rBxjqQtOVQeRK0sNmb4/QKlAnnomlTN5Jxnd6THG9mkHYL4wWJ7bVKdf/NH1SMAXwomUFyhtfQzwHUDcwnyWJy+nX9ADbDj5NVvP/oPH9Qw8lEeppQbikzLMF5ErdTV5ees8e1Vnr5Z3R2B98bIM5BfyLrAyZSUAk2LizG19ZkhkmdcPC/J3yZ69iQR8IZxIZQHT+v8L4xZy7tI5GtdvzEurQ5i8YTJZl06xaPSiUhul926fXuXBVkcsF1zeHYFlW8b2aWfO209eN4XX1r5Mm0ZtCGsczvpEb1r4OecG5PYgAV8IF1Q6UDcG4J1B7+Bv8OflNS8Tfyiewe0H26zpt2QZSE0rRpqCv+UFY/raFLtUt1R211DeHUFcdAhb0xczff/rGAtnMWXFEc7lneLNra+RX5RHypmDDG3zBO/9nIRSqsx7cRdSpSOEEyuvysVWJc+Z3Hwa5I+koXcj5uyec0WvaVnbbv2apuC7KjHdbvX3Va3lz8ox8tHKJP65+jcWHHybnScTOJw/l55dNjMn+TE0hUT4jQTgpYGPlPte3IX08IVwYuWlVUwLreUaC8nKKU5hzN50iKmrD9OsyU18t/c73h30LvU9g8r0dK1f0xQgTQuO2ZrYZOvxq1GV18rKMfLcgl3EJ2Vw1msu2fWy8PdsyYdb3gOggaEBj3eezg+/NubVGx+jf2gP+ofav83ORAK+EE6svMBlvVtT8cVAAZCbNYxC35XcvfBubgx6lllbfie/cDTjB0WVec28gjyeXzKNlTvasCChMTMejC6TArHnssFVeS1TRVLPdj78L2MJresNIP/cbXRovYav4qbQvnF7cvM86BR41Ob3x9WWTbgSSmvt6DaUKzo6WickJDi6GUI4JVs1+rM3HeKisYgD51cyL+VvGAuLU0GNfQLpGtyFRfcsonH9xubXGPvDI8zZPZO2Xg/ChVEMjAyq8SoWW1s72sq1m47L4Hte+eUFVty/gdRjzUu9X3fM0yultmuto209Jz18IVyUrQqeZ4ZEMn1tCr+sD+O1gStp3fwwAT4BzNszj/l75vPs4i+YEvs0gX4Gvt+7hDm7Z+KhG3KiaCEjw+4utz7dnsHV1taOxaWWAIqxfUIJ9DMQ6Gdg3I3t6fzvL+ndujdDwvtCuO3XccfevC0S8IVwM6XXjLmRrBwjx050Ymm91SzYu4jrmw2nV2Q2L696G08dyPCW/2bl6UdI1W/QImCwzde8kuBqOQvWFLQral9MVDCLdx1jwuDiDV1MK1haTqiKPxzPvtP7+M/w/1T4PkUxCfhCuBnLnr/lwGdQkz6kFa7ks32j+Mu63QD0aDyeD+64jT2Zc7n929v5fMfnPN3r6TKveSXB1XIWLGjzIm6AOdVU3+DJ2D6hPDYgjOlrU8yzimOigkk4nEVEcwMxnf0AmLlzJo8sfoQAnwBGdR5V6ft0x/SONQn4QrgZy542QHxSBmFBfuzN7EWe94/kFeZxT/irnLmUyb6DA1iVmM5jA0bSPbgHf1//Kfd3fowm/t6lXvNKBkHjokM4c/E8XsqAaRE3k8sXgss9eMuLyBebdvLTkfcpOLWaGQf8SH46mfl75tOhSQdWPLCCht4NK3y/T8/bwYbkTHKNhTwzpEMVv2OuQwK+EG7Gsqc9YXC4uQe9KjGEkdc+xY87i+vqJwwO587wy73wzo3uYFf6JN5fs5gXBo+sUo85K8fIvK0pTN9/F4PbD+L9wf8utaZOrrHA3MM3PWa6iBTpIualjie73k5uaj2INYdXMj1hOr8e+5X7utxHg3otzBO/TO+v+P2km0tKNyRnlrSk7hap1AYJ+EK4GVONvnUu3TTbNi768vrwlsH8naGPsyj1A3ac/YKFCdfx9rLNbE45zYf3XEugn4EjZ4/QqmErvDwuhxVTKmVt2nf8tH8DF7wOMfu32bxx0xvmO4Kfk39mVca/6BDYgbYN2jJk7lfU9/TnUp4/7YMC6NqsIztPJvDZrbNQuf25VPAwz654FoBz50KZvemwOb8PxYO86w4Ur4SZayxgbJ92pd6vO5OyTCHcVEV57fKe+/uGv/Pi6hfp1qwHu09tp0HBSN4b9B5LTjzH0oNLiQufxLTbXzF/zfS1Kby9bCtp9e8DwLOoGUUepwms35iRkXfTzf9JXk+4CW9Pb07lpKPR9G7dm/2nDnEuLxulLlFEIbEdYrmtxUe893MSI64/z7/2FL9ey0vT+L9BN5UaD1iYcJTMHCMz1qUyYXCE26VwaqwsUykVB7wOdAJ6aq1tRmel1FBgKuAJfK61fq865xVCVK6yqpiKKmvKe+6pXk+RciaFlQe34lN4LRe8fmRjVgBLDy4F4Kek5SxMeMj8NXHRIcz9/SvSMuG6prHEhIwlsHEavx5fx8xd0/Eq+okCjzMMDprCnqwiHujVig9G3kvGhYvM3pTC4Qu/0S20kEd7PMCh07msP3ia9NOB+BReR77XXp4dOICxfdqVem+mZZibWKz3I4pVN6WzB7gTmF7eAUopT+BTYAiQBmxTSi3WWidW89xCiAqUtza8SUWVNeU9d+JsEfmnH+X7u/7Or4cy+HjPHXyzZzYBPgHc3P42fjrwP+7q0cp8fKCfgcZNfqfRhWCWj11IU38fsnKMBBQNJ7JxD97bPAk/rwAO/hHGsMgWvBRTvJl4UIP6NPCpz0/xAXQN6oiH8mBVYjobU4pz8Tc0fZ8P7+1E91Ytbb53d51JW5lqBXyt9T4ApVRFh/UEkrXWqSXHzgdGAhLwhahBlrl6W0HdVlC03gfW+rlxcxJIycgBYNbDPfFv/CIPLnqQuzvdTb82/ViQOJeTuck09e9CxoWLjF88maWpixl33Tia+vsAl+8enh50F/1a/E6/0K6Eencpkz6yXo3TtNb99iNn2ZB8ml+TL9ImQMotq6I2Bm1bAZZL0qUBvco7WCk1DhgH0KZNm5ptmRAuzDSztioqS/OkZOQQ2sSXiOAGZOUYGd1lNLvTd/PnHn82D9b+9ee/8smtnzD86wdJPreNroEDeWfwO+bXsdzu8GjqGEIjO1Z44bFs0zNDIst9Tnr0las04CulVgHNbTz1stb6xys4h63uf7kjxVrrGcAMKB60vYLXF0LYyZWkeXKNhUxdfZAmJXcI/7j5HwBorXnzpjf5YPMH9Pq8F+fzznNP+Kt8OvIlAnwu1+2Xt90hXB53MPXiTUsqWG6/aHlnIrNpq6bSgK+1jqnmOdIAy59Ga+B4NV9TCGFnV7r5iK1ADcWp3VcGvEJIgwge/t+99GrVh3n3vV4m5VtR2shy3GFgZBCgmLr6YLnbL0quvmpqI6WzDYhQSrUDjgGjgftq4bxCiHLYCu6We9xOGdW93Jx4RUE2K8fIL7va0cT4LMNbj7A5vlfReaznCBTTpdb1F1evWjteKaXuUEqlATcAS5RSy0seb6mUWgqgtS4AxgPLgX3AAq313uo1WwhRHbZ2fIqLDmFgZBDxSRlV3gnKtEvW7E2H+eXAaYaH38MT/W+weWxF5zGNOzwzpIN5RUxfgxdTVx90u92paoJMvBLCDZWXvjE9brk0QUW9atPxprz+hMHh5klQFa1JX5XFzGThs6qpaOKVBHwhRBnT16Ywedl+Jg4rW0Fj6zjrQF/V1xH2IxugCCGq5EqrX0qvrV+6B29a8tiywqYi0pOveRLwhRBlXGn1S3nHWVbblFdhY+trpKa+ZknAF0JctfJ65ZXN8rVFauprngR8IcRVK69XbmuW75XW+YuaIwFfCFElloH7Snrl1pU8ICkbR6lWHb4Qwv1Y1vCbeuWmHntWjpGPVibx0coDZOUYSx0PmonDOkrKxoGkhy+EqJKKevWWg7W7084yZVT3cit5RO2TgC+EqJKKcu2mwdrtR86aZ9I+NiBMUjh1hAR8IYTdmAZrLfP8ou6QHL4QolpM6+iYcvZAmdy+qBsk4AshqsXWQmyibpKUjhCiWqozYUqWU6hd0sMXQlRLddI3V3t3YCuNJConPXwhhMNc7d2BrLtzdSTgCyEc5mqXU5B1d66OBHwhhNORdXeujuTwhRDCTUjAF0IINyEBXwgh3IQEfCGEcBMS8IUQwk1IwBdCCDchAV8IIdyE0lo7ug3lUkplAEeu8subAqft2Jza5uztB+d/D87efnD+9+Ds7Yfafw9ttdZBtp6o0wG/OpRSCVrraEe342o5e/vB+d+Ds7cfnP89OHv7oW69B0npCCGEm5CAL4QQbsKVA/4MRzegmpy9/eD878HZ2w/O/x6cvf1Qh96Dy+bwhRBClObKPXwhhBAWJOALIYSbcLmAr5QaqpRKUkolK6VedHR7qkopNVMpdUoptcfRbbkaSqkQpVS8UmqfUmqvUmqCo9tUVUopH6XUVqXUbyXv4Q1Ht+lqKKU8lVI7lVI/ObotV0MpdVgp9btSapdSKsHR7akqpVSAUuo7pdT+kr+HGxzeJlfK4SulPIEDwBAgDdgG3Ku1TnRow6pAKXUjkA3M0Vp3cXR7qkop1QJoobXeoZRqAGwHbneyn4EC/LTW2UqpesAGYILWeouDm1YlSqlngWigodY61tHtqSql1GEgWmvtlBOvlFKzgfVa68+VUgbAV2t91pFtcrUefk8gWWudqrU2AvOBkQ5uU5VordcBWY5ux9XSWp/QWu8o+fwCsA9o5dhWVY0ull3y33ol/5yqZ6SUag3cBnzu6La4I6VUQ+BG4AsArbXR0cEeXC/gtwKOWvw/DScLNq5EKRUKXAv86tiWVF1JOmQXcApYqbV2tvfwT+B5oMjRDakGDaxQSm1XSo1zdGOqqD2QAcwqSat9rpTyc3SjXC3gKxuPOVXPzFUopfyB74G/aq3PO7o9VaW1LtRadwdaAz2VUk6TXlNKxQKntNbbHd2Wauqrtb4OGAb8pSTd6Sy8gOuAz7TW1wI5gMPHFF0t4KcBltvYSiV3wgAAAVFJREFUtwaOO6gtbqsk7/09MFdr/V9Ht6c6Sm7DfwGGOrgpVdEXGFGSA58PDFJKfe3YJlWd1vp4ycdTwA8Up2ydRRqQZnFn+B3FFwCHcrWAvw2IUEq1KxkkGQ0sdnCb3ErJgOcXwD6t9YeObs/VUEoFKaUCSj6vD8QA+x3bqiuntZ6otW6ttQ6l+G9gjdb6AQc3q0qUUn4lg/6UpEJuBpymck1rfRI4qpSKLHloMODwwgUvRzfAnrTWBUqp8cBywBOYqbXe6+BmVYlSah5wE9BUKZUGvKa1/sKxraqSvsAY4PeSHDjAS1rrpQ5sU1W1AGaXVH15AAu01k5Z2ujEgoEfivsPeAHfaK1/dmyTquwpYG5J5zMVeNjB7XGtskwhhBDlc7WUjhBCiHJIwBdCCDchAV8IIdyEBHwhhHATEvCFEMJNSMAXQgg3IQFfCCHcxP8DYWdpnFgQIHMAAAAASUVORK5CYII=\n",
Olivier Michel's avatar
Olivier Michel committed
170
171
172
173
174
175
176
177
178
179
180
181
182
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.ensemble import ExtraTreesRegressor\n",
    "\n",
183
    "clf2=ExtraTreesRegressor(n_estimators=1000, criterion='mse', max_depth=8,\\\n",
Olivier Michel's avatar
Olivier Michel committed
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
    "                         random_state=None)\n",
    "clf2=clf2.fit(X,y.ravel())\n",
    "\n",
    "Ntest=300\n",
    "XX = np.linspace(X.min(),X.max(),Ntest)\n",
    "y_ref=np.sin(XX)\n",
    "XX = XX.reshape(len(XX),1)\n",
    "yp2=clf2.predict(XX)\n",
    "plt.scatter(X,y,s=1)\n",
    "plt.plot(XX,yp2, color='green')\n",
    "\n",
    "# MSE evaluation\n",
    "\n",
    "error=np.square(y_ref-yp2).sum()/Ntest\n",
    "print('MSE = {}'.format(error))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
205
    "## Exercize 12\n",
Olivier Michel's avatar
Olivier Michel committed
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
    "- Study the Extremely Randomized Regressor behaviour for max_depth parameter values (change it in the code above) ranging from 1 to 6. \n",
    "- Explain the green curve observed for max_depth=1\n",
    "- Propose a method for setting the optimal value of max_depth parameter. Implement it (hint: look at notebook N2_a_regression_tree)\n",
    "- Why does the Extremely Randomized Regressor exhibit good performances, although splits are chosen at random? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
243
   "version": "3.8.2"
Olivier Michel's avatar
Olivier Michel committed
244
245
246
247
248
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}