Newer
Older
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img width=\"800px\" src=\"../fidle/img/00-Fidle-header-01.svg\"></img>\n",
"# <!-- TITLE --> [GRAD1] - Linear regression with gradient descent\n",
"<!-- DESC --> Low level implementation of a solution by gradient descent. Basic and stochastic approach.\n",
"<!-- AUTHOR : Jean-Luc Parouty (CNRS/SIMaP) -->\n",
"\n",
"\n",
"## Objectives :\n",
" - To illustrate the iterative approach of a gradient descent\n",
"\n",
"## What we're going to do :\n",
"\n",
"Equation : $ Y = X.\\Theta + N$ \n",
"and $\\Theta = (a,b)$ a vector as y = a.x + b\n",
"We will calculate a loss function and its gradient. \n",
"We will descend this gradient in order to find a minimum value of our loss function.\n",
"\n",
"$\n",
"\\triangledown_\\theta MSE(\\Theta)=\\begin{bmatrix}\n",
"\\frac{\\partial}{\\partial \\theta_0}MSE(\\Theta)\\\\\n",
"\\frac{\\partial}{\\partial \\theta_1}MSE(\\Theta)\\\\\n",
"\\vdots\\\\\n",
"\\frac{\\partial}{\\partial \\theta_n}MSE(\\Theta)\n",
"\\end{bmatrix}=\\frac2m X^T\\cdot(X\\cdot\\Theta-Y)\n",
"$ \n",
"\n",
"and : \n",
"\n",
"$\\Theta \\leftarrow \\Theta - \\eta \\cdot \\triangledown_\\theta MSE(\\Theta)$\n",
"\n",
"where $\\eta$ is the learning rate\n",
"\n",
"## Step 1 - Import and init\n"
"import numpy as np\n",
"import sys\n",
"\n",
"sys.path.append('..')\n",
"import fidle.pwk as pwk\n",
"\n",
"from modules.RegressionCooker import RegressionCooker \n",
"\n",
"# ---- Init Fidle stuffs\n",
"#\n",
"datasets_dir = pwk.init('GRAD1')\n",
"\n",
"# ---- Instanciate a Regression Cooker\n",
"#\n",
"cooker = RegressionCooker(pwk)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 2 - Get a dataset"
]
},
{
"cell_type": "code",
"X,Y = cooker.get_dataset(1000000)\n",
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 3 : Data normalization"
"X_norm = ( X - X.mean() ) / X.std()\n",
"Y_norm = ( Y - Y.mean() ) / Y.std()\n",
"cooker.vector_infos('X origine',X)\n",
"cooker.vector_infos('X normalized',X_norm)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"theta = cooker.basic_descent(X_norm, Y_norm, epochs=200, eta=0.01)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Step 5 - Minibatch descent"
"theta = cooker.minibatch_descent(X_norm, Y_norm, epochs=10, batchs=20, batch_size=10, eta=0.01)"
"source": [
"pwk.end()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"<img width=\"80px\" src=\"../fidle/img/00-Fidle-logo-01.svg\"></img>"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
}
},
"nbformat": 4,
"nbformat_minor": 4
}