{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Práctica: Feature Engineering, Scaling y Vectorización\n",
    "\n",
    "## Objetivos\n",
    "En esta práctica aprenderás a:\n",
    "- Implementar **regresión polinomial** (feature engineering) evaluando múltiples grados y seleccionando el óptimo\n",
    "- Implementar los **3 métodos de feature scaling** vistos en clase y comparar cuál produce menor MSE\n",
    "- Escribir todo el código de forma **vectorizada** usando operaciones de matrices y producto punto\n",
    "\n",
    "---\n",
    "\n",
    "## Herramientas\n",
    "- NumPy: operaciones vectorizadas\n",
    "- Matplotlib: visualización\n",
    "- sklearn (solo al final para verificar)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import time\n",
    "import copy\n",
    "\n",
    "np.set_printoptions(precision=4)\n",
    "plt.style.use('seaborn-v0_8-darkgrid')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# 1. Dataset y Visualización\n",
    "\n",
    "Usaremos datos de precios de casas. La relación entre tamaño y precio **no es lineal** — una recta no captura bien los datos."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x_train shape: (12,)\n",
      "y_train shape: (12,)\n",
      "Número de ejemplos: m = 12\n"
     ]
    }
   ],
   "source": [
    "# Dataset: tamaño (1000 sqft) → precio ($1000s)\n",
    "x_train = np.array([0.5, 0.8, 1.0, 1.2, 1.5, 1.8, 2.0, 2.2, 2.5, 2.8, 3.0, 3.5])\n",
    "y_train = np.array([90, 120, 140, 155, 180, 210, 230, 260, 300, 350, 380, 460])\n",
    "\n",
    "print(f\"x_train shape: {x_train.shape}\")\n",
    "print(f\"y_train shape: {y_train.shape}\")\n",
    "print(f\"Número de ejemplos: m = {x_train.shape[0]}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 46196 characters]",
      "text/plain": [
       "<Figure size 800x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "→ Observación: la relación NO es lineal. Necesitamos features polinomiales.\n"
     ]
    }
   ],
   "source": [
    "# Visualizar los datos\n",
    "plt.figure(figsize=(8, 5))\n",
    "plt.scatter(x_train, y_train, color='royalblue', s=80, zorder=5, label='Datos reales')\n",
    "\n",
    "# Ajuste lineal simple para mostrar que no es suficiente\n",
    "w_simple = np.polyfit(x_train, y_train, 1)\n",
    "x_line = np.linspace(0.3, 3.7, 100)\n",
    "plt.plot(x_line, np.polyval(w_simple, x_line), 'r--', alpha=0.7, label='Ajuste lineal (insuficiente)')\n",
    "\n",
    "plt.xlabel('Tamaño (1000 sqft)')\n",
    "plt.ylabel('Precio ($1000s)')\n",
    "plt.title('Precio de Casas vs Tamaño')\n",
    "plt.legend()\n",
    "plt.grid(True, alpha=0.3)\n",
    "plt.show()\n",
    "\n",
    "print(\"\\n→ Observación: la relación NO es lineal. Necesitamos features polinomiales.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# 2. Repaso: Producto Punto y Multiplicación de Matrices\n",
    "\n",
    "Antes de implementar las funciones vectorizadas, repasemos cómo funciona la multiplicación de matrices y por qué es clave para hacer predicciones eficientes.\n",
    "\n",
    "## 2.1 El Producto Punto (Dot Product)\n",
    "\n",
    "El producto punto entre dos vectores $\\mathbf{w}$ y $\\mathbf{x}$ se define como:\n",
    "\n",
    "$$\\mathbf{w} \\cdot \\mathbf{x} = \\sum_{j=0}^{n-1} w_j \\cdot x_j$$\n",
    "\n",
    "**Ejemplo numérico:**\n",
    "\n",
    "$$\\mathbf{w} = [2, 3, 1], \\quad \\mathbf{x} = [4, 1, 5]$$\n",
    "$$\\mathbf{w} \\cdot \\mathbf{x} = 2 \\times 4 + 3 \\times 1 + 1 \\times 5 = 8 + 3 + 5 = 16$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Con loop:   w · x = 16\n",
      "Con np.dot: w · x = 16\n",
      "\n",
      "✓ Ambos dan el mismo resultado, pero np.dot es mucho más rápido para vectores grandes.\n"
     ]
    }
   ],
   "source": [
    "# Ejemplo: producto punto con loop vs np.dot\n",
    "w = np.array([2, 3, 1])\n",
    "x = np.array([4, 1, 5])\n",
    "\n",
    "# Versión con loop (LENTA)\n",
    "resultado_loop = 0\n",
    "for j in range(len(w)):\n",
    "    resultado_loop += w[j] * x[j]\n",
    "\n",
    "# Versión vectorizada (RÁPIDA)\n",
    "resultado_dot = np.dot(w, x)\n",
    "\n",
    "print(f\"Con loop:   w · x = {resultado_loop}\")\n",
    "print(f\"Con np.dot: w · x = {resultado_dot}\")\n",
    "print(f\"\\n✓ Ambos dan el mismo resultado, pero np.dot es mucho más rápido para vectores grandes.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2 Multiplicación Matriz × Vector = Colección de Dot Products\n",
    "\n",
    "Cuando multiplicamos una matriz $\\mathbf{X}$ de shape $(m, n)$ por un vector $\\mathbf{w}$ de shape $(n,)$, el resultado es un vector de $m$ elementos donde **cada elemento es el producto punto de una fila de X con w**:\n",
    "\n",
    "$$\\mathbf{X} \\cdot \\mathbf{w} = \\begin{pmatrix} \\text{fila}_0 \\cdot \\mathbf{w} \\\\ \\text{fila}_1 \\cdot \\mathbf{w} \\\\ \\vdots \\\\ \\text{fila}_{m-1} \\cdot \\mathbf{w} \\end{pmatrix}$$\n",
    "\n",
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/8/86/Matrix_multiplication_.gif\" width=\"400\">\n",
    "\n",
    "**Esto es exactamente lo que necesitamos para hacer predicciones:**\n",
    "- Cada fila de $\\mathbf{X}$ es un ejemplo (una casa con sus features)\n",
    "- $\\mathbf{w}$ son los pesos del modelo\n",
    "- El resultado es un vector con la predicción para cada casa\n",
    "\n",
    "$$\\text{predicciones} = \\mathbf{X} \\cdot \\mathbf{w} + b$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Predicción con LOOP (lento) ===\n",
      "  Casa 0: fila [1.5 3. ] · w [100  20] + 50 = 260.0\n",
      "  Casa 1: fila [2. 4.] · w [100  20] + 50 = 330.0\n",
      "  Casa 2: fila [1. 2.] · w [100  20] + 50 = 190.0\n",
      "\n",
      "=== Predicción VECTORIZADA (rápido) ===\n",
      "  X @ w + b = [260. 330. 190.]\n",
      "\n",
      "✓ Una sola operación reemplaza el loop completo!\n",
      "  Shapes: X((3, 2)) @ w((2,)) + b → predicciones((3,))\n"
     ]
    }
   ],
   "source": [
    "# Demostración: predecir para 3 casas con 2 features cada una\n",
    "X_demo = np.array([\n",
    "    [1.5, 3],   # casa 0: 1500sqft, 3 cuartos\n",
    "    [2.0, 4],   # casa 1: 2000sqft, 4 cuartos\n",
    "    [1.0, 2],   # casa 2: 1000sqft, 2 cuartos\n",
    "])\n",
    "w_demo = np.array([100, 20])  # $100k por 1000sqft, $20k por cuarto\n",
    "b_demo = 50  # base $50k\n",
    "\n",
    "print(\"=== Predicción con LOOP (lento) ===\")\n",
    "for i in range(X_demo.shape[0]):\n",
    "    pred_i = np.dot(X_demo[i], w_demo) + b_demo\n",
    "    print(f\"  Casa {i}: fila {X_demo[i]} · w {w_demo} + {b_demo} = {pred_i}\")\n",
    "\n",
    "print(\"\\n=== Predicción VECTORIZADA (rápido) ===\")\n",
    "predicciones = X_demo @ w_demo + b_demo  # equivale a np.dot(X_demo, w_demo) + b_demo\n",
    "print(f\"  X @ w + b = {predicciones}\")\n",
    "print(f\"\\n✓ Una sola operación reemplaza el loop completo!\")\n",
    "print(f\"  Shapes: X({X_demo.shape}) @ w({w_demo.shape}) + b → predicciones({predicciones.shape})\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.3 Ejercicio: Implementar predicción vectorizada\n",
    "\n",
    "Implementa una función que prediga el precio de **todas** las casas simultáneamente usando código vectorizado.\n",
    "\n",
    "**Reglas:**\n",
    "- NO puedes usar loops\n",
    "- Usa `@` o `np.dot` para la multiplicación matriz-vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict_all(X, w, b):\n",
    "    \"\"\"\n",
    "    Predice para todos los ejemplos simultáneamente.\n",
    "    \n",
    "    Args:\n",
    "        X (ndarray (m,n)): matriz de features, m ejemplos, n features\n",
    "        w (ndarray (n,)):  vector de pesos\n",
    "        b (scalar):        bias\n",
    "    \n",
    "    Returns:\n",
    "        predictions (ndarray (m,)): vector de predicciones\n",
    "    \"\"\"\n",
    "    # X @ w hace el dot product fila por fila → un valor por cada ejemplo\n",
    "    predictions = X @ w + b\n",
    "    return predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "<summary><span style=\"color: green; font-weight: bold;\">💡 Pista</span></summary>\n",
    "\n",
    "```python\n",
    "# La multiplicación matriz × vector hace dot product fila por fila:\n",
    "predictions = X @ w + b\n",
    "# equivalente a: np.dot(X, w) + b\n",
    "```\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicciones: [260. 330. 190.]\n",
      "Esperado:     [260. 330. 190.]\n",
      "\n",
      "✓ ¡Correcto!\n"
     ]
    }
   ],
   "source": [
    "# Verificar tu implementación\n",
    "preds = predict_all(X_demo, w_demo, b_demo)\n",
    "print(f\"Predicciones: {preds}\")\n",
    "print(f\"Esperado:     [260. 330. 190.]\")\n",
    "assert preds is not None, \"La función retorna None — implementa el código\"\n",
    "assert np.allclose(preds, [260, 330, 190]), f\"Error: esperado [260, 330, 190], obtuviste {preds}\"\n",
    "print(\"\\n✓ ¡Correcto!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# 3. Feature Engineering: Regresión Polinomial\n",
    "\n",
    "## 3.1 La Idea\n",
    "\n",
    "Si los datos no son lineales, podemos crear **nuevas features** a partir de las existentes:\n",
    "\n",
    "$$f(x) = w_1 x + w_2 x^2 + w_3 x^3 + \\ldots + w_d x^d + b$$\n",
    "\n",
    "Esto transforma un problema no-lineal en uno **lineal en las features**:\n",
    "\n",
    "| Feature original | Features polinomiales (grado 3) |\n",
    "|:---:|:---:|\n",
    "| $x$ | $x_1 = x$, $x_2 = x^2$, $x_3 = x^3$ |\n",
    "\n",
    "Ahora el modelo es lineal en $[x_1, x_2, x_3]$:\n",
    "$$f(\\mathbf{x}) = \\mathbf{w} \\cdot \\mathbf{x} + b$$\n",
    "\n",
    "**⚠️ Importante:** Como $x^2, x^3$ crecen rápidamente, es **crucial** aplicar feature scaling después de crear las features polinomiales."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.2 Crear Features Polinomiales"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x original: [1. 2. 3.]\n",
      "\n",
      "X polinomial (grado 3):\n",
      "  [x,  x²,  x³]\n",
      "[[ 1.  1.  1.]\n",
      " [ 2.  4.  8.]\n",
      " [ 3.  9. 27.]]\n",
      "\n",
      "Shape: (3, 3) → 3 ejemplos, 3 features\n"
     ]
    }
   ],
   "source": [
    "def create_polynomial_features(x, degree):\n",
    "    \"\"\"\n",
    "    Crea una matriz de features polinomiales.\n",
    "    \n",
    "    Args:\n",
    "        x (ndarray (m,)): vector de una sola feature\n",
    "        degree (int):     grado máximo del polinomio\n",
    "    \n",
    "    Returns:\n",
    "        X (ndarray (m, degree)): matriz con columnas [x, x², x³, ..., x^degree]\n",
    "    \n",
    "    Ejemplo:\n",
    "        x = [2, 3], degree = 3\n",
    "        X = [[2, 4, 8],\n",
    "             [3, 9, 27]]\n",
    "    \"\"\"\n",
    "    # Nota: esto es vectorizado — no hay loop sobre los m ejemplos\n",
    "    # x**k eleva TODOS los elementos simultáneamente (operación element-wise)\n",
    "    return np.column_stack([x**k for k in range(1, degree + 1)])\n",
    "\n",
    "\n",
    "# Demostración\n",
    "x_ejemplo = np.array([1.0, 2.0, 3.0])\n",
    "X_poly = create_polynomial_features(x_ejemplo, degree=3)\n",
    "print(f\"x original: {x_ejemplo}\")\n",
    "print(f\"\\nX polinomial (grado 3):\")\n",
    "print(f\"  [x,  x²,  x³]\")\n",
    "print(X_poly)\n",
    "print(f\"\\nShape: {X_poly.shape} → {X_poly.shape[0]} ejemplos, {X_poly.shape[1]} features\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.3 Funciones de Entrenamiento\n",
    "\n",
    "### Función de Costo:\n",
    "$$J(\\mathbf{w}, b) = \\frac{1}{2m} \\sum_{i=0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)})^2$$\n",
    "\n",
    "### Gradientes:\n",
    "$$\\frac{\\partial J}{\\partial w_j} = \\frac{1}{m} \\sum_{i=0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)}) \\cdot x_j^{(i)}$$\n",
    "\n",
    "$$\\frac{\\partial J}{\\partial b} = \\frac{1}{m} \\sum_{i=0}^{m-1} (f_{\\mathbf{w},b}(\\mathbf{x}^{(i)}) - y^{(i)})$$\n",
    "\n",
    "A continuación se presenta la implementación **con loops** (referencia). Tu tarea será implementar la versión **vectorizada**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost (loop):     34651.0417\n",
      "Gradient (loop): dj_dw = [ -552.25  -1450.025], dj_db = -239.5833\n",
      "\n",
      "✓ Estas funciones son CORRECTAS pero LENTAS. Ahora implementa la versión vectorizada.\n"
     ]
    }
   ],
   "source": [
    "# ============================================================\n",
    "# REFERENCIA: Implementación CON LOOPS (lenta pero clara)\n",
    "# ============================================================\n",
    "\n",
    "def compute_cost_loop(X, y, w, b):\n",
    "    \"\"\"\n",
    "    Calcula el costo MSE usando loops.\n",
    "    Funciona pero es LENTA para datasets grandes.\n",
    "    \"\"\"\n",
    "    m = X.shape[0]\n",
    "    cost = 0.0\n",
    "    for i in range(m):\n",
    "        f_wb_i = np.dot(X[i], w) + b    # predicción para ejemplo i\n",
    "        cost += (f_wb_i - y[i])**2        # error cuadrado acumulado\n",
    "    cost = cost / (2 * m)\n",
    "    return cost\n",
    "\n",
    "\n",
    "def compute_gradient_loop(X, y, w, b):\n",
    "    \"\"\"\n",
    "    Calcula gradientes usando loops.\n",
    "    Funciona pero es LENTA para datasets grandes.\n",
    "    \"\"\"\n",
    "    m, n = X.shape\n",
    "    dj_dw = np.zeros(n)\n",
    "    dj_db = 0.0\n",
    "    \n",
    "    for i in range(m):\n",
    "        err = (np.dot(X[i], w) + b) - y[i]\n",
    "        for j in range(n):\n",
    "            dj_dw[j] += err * X[i, j]\n",
    "        dj_db += err\n",
    "    \n",
    "    dj_dw = dj_dw / m\n",
    "    dj_db = dj_db / m\n",
    "    return dj_dw, dj_db\n",
    "\n",
    "\n",
    "# Verificar que funcionan\n",
    "X_test = create_polynomial_features(x_train, 2)\n",
    "w_test = np.zeros(2)\n",
    "b_test = 0.0\n",
    "\n",
    "cost_loop = compute_cost_loop(X_test, y_train, w_test, b_test)\n",
    "dj_dw_loop, dj_db_loop = compute_gradient_loop(X_test, y_train, w_test, b_test)\n",
    "print(f\"Cost (loop):     {cost_loop:.4f}\")\n",
    "print(f\"Gradient (loop): dj_dw = {dj_dw_loop}, dj_db = {dj_db_loop:.4f}\")\n",
    "print(\"\\n✓ Estas funciones son CORRECTAS pero LENTAS. Ahora implementa la versión vectorizada.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.4 Ejercicio: Implementar costo vectorizado\n",
    "\n",
    "Implementa `compute_cost` **sin loops**. El resultado debe ser idéntico al de `compute_cost_loop`.\n",
    "\n",
    "**Recordatorio de cómo vectorizar:**\n",
    "- `X @ w + b` calcula TODAS las predicciones de golpe (un dot product por fila de X)\n",
    "- `predictions - y` calcula todos los errores de golpe (operación element-wise)\n",
    "- `np.mean(errors**2)` calcula la media de los errores al cuadrado\n",
    "\n",
    "Observa la relación:\n",
    "\n",
    "| Loop | Vectorizado |\n",
    "|:-----|:------------|\n",
    "| `for i in range(m): f_wb_i = np.dot(X[i], w) + b` | `predictions = X @ w + b` |\n",
    "| `for i: cost += (f_wb_i - y[i])**2` | `cost = np.mean((predictions - y)**2) / 2` |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_cost(X, y, w, b):\n",
    "    \"\"\"\n",
    "    Calcula el costo MSE de forma VECTORIZADA (sin loops).\n",
    "    \n",
    "    Args:\n",
    "        X (ndarray (m,n)): features\n",
    "        y (ndarray (m,)):  targets\n",
    "        w (ndarray (n,)):  pesos\n",
    "        b (scalar):        bias\n",
    "    \n",
    "    Returns:\n",
    "        cost (scalar): valor del costo J(w,b)\n",
    "    \"\"\"\n",
    "    predictions = X @ w + b           # todas las predicciones de golpe → shape (m,)\n",
    "    errors = predictions - y           # todos los errores de golpe      → shape (m,)\n",
    "    cost = np.mean(errors**2) / 2     # equivale a sum(errors²) / (2*m)\n",
    "    return cost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "<summary><span style=\"color: green; font-weight: bold;\">💡 Pista — compute_cost vectorizado</span></summary>\n",
    "\n",
    "```python\n",
    "predictions = X @ w + b            # (m,n) @ (n,) → (m,)\n",
    "errors = predictions - y            # (m,)\n",
    "cost = np.mean(errors**2) / 2       # scalar\n",
    "return cost\n",
    "```\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost (loop):        34651.041667\n",
      "Cost (vectorizado): 34651.041666666664\n",
      "\n",
      "✓ ¡Correcto! Mismo resultado, sin loops.\n"
     ]
    }
   ],
   "source": [
    "# Verificar: tu resultado debe ser igual al de la versión con loop\n",
    "cost_vec = compute_cost(X_test, y_train, w_test, b_test)\n",
    "print(f\"Cost (loop):        {cost_loop:.6f}\")\n",
    "print(f\"Cost (vectorizado): {cost_vec}\")\n",
    "assert cost_vec is not None, \"compute_cost retorna None — implementa el código\"\n",
    "assert np.isclose(cost_vec, cost_loop), f\"Error: esperado {cost_loop:.6f}, obtuviste {cost_vec}\"\n",
    "print(\"\\n✓ ¡Correcto! Mismo resultado, sin loops.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.5 Ejercicio: Implementar gradiente vectorizado\n",
    "\n",
    "Implementa `compute_gradient` **sin loops**. El resultado debe ser idéntico al de `compute_gradient_loop`.\n",
    "\n",
    "**Truco clave para `dj_dw`:**\n",
    "\n",
    "El loop original hace:\n",
    "```\n",
    "for i in range(m):\n",
    "    err = predicción_i - y_i\n",
    "    for j in range(n):\n",
    "        dj_dw[j] += err * X[i, j]\n",
    "```\n",
    "\n",
    "Esto es equivalente a multiplicar la **transpuesta** de X por el vector de errores:\n",
    "\n",
    "$$\\frac{\\partial J}{\\partial \\mathbf{w}} = \\frac{1}{m} \\mathbf{X}^T \\cdot \\mathbf{errors}$$\n",
    "\n",
    "¿Por qué? Porque $\\mathbf{X}^T$ tiene shape $(n, m)$. Al multiplicar por `errors` de shape $(m,)$:\n",
    "- Cada **fila** de $\\mathbf{X}^T$ (que es una **columna** de $\\mathbf{X}$, es decir, todos los valores de una feature) hace dot product con el vector de errores.\n",
    "- Resultado: un vector de shape $(n,)$ con el gradiente de cada peso.\n",
    "\n",
    "<img src=\"https://anakarinarm.github.io/python-novice-inflammation-2016-09-20-ubc/fig/python-operations-across-axes.png\" width=\"500\">"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compute_gradient(X, y, w, b):\n",
    "    \"\"\"\n",
    "    Calcula gradientes de forma VECTORIZADA (sin loops).\n",
    "    \n",
    "    Args:\n",
    "        X (ndarray (m,n)): features\n",
    "        y (ndarray (m,)):  targets\n",
    "        w (ndarray (n,)):  pesos\n",
    "        b (scalar):        bias\n",
    "    \n",
    "    Returns:\n",
    "        dj_dw (ndarray (n,)): gradiente respecto a w\n",
    "        dj_db (scalar):       gradiente respecto a b\n",
    "    \"\"\"\n",
    "    m = X.shape[0]\n",
    "    predictions = X @ w + b       # todas las predicciones de golpe → shape (m,)\n",
    "    errors = predictions - y       # todos los errores de golpe      → shape (m,)\n",
    "\n",
    "    dj_dw = (X.T @ errors) / m   # (n,m) @ (m,) → (n,): gradiente por cada peso\n",
    "    dj_db = np.mean(errors)       # promedio de errores → gradiente del bias\n",
    "    return dj_dw, dj_db"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "<summary><span style=\"color: green; font-weight: bold;\">💡 Pista — compute_gradient vectorizado</span></summary>\n",
    "\n",
    "```python\n",
    "m = X.shape[0]\n",
    "predictions = X @ w + b          # (m,)\n",
    "errors = predictions - y          # (m,)\n",
    "\n",
    "dj_dw = (X.T @ errors) / m       # (n,m) @ (m,) → (n,)\n",
    "dj_db = np.mean(errors)           # scalar\n",
    "return dj_dw, dj_db\n",
    "```\n",
    "`X.T @ errors` hace un dot product por cada feature (fila de X.T) con el vector de errores.\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Gradient (loop):        dj_dw = [ -552.25  -1450.025], dj_db = -239.5833\n",
      "Gradient (vectorizado): dj_dw = [ -552.25  -1450.025], dj_db = -239.58333333333334\n",
      "\n",
      "✓ ¡Correcto! Mismo resultado, sin loops.\n"
     ]
    }
   ],
   "source": [
    "# Verificar: tu resultado debe ser igual al de la versión con loop\n",
    "dj_dw_vec, dj_db_vec = compute_gradient(X_test, y_train, w_test, b_test)\n",
    "print(f\"Gradient (loop):        dj_dw = {dj_dw_loop}, dj_db = {dj_db_loop:.4f}\")\n",
    "print(f\"Gradient (vectorizado): dj_dw = {dj_dw_vec}, dj_db = {dj_db_vec}\")\n",
    "assert dj_dw_vec is not None, \"compute_gradient retorna None — implementa el código\"\n",
    "assert np.allclose(dj_dw_vec, dj_dw_loop), f\"Error en dj_dw: esperado {dj_dw_loop}, obtuviste {dj_dw_vec}\"\n",
    "assert np.isclose(dj_db_vec, dj_db_loop), f\"Error en dj_db: esperado {dj_db_loop}, obtuviste {dj_db_vec}\"\n",
    "print(\"\\n✓ ¡Correcto! Mismo resultado, sin loops.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost con loop:        924.7 ms (100 repeticiones)\n",
      "Cost vectorizado:     4.1 ms (100 repeticiones)\n",
      "\n",
      "→ Speedup: 226x más rápido con vectorización!\n"
     ]
    }
   ],
   "source": [
    "# Comparar velocidad: loop vs vectorizado\n",
    "# Usamos un dataset más grande para notar la diferencia\n",
    "X_big = np.random.randn(5000, 10)\n",
    "y_big = np.random.randn(5000)\n",
    "w_big = np.random.randn(10)\n",
    "\n",
    "tic = time.time()\n",
    "for _ in range(100):\n",
    "    compute_cost_loop(X_big, y_big, w_big, 0.0)\n",
    "time_loop = time.time() - tic\n",
    "\n",
    "tic = time.time()\n",
    "for _ in range(100):\n",
    "    compute_cost(X_big, y_big, w_big, 0.0)\n",
    "time_vec = time.time() - tic\n",
    "\n",
    "print(f\"Cost con loop:        {time_loop*1000:.1f} ms (100 repeticiones)\")\n",
    "print(f\"Cost vectorizado:     {time_vec*1000:.1f} ms (100 repeticiones)\")\n",
    "print(f\"\\n→ Speedup: {time_loop/time_vec:.0f}x más rápido con vectorización!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.6 Gradient Descent (dado)\n",
    "\n",
    "Esta función usa tus implementaciones vectorizadas de `compute_cost` y `compute_gradient` para entrenar el modelo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "✓ gradient_descent definido (usa TUS funciones vectorizadas).\n"
     ]
    }
   ],
   "source": [
    "def gradient_descent(X, y, w_init, b_init, alpha, num_iters):\n",
    "    \"\"\"\n",
    "    Ejecuta gradient descent usando las funciones vectorizadas del estudiante.\n",
    "    \"\"\"\n",
    "    w = copy.deepcopy(w_init)\n",
    "    b = b_init\n",
    "    J_history = []\n",
    "    \n",
    "    for i in range(num_iters):\n",
    "        dj_dw, dj_db = compute_gradient(X, y, w, b)\n",
    "        w = w - alpha * dj_dw\n",
    "        b = b - alpha * dj_db\n",
    "        J_history.append(compute_cost(X, y, w, b))\n",
    "        \n",
    "        if i % (num_iters // 5) == 0:\n",
    "            print(f\"  Iter {i:5d}: Cost = {J_history[-1]:.4f}\")\n",
    "    \n",
    "    return w, b, J_history\n",
    "\n",
    "print(\"✓ gradient_descent definido (usa TUS funciones vectorizadas).\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.7 Ejercicio: Evaluar grados 1 a 6\n",
    "\n",
    "Entrena un modelo para cada grado polinomial (1 a 6). Para cada grado debes:\n",
    "1. Crear las features polinomiales\n",
    "2. Aplicar z-score normalization\n",
    "3. Entrenar con gradient descent\n",
    "4. Guardar el MSE final\n",
    "\n",
    "**Recuerda:** La función z-score es:\n",
    "$$x_{\\text{scaled}} = \\frac{x - \\mu}{\\sigma}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "==================================================\n",
      "Grado 1:\n",
      "==================================================\n",
      "  Iter     0: Cost = 28088.9883\n",
      "  Iter  1000: Cost = 113.9187\n",
      "  Iter  2000: Cost = 113.9187\n",
      "  Iter  3000: Cost = 113.9187\n",
      "  Iter  4000: Cost = 113.9187\n",
      "  MSE final: 227.8373\n",
      "\n",
      "==================================================\n",
      "Grado 2:\n",
      "==================================================\n",
      "  Iter     0: Cost = 27081.8270\n",
      "  Iter  1000: Cost = 7.7805\n",
      "  Iter  2000: Cost = 7.7385\n",
      "  Iter  3000: Cost = 7.7381\n",
      "  Iter  4000: Cost = 7.7381\n",
      "  MSE final: 15.4762\n",
      "\n",
      "==================================================\n",
      "Grado 3:\n",
      "==================================================\n",
      "  Iter     0: Cost = 26237.9740\n",
      "  Iter  1000: Cost = 7.2174\n",
      "  Iter  2000: Cost = 7.2173\n",
      "  Iter  3000: Cost = 7.2171\n",
      "  Iter  4000: Cost = 7.2170\n",
      "  MSE final: 14.4338\n",
      "\n",
      "==================================================\n",
      "Grado 4:\n",
      "==================================================\n",
      "  Iter     0: Cost = 25565.3658\n",
      "  Iter  1000: Cost = 7.2402\n",
      "  Iter  2000: Cost = 7.1859\n",
      "  Iter  3000: Cost = 7.1448\n",
      "  Iter  4000: Cost = 7.1122\n",
      "  MSE final: 14.1703\n",
      "\n",
      "==================================================\n",
      "Grado 5:\n",
      "==================================================\n",
      "  Iter     0: Cost = 25050.2938\n",
      "  Iter  1000: Cost = 6.7447\n",
      "  Iter  2000: Cost = 6.6333\n",
      "  Iter  3000: Cost = 6.5485\n",
      "  Iter  4000: Cost = 6.4739\n",
      "  MSE final: 12.8073\n",
      "\n",
      "==================================================\n",
      "Grado 6:\n",
      "==================================================\n",
      "  Iter     0: Cost = 24673.3149\n",
      "  Iter  1000: Cost = 6.1382\n",
      "  Iter  2000: Cost = 5.9781\n",
      "  Iter  3000: Cost = 5.8256\n",
      "  Iter  4000: Cost = 5.6784\n",
      "  MSE final: 11.0721\n"
     ]
    }
   ],
   "source": [
    "degrees = [1, 2, 3, 4, 5, 6]\n",
    "results = {}  # almacenará {grado: {'w': w, 'b': b, 'mse': mse, 'mu': mu, 'sigma': sigma}}\n",
    "\n",
    "alpha = 0.1\n",
    "num_iters = 5000\n",
    "\n",
    "for deg in degrees:\n",
    "    print(f\"\\n{'='*50}\")\n",
    "    print(f\"Grado {deg}:\")\n",
    "    print(f\"{'='*50}\")\n",
    "    \n",
    "    # Paso 1: Crear features polinomiales [x, x², x³, ..., x^deg]\n",
    "    X_poly = create_polynomial_features(x_train, deg)\n",
    "    \n",
    "    # Paso 2: Z-score normalization\n",
    "    # axis=0 → calcula media/std por columna (una por feature)\n",
    "    mu = X_poly.mean(axis=0)\n",
    "    sigma = X_poly.std(axis=0)\n",
    "    X_norm = (X_poly - mu) / sigma\n",
    "    \n",
    "    # Paso 3: Entrenar\n",
    "    w_init = np.zeros(deg)\n",
    "    b_init = 0.0\n",
    "    w_final, b_final, J_hist = gradient_descent(X_norm, y_train, w_init, b_init, alpha, num_iters)\n",
    "    \n",
    "    # Paso 4: Calcular MSE final\n",
    "    predictions = X_norm @ w_final + b_final\n",
    "    mse = np.mean((predictions - y_train)**2)\n",
    "    \n",
    "    results[deg] = {'w': w_final, 'b': b_final, 'mse': mse, 'mu': mu, 'sigma': sigma}\n",
    "    print(f\"  MSE final: {mse:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "<summary><span style=\"color: green; font-weight: bold;\">💡 Pista — Paso 1</span></summary>\n",
    "\n",
    "```python\n",
    "X_poly = create_polynomial_features(x_train, deg)\n",
    "```\n",
    "\n",
    "</details>\n",
    "\n",
    "<details>\n",
    "<summary><span style=\"color: green; font-weight: bold;\">💡 Pista — Paso 2 (z-score)</span></summary>\n",
    "\n",
    "```python\n",
    "mu = X_poly.mean(axis=0)      # media de cada columna\n",
    "sigma = X_poly.std(axis=0)    # std de cada columna\n",
    "X_norm = (X_poly - mu) / sigma\n",
    "```\n",
    "Nota: `axis=0` opera sobre las filas → resultado tiene shape (n,), una media/std por feature.\n",
    "\n",
    "</details>\n",
    "\n",
    "<details>\n",
    "<summary><span style=\"color: green; font-weight: bold;\">💡 Pista — Paso 4 (MSE)</span></summary>\n",
    "\n",
    "```python\n",
    "predictions = X_norm @ w_final + b_final\n",
    "mse = np.mean((predictions - y_train)**2)\n",
    "```\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.8 Visualización Comparativa\n",
    "\n",
    "Ejecuta la siguiente celda para ver las curvas de cada grado y comparar MSE."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 126004 characters]",
      "text/plain": [
       "<Figure size 1400x900 with 6 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "========================================\n",
      "Grado   MSE            Ranking\n",
      "========================================\n",
      "  6     11.07          1 ← MEJOR\n",
      "  5     12.81          2\n",
      "  4     14.17          3\n",
      "  3     14.43          4\n",
      "  2     15.48          5\n",
      "  1     227.84         6\n"
     ]
    }
   ],
   "source": [
    "# Visualización: curvas ajustadas para cada grado\n",
    "fig, axes = plt.subplots(2, 3, figsize=(14, 9))\n",
    "x_plot = np.linspace(0.3, 3.7, 200)\n",
    "\n",
    "for idx, deg in enumerate(degrees):\n",
    "    ax = axes[idx // 3, idx % 3]\n",
    "    ax.scatter(x_train, y_train, color='royalblue', s=40, zorder=5)\n",
    "    \n",
    "    if results[deg]['mse'] is not None:\n",
    "        # Generar curva de predicción\n",
    "        X_plot_poly = create_polynomial_features(x_plot, deg)\n",
    "        X_plot_norm = (X_plot_poly - results[deg]['mu']) / results[deg]['sigma']\n",
    "        y_plot = X_plot_norm @ results[deg]['w'] + results[deg]['b']\n",
    "        ax.plot(x_plot, y_plot, 'r-', linewidth=2)\n",
    "        ax.set_title(f\"Grado {deg} — MSE: {results[deg]['mse']:.1f}\", fontsize=11)\n",
    "    else:\n",
    "        ax.set_title(f\"Grado {deg} — Sin completar\", fontsize=11, color='gray')\n",
    "    \n",
    "    ax.set_xlabel('Tamaño')\n",
    "    ax.set_ylabel('Precio')\n",
    "    ax.set_ylim([50, 520])\n",
    "    ax.grid(True, alpha=0.3)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.suptitle('Regresión Polinomial: Comparación de Grados', fontsize=13, y=1.02)\n",
    "plt.show()\n",
    "\n",
    "# Tabla de resultados\n",
    "print(\"\\n\" + \"=\"*40)\n",
    "print(f\"{'Grado':<8}{'MSE':<15}{'Ranking'}\")\n",
    "print(\"=\"*40)\n",
    "valid_results = {k: v for k, v in results.items() if v['mse'] is not None}\n",
    "if valid_results:\n",
    "    sorted_degrees = sorted(valid_results.keys(), key=lambda d: valid_results[d]['mse'])\n",
    "    for rank, deg in enumerate(sorted_degrees, 1):\n",
    "        marker = \" ← MEJOR\" if rank == 1 else \"\"\n",
    "        print(f\"  {deg:<6}{valid_results[deg]['mse']:<15.2f}{rank}{marker}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.9 Pregunta de Reflexión\n",
    "\n",
    "Basándote en los resultados:\n",
    "1. ¿Cuál es el mejor grado polinomial para estos datos?\n",
    "2. ¿Por qué grados muy altos (5, 6) no necesariamente dan mejor MSE de entrenamiento?\n",
    "3. ¿Qué riesgo existe al usar un grado muy alto? (piensa en datos nuevos)\n",
    "\n",
    "*Escribe tu respuesta abajo:*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Tu respuesta aquí:**\n",
    "\n",
    "1. El mejor grado es el 3 (MSE = 14.4). Con este grado la curva se ajusta bien a los datos dando equilibrio entre capturar la forma real y no complicarse de más.\n",
    "\n",
    "2. Los grados altos no necesariamente ganan porque con solo 12 datos y un número fijo de iteraciones, gradient descent no tiene suficiente tiempo para encontrar buenos valores para tantos parámetros a la vez.\n",
    "\n",
    "3. El riesgo es el **overfitting**: la curva se tuerce tanto para pasar por todos los puntos de entrenamiento que si le muestras una casa nueva, predice un precio absurdo. Memoriza en lugar de aprender."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# 4. Feature Scaling: Comparativa de 3 Métodos\n",
    "\n",
    "## 4.1 El Problema\n",
    "\n",
    "Cuando las features tienen **escalas muy diferentes**, gradient descent converge lentamente porque los gradientes están dominados por las features grandes.\n",
    "\n",
    "Ejemplo: `size` va de 852 a 2104, pero `bedrooms` solo de 2 a 5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Rangos de cada feature (sin escalar):\n",
      "  size:     [852, 2200]  rango = 1348\n",
      "  bedrooms: [2, 5]    rango = 3\n",
      "  age:      [20, 55]   rango = 35\n",
      "\n",
      "→ El rango de 'size' es ~400x mayor que 'bedrooms'. Esto causa problemas en GD.\n"
     ]
    }
   ],
   "source": [
    "# Dataset multi-feature para la comparativa de scaling\n",
    "# size(sqft), bedrooms, age(años) → price($1000s)\n",
    "X_multi = np.array([\n",
    "    [2104, 5, 45],\n",
    "    [1416, 3, 40],\n",
    "    [852,  2, 35],\n",
    "    [1534, 3, 30],\n",
    "    [2000, 4, 50],\n",
    "    [1200, 2, 25],\n",
    "    [1800, 4, 42],\n",
    "    [950,  2, 20],\n",
    "    [1600, 3, 38],\n",
    "    [2200, 5, 55]\n",
    "])\n",
    "y_multi = np.array([460, 232, 178, 252, 380, 195, 320, 160, 275, 490])\n",
    "\n",
    "print(\"Rangos de cada feature (sin escalar):\")\n",
    "print(f\"  size:     [{X_multi[:,0].min():.0f}, {X_multi[:,0].max():.0f}]  rango = {X_multi[:,0].max() - X_multi[:,0].min():.0f}\")\n",
    "print(f\"  bedrooms: [{X_multi[:,1].min():.0f}, {X_multi[:,1].max():.0f}]    rango = {X_multi[:,1].max() - X_multi[:,1].min():.0f}\")\n",
    "print(f\"  age:      [{X_multi[:,2].min():.0f}, {X_multi[:,2].max():.0f}]   rango = {X_multi[:,2].max() - X_multi[:,2].min():.0f}\")\n",
    "print(\"\\n→ El rango de 'size' es ~400x mayor que 'bedrooms'. Esto causa problemas en GD.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2 Método 1: Min-Max Scaling\n",
    "\n",
    "$$x_{\\text{scaled}} = \\frac{x}{\\max(x)}$$\n",
    "\n",
    "Resultado: cada feature queda en rango $[0, 1]$ aproximadamente.\n",
    "\n",
    "**Ejercicio:** Implementa la función. Debe ser vectorizada (sin loops)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [],
   "source": [
    "def min_max_scale(X):\n",
    "    \"\"\"\n",
    "    Aplica Min-Max Scaling: divide cada feature por su valor máximo.\n",
    "    \n",
    "    Args:\n",
    "        X (ndarray (m,n)): matriz de features\n",
    "    \n",
    "    Returns:\n",
    "        X_scaled (ndarray (m,n)): features escaladas\n",
    "        params (dict): parámetros para re-escalar datos nuevos\n",
    "    \"\"\"\n",
    "    # X.max(axis=0) calcula el máximo de cada columna → shape (n,)\n",
    "    # NumPy broadcasting divide cada columna de X por su máximo automáticamente\n",
    "    X_scaled = X / X.max(axis=0)\n",
    "    \n",
    "    params = {'max': X.max(axis=0)}\n",
    "    return X_scaled, params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "<summary><span style=\"color: green; font-weight: bold;\">💡 Pista</span></summary>\n",
    "\n",
    "```python\n",
    "X_scaled = X / X.max(axis=0)\n",
    "```\n",
    "`X.max(axis=0)` retorna el máximo de cada columna → shape (n,).  \n",
    "NumPy broadcasting divide cada columna de X por su máximo.\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3 Método 2: Mean Normalization\n",
    "\n",
    "$$x_{\\text{scaled}} = \\frac{x - \\mu}{\\max(x) - \\min(x)}$$\n",
    "\n",
    "Resultado: cada feature queda centrada en 0, rango $\\approx [-0.5, 0.5]$.\n",
    "\n",
    "**Ejercicio:** Implementa la función de forma vectorizada."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [],
   "source": [
    "def mean_normalize(X):\n",
    "    \"\"\"\n",
    "    Aplica Mean Normalization.\n",
    "    \n",
    "    Args:\n",
    "        X (ndarray (m,n)): matriz de features\n",
    "    \n",
    "    Returns:\n",
    "        X_scaled (ndarray (m,n)): features escaladas\n",
    "        params (dict): parámetros {mu, rango} para re-escalar datos nuevos\n",
    "    \"\"\"\n",
    "    mu = X.mean(axis=0)                      # media de cada columna → shape (n,)\n",
    "    rango = X.max(axis=0) - X.min(axis=0)   # rango de cada columna → shape (n,)\n",
    "    X_scaled = (X - mu) / rango             # centra en 0 y escala por el rango\n",
    "    \n",
    "    params = {'mu': mu, 'rango': rango}\n",
    "    return X_scaled, params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "<summary><span style=\"color: green; font-weight: bold;\">💡 Pista</span></summary>\n",
    "\n",
    "```python\n",
    "mu = X.mean(axis=0)\n",
    "rango = X.max(axis=0) - X.min(axis=0)\n",
    "X_scaled = (X - mu) / rango\n",
    "```\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.4 Método 3: Z-Score Normalization (Estandarización)\n",
    "\n",
    "$$x_{\\text{scaled}} = \\frac{x - \\mu}{\\sigma}$$\n",
    "\n",
    "Resultado: cada feature tiene media = 0 y desviación estándar = 1.\n",
    "\n",
    "**Ejercicio:** Implementa la función de forma vectorizada."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [],
   "source": [
    "def z_score_normalize(X):\n",
    "    \"\"\"\n",
    "    Aplica Z-Score Normalization (estandarización).\n",
    "    \n",
    "    Args:\n",
    "        X (ndarray (m,n)): matriz de features\n",
    "    \n",
    "    Returns:\n",
    "        X_scaled (ndarray (m,n)): features escaladas\n",
    "        params (dict): parámetros {mu, sigma} para re-escalar datos nuevos\n",
    "    \"\"\"\n",
    "    mu = X.mean(axis=0)      # media de cada columna    → shape (n,)\n",
    "    sigma = X.std(axis=0)    # std de cada columna      → shape (n,)\n",
    "    X_scaled = (X - mu) / sigma  # centra en 0 y escala a std=1\n",
    "    \n",
    "    params = {'mu': mu, 'sigma': sigma}\n",
    "    return X_scaled, params"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "<summary><span style=\"color: green; font-weight: bold;\">💡 Pista</span></summary>\n",
    "\n",
    "```python\n",
    "mu = X.mean(axis=0)\n",
    "sigma = X.std(axis=0)\n",
    "X_scaled = (X - mu) / sigma\n",
    "```\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.5 Comparativa: Entrenar con cada método\n",
    "\n",
    "Ahora entrenaremos el modelo **4 veces** con los mismos datos pero diferente scaling, y compararemos:\n",
    "1. Sin scaling\n",
    "2. Min-Max\n",
    "3. Mean Normalization\n",
    "4. Z-Score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "==================================================\n",
      "Sin Scaling (alpha muy pequeño):\n",
      "==================================================\n",
      "  Iter     0: Cost = 26986.9952\n",
      "  Iter   600: Cost = 738.4275\n",
      "  Iter  1200: Cost = 738.3550\n",
      "  Iter  1800: Cost = 738.2828\n",
      "  Iter  2400: Cost = 738.2108\n",
      "\n",
      "==================================================\n",
      "Min-Max Scaling:\n",
      "==================================================\n",
      "  Iter     0: Cost = 28657.0633\n",
      "  Iter   600: Cost = 270.1990\n",
      "  Iter  1200: Cost = 253.4211\n",
      "  Iter  1800: Cost = 247.0957\n",
      "  Iter  2400: Cost = 243.2751\n",
      "\n",
      "==================================================\n",
      "Mean Normalization:\n",
      "==================================================\n",
      "  Iter     0: Cost = 40764.4710\n",
      "  Iter   600: Cost = 248.6136\n",
      "  Iter  1200: Cost = 239.7831\n",
      "  Iter  1800: Cost = 236.1698\n",
      "  Iter  2400: Cost = 234.2520\n",
      "\n",
      "==================================================\n",
      "Z-Score Normalization:\n",
      "==================================================\n",
      "  Iter     0: Cost = 38374.3762\n",
      "  Iter   600: Cost = 232.0343\n",
      "  Iter  1200: Cost = 231.6551\n",
      "  Iter  1800: Cost = 231.6508\n",
      "  Iter  2400: Cost = 231.6507\n"
     ]
    }
   ],
   "source": [
    "# Configuración del experimento\n",
    "alpha_scaling = 1e-7  # learning rate pequeño (necesario sin scaling)\n",
    "alpha_scaled = 0.1     # learning rate normal (con scaling)\n",
    "iters = 3000\n",
    "n_features = X_multi.shape[1]\n",
    "\n",
    "scaling_results = {}\n",
    "\n",
    "# --- Experimento 1: Sin scaling ---\n",
    "print(\"=\"*50)\n",
    "print(\"Sin Scaling (alpha muy pequeño):\")\n",
    "print(\"=\"*50)\n",
    "w0, b0, J0 = gradient_descent(X_multi, y_multi, np.zeros(n_features), 0.0, alpha_scaling, iters)\n",
    "mse0 = np.mean((X_multi @ w0 + b0 - y_multi)**2)\n",
    "scaling_results['Sin scaling'] = {'J_hist': J0, 'mse': mse0, 'color': 'red'}\n",
    "\n",
    "# --- Experimento 2: Min-Max ---\n",
    "print(\"\\n\" + \"=\"*50)\n",
    "print(\"Min-Max Scaling:\")\n",
    "print(\"=\"*50)\n",
    "X_mm, params_mm = min_max_scale(X_multi)\n",
    "if X_mm is not None:\n",
    "    w1, b1, J1 = gradient_descent(X_mm, y_multi, np.zeros(n_features), 0.0, alpha_scaled, iters)\n",
    "    mse1 = np.mean((X_mm @ w1 + b1 - y_multi)**2)\n",
    "    scaling_results['Min-Max'] = {'J_hist': J1, 'mse': mse1, 'color': 'blue'}\n",
    "else:\n",
    "    print(\"  ⚠️ min_max_scale() no implementada\")\n",
    "\n",
    "# --- Experimento 3: Mean Normalization ---\n",
    "print(\"\\n\" + \"=\"*50)\n",
    "print(\"Mean Normalization:\")\n",
    "print(\"=\"*50)\n",
    "X_mn, params_mn = mean_normalize(X_multi)\n",
    "if X_mn is not None:\n",
    "    w2, b2, J2 = gradient_descent(X_mn, y_multi, np.zeros(n_features), 0.0, alpha_scaled, iters)\n",
    "    mse2 = np.mean((X_mn @ w2 + b2 - y_multi)**2)\n",
    "    scaling_results['Mean Norm'] = {'J_hist': J2, 'mse': mse2, 'color': 'green'}\n",
    "else:\n",
    "    print(\"  ⚠️ mean_normalize() no implementada\")\n",
    "\n",
    "# --- Experimento 4: Z-Score ---\n",
    "print(\"\\n\" + \"=\"*50)\n",
    "print(\"Z-Score Normalization:\")\n",
    "print(\"=\"*50)\n",
    "X_zs, params_zs = z_score_normalize(X_multi)\n",
    "if X_zs is not None:\n",
    "    w3, b3, J3 = gradient_descent(X_zs, y_multi, np.zeros(n_features), 0.0, alpha_scaled, iters)\n",
    "    mse3 = np.mean((X_zs @ w3 + b3 - y_multi)**2)\n",
    "    scaling_results['Z-Score'] = {'J_hist': J3, 'mse': mse3, 'color': 'purple'}\n",
    "else:\n",
    "    print(\"  ⚠️ z_score_normalize() no implementada\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.6 Visualización de Convergencia"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 77288 characters]",
      "text/plain": [
       "<Figure size 1400x500 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "==================================================\n",
      "Método            MSE Final      Convergió?\n",
      "==================================================\n",
      "  Sin scaling     1476.28        Parcial\n",
      "  Min-Max         481.47         Sí\n",
      "  Mean Norm       466.32         Sí\n",
      "  Z-Score         463.30         Sí\n"
     ]
    }
   ],
   "source": [
    "# Gráfica comparativa de convergencia\n",
    "fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 5))\n",
    "\n",
    "# Panel izquierdo: todas las curvas\n",
    "for name, data in scaling_results.items():\n",
    "    ax1.plot(data['J_hist'], color=data['color'], linewidth=2, label=f\"{name} (MSE={data['mse']:.1f})\")\n",
    "\n",
    "ax1.set_xlabel('Iteración')\n",
    "ax1.set_ylabel('Costo J')\n",
    "ax1.set_title('Convergencia: Costo vs Iteraciones')\n",
    "ax1.legend()\n",
    "ax1.grid(True, alpha=0.3)\n",
    "\n",
    "# Panel derecho: zoom en las curvas con scaling (sin la roja que domina)\n",
    "for name, data in scaling_results.items():\n",
    "    if name != 'Sin scaling':\n",
    "        ax2.plot(data['J_hist'], color=data['color'], linewidth=2, label=f\"{name} (MSE={data['mse']:.1f})\")\n",
    "\n",
    "ax2.set_xlabel('Iteración')\n",
    "ax2.set_ylabel('Costo J')\n",
    "ax2.set_title('Zoom: Solo métodos con scaling')\n",
    "ax2.legend()\n",
    "ax2.grid(True, alpha=0.3)\n",
    "\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "# Tabla resumen\n",
    "print(\"\\n\" + \"=\"*50)\n",
    "print(f\"{'Método':<18}{'MSE Final':<15}{'Convergió?'}\")\n",
    "print(\"=\"*50)\n",
    "for name, data in scaling_results.items():\n",
    "    converged = \"Sí\" if data['J_hist'][-1] < data['J_hist'][0] * 0.01 else \"Parcial\"\n",
    "    print(f\"  {name:<16}{data['mse']:<15.2f}{converged}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# 5. Integración: Pipeline Completo\n",
    "\n",
    "## 5.1 Ejercicio Final\n",
    "\n",
    "Combina todo lo aprendido. Usando el dataset original (`x_train`, `y_train`):\n",
    "\n",
    "1. Crea features polinomiales del grado óptimo (el que encontraste en la sección 3)\n",
    "2. Aplica el mejor método de scaling (el que encontraste en la sección 4)\n",
    "3. Entrena el modelo\n",
    "4. Grafica la predicción vs los datos reales\n",
    "5. Reporta el MSE final"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Iter     0: Cost = 26237.9740\n",
      "  Iter  1000: Cost = 7.2174\n",
      "  Iter  2000: Cost = 7.2173\n",
      "  Iter  3000: Cost = 7.2171\n",
      "  Iter  4000: Cost = 7.2170\n",
      "MSE final del pipeline: 14.4338\n"
     ]
    }
   ],
   "source": [
    "# ============================================\n",
    "# PIPELINE COMPLETO\n",
    "# ============================================\n",
    "\n",
    "# Paso 1: Elegir el mejor grado (basado en sección 3)\n",
    "best_degree = 3  # grado óptimo: mejor equilibrio entre MSE bajo y no overfitting\n",
    "\n",
    "# Paso 2: Crear features polinomiales [x, x², x³]\n",
    "X_final = create_polynomial_features(x_train, best_degree)\n",
    "\n",
    "# Paso 3: Aplicar z-score normalization (mejor método de sección 4)\n",
    "mu = X_final.mean(axis=0)      # media de cada columna\n",
    "sigma = X_final.std(axis=0)    # std de cada columna\n",
    "X_final_norm = (X_final - mu) / sigma\n",
    "\n",
    "# Paso 4: Entrenar con gradient descent\n",
    "w_fin, b_fin, J_fin = gradient_descent(X_final_norm, y_train,\n",
    "                                        np.zeros(best_degree), 0.0, 0.1, 5000)\n",
    "\n",
    "# Paso 5: Calcular MSE final\n",
    "preds = X_final_norm @ w_fin + b_fin\n",
    "mse_final = np.mean((preds - y_train)**2)\n",
    "\n",
    "print(f\"MSE final del pipeline: {mse_final:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>\n",
    "<summary><span style=\"color: green; font-weight: bold;\">💡 Pista — Pipeline completo</span></summary>\n",
    "\n",
    "```python\n",
    "best_degree = 2  # o el que hayas encontrado\n",
    "\n",
    "# Features polinomiales\n",
    "X_final = create_polynomial_features(x_train, best_degree)\n",
    "\n",
    "# Scaling (z-score)\n",
    "mu = X_final.mean(axis=0)\n",
    "sigma = X_final.std(axis=0)\n",
    "X_final_norm = (X_final - mu) / sigma\n",
    "\n",
    "# Entrenar\n",
    "w_fin, b_fin, J_fin = gradient_descent(X_final_norm, y_train, \n",
    "                                        np.zeros(best_degree), 0.0, 0.1, 5000)\n",
    "\n",
    "# MSE\n",
    "preds = X_final_norm @ w_fin + b_fin\n",
    "mse_final = np.mean((preds - y_train)**2)\n",
    "```\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 48468 characters]",
      "text/plain": [
       "<Figure size 900x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Visualización del resultado final (ejecuta después de completar el pipeline)\n",
    "if mse_final is not None:\n",
    "    plt.figure(figsize=(9, 5))\n",
    "    plt.scatter(x_train, y_train, color='royalblue', s=80, zorder=5, label='Datos reales')\n",
    "    \n",
    "    # Curva de predicción\n",
    "    x_plot = np.linspace(0.3, 3.7, 200)\n",
    "    X_plot_poly = create_polynomial_features(x_plot, best_degree)\n",
    "    X_plot_norm = (X_plot_poly - mu) / sigma\n",
    "    y_plot = X_plot_norm @ w_fin + b_fin\n",
    "    \n",
    "    plt.plot(x_plot, y_plot, 'r-', linewidth=2.5, label=f'Modelo (grado {best_degree})')\n",
    "    plt.xlabel('Tamaño (1000 sqft)')\n",
    "    plt.ylabel('Precio ($1000s)')\n",
    "    plt.title(f'Pipeline Final — Grado {best_degree}, MSE = {mse_final:.2f}')\n",
    "    plt.legend()\n",
    "    plt.grid(True, alpha=0.3)\n",
    "    plt.show()\n",
    "else:\n",
    "    print(\"⚠️ Completa el pipeline arriba primero.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# 6. Bonus: Verificar con sklearn\n",
    "\n",
    "Compara tu implementación manual con la de scikit-learn. Los resultados deben ser similares."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MSE sklearn:    15.4762\n",
      "MSE tu código:  14.4338\n",
      "\n",
      "✓ Resultados similares!\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 49100 characters]",
      "text/plain": [
       "<Figure size 900x500 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Nota: sklearn usa la solución analítica (Normal Equation), no gradient descent.\n",
      "Por eso converge al óptimo exacto. Tu GD se acerca con suficientes iteraciones.\n"
     ]
    }
   ],
   "source": [
    "from sklearn.preprocessing import PolynomialFeatures, StandardScaler\n",
    "from sklearn.linear_model import LinearRegression\n",
    "from sklearn.pipeline import Pipeline\n",
    "\n",
    "# Pipeline de sklearn equivalente a tu implementación manual\n",
    "degree_to_test = 2  # usa el mismo grado que encontraste óptimo\n",
    "\n",
    "pipe = Pipeline([\n",
    "    ('poly', PolynomialFeatures(degree_to_test, include_bias=False)),\n",
    "    ('scaler', StandardScaler()),\n",
    "    ('reg', LinearRegression())\n",
    "])\n",
    "\n",
    "# Entrenar\n",
    "pipe.fit(x_train.reshape(-1, 1), y_train)\n",
    "\n",
    "# Predecir\n",
    "y_pred_sklearn = pipe.predict(x_train.reshape(-1, 1))\n",
    "mse_sklearn = np.mean((y_pred_sklearn - y_train)**2)\n",
    "\n",
    "print(f\"MSE sklearn:    {mse_sklearn:.4f}\")\n",
    "if mse_final is not None:\n",
    "    print(f\"MSE tu código:  {mse_final:.4f}\")\n",
    "    print(f\"\\n{'✓ Resultados similares!' if abs(mse_sklearn - mse_final) < 50 else '⚠️ Hay diferencia significativa — revisa tu implementación'}\")\n",
    "else:\n",
    "    print(\"(Completa la sección 5 para comparar)\")\n",
    "\n",
    "# Visualizar\n",
    "plt.figure(figsize=(9, 5))\n",
    "plt.scatter(x_train, y_train, color='royalblue', s=80, zorder=5, label='Datos reales')\n",
    "\n",
    "x_plot = np.linspace(0.3, 3.7, 200)\n",
    "y_plot_sk = pipe.predict(x_plot.reshape(-1, 1))\n",
    "plt.plot(x_plot, y_plot_sk, 'g-', linewidth=2.5, label=f'sklearn (grado {degree_to_test})')\n",
    "\n",
    "plt.xlabel('Tamaño (1000 sqft)')\n",
    "plt.ylabel('Precio ($1000s)')\n",
    "plt.title(f'Verificación con sklearn — MSE: {mse_sklearn:.2f}')\n",
    "plt.legend()\n",
    "plt.grid(True, alpha=0.3)\n",
    "plt.show()\n",
    "\n",
    "print(\"\\nNota: sklearn usa la solución analítica (Normal Equation), no gradient descent.\")\n",
    "print(\"Por eso converge al óptimo exacto. Tu GD se acerca con suficientes iteraciones.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# 7. Conclusión\n",
    "\n",
    "## ¡Felicitaciones!\n",
    "\n",
    "En esta práctica aprendiste a:\n",
    "\n",
    "- **Vectorizar** operaciones usando el producto punto y multiplicación de matrices (`X @ w + b`) en lugar de loops\n",
    "- Crear **features polinomiales** para capturar relaciones no-lineales, y evaluar qué grado es óptimo\n",
    "- Implementar **3 métodos de feature scaling** (Min-Max, Mean Normalization, Z-Score) y comparar su efecto en la convergencia\n",
    "- Construir un **pipeline completo**: features → scaling → entrenamiento → evaluación\n",
    "\n",
    "### Conceptos Clave:\n",
    "1. La multiplicación matriz × vector es una colección de productos punto (una predicción por fila)\n",
    "2. Feature scaling es **esencial** cuando las features tienen escalas diferentes\n",
    "3. Más features (mayor grado) no siempre es mejor → riesgo de **overfitting**\n",
    "4. El pipeline correcto es: crear features → escalar → entrenar"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
