Adding Lab2

2026-08-27 10:43:58 -06:00
parent b930ae3f49
commit 1687cc63a0
2 changed files with 344 additions and 0 deletions
@@ -0,0 +1,343 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "FA46Pv6apDow",
+   "metadata": {
+    "id": "FA46Pv6apDow"
+   },
+   "source": [
+    "# Laboratorio LSTM para Predicción de Palabras en PyTorch\n",
+    "\n",
+    "En este laboratorio construiremos un modelo LSTM para predecir la siguiente palabra dada una secuencia de palabras.\n",
+    "Usaremos un corpus pequeño para facilitar la comprensión.\n",
+    "\n",
+    "## Objetivos\n",
+    "- Preparar datos para modelar secuencias\n",
+    "- Implementar un modelo LSTM en PyTorch\n",
+    "- Entrenar el modelo para predecir la siguiente palabra\n",
+    "- Evaluar el modelo\n",
+    "\n",
+    "## Instrucciones Generales\n",
+    "\n",
+    "1. Completar las partes marcadas con `# COMPLETAR`\n",
+    "2. Este laboratorio sera realizado de manera individual\n",
+    "3. **Formato de Entrega:** Su solución debe subirla en un archivo ZIP enviado por GES y debe contener el archivo .ipynb con sus respuestas a cada uno de los incisos que se necesitan `# COMPLETAR`."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "bdXnzzlXpDo0",
+   "metadata": {
+    "id": "bdXnzzlXpDo0"
+   },
+   "source": [
+    "## Paso 1: Corpus y tokenización"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "7lG9qusMpDo1",
+   "metadata": {
+    "id": "7lG9qusMpDo1"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Tokens: ['el', 'gato', 'come', 'pescado', 'el', 'perro', 'come', 'hueso', 'el', 'gato', 'juega', 'con', 'el', 'perro', 'el', 'pez', 'nada', 'en', 'el', 'agua', 'el', 'perro', 'corre', 'rápido']\n"
+     ]
+    }
+   ],
+   "source": [
+    "# Corpus simple\n",
+    "corpus = [\n",
+    "    \"el gato come pescado\",\n",
+    "    \"el perro come hueso\",\n",
+    "    \"el gato juega con el perro\",\n",
+    "    \"el pez nada en el agua\",\n",
+    "    \"el perro corre rápido\"\n",
+    "]\n",
+    "\n",
+    "def tokenize_corpus(corpus):\n",
+    "    tokens = []\n",
+    "    for sentence in corpus:\n",
+    "        tokens.extend(sentence.lower().split())\n",
+    "    return tokens\n",
+    "\n",
+    "tokens = tokenize_corpus(corpus)\n",
+    "print(\"Tokens:\", tokens)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "QdIyZmm7pDo3",
+   "metadata": {
+    "id": "QdIyZmm7pDo3"
+   },
+   "source": [
+    "## Paso 2: Crear vocabulario e índices"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "Wxl8oFqEpDo3",
+   "metadata": {
+    "id": "Wxl8oFqEpDo3"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Vocabulario: ['agua', 'come', 'con', 'corre', 'el', 'en', 'gato', 'hueso', 'juega', 'nada', 'perro', 'pescado', 'pez', 'rápido']\n"
+     ]
+    }
+   ],
+   "source": [
+    "vocab = sorted(set(tokens))\n",
+    "word_to_idx = {word: i for i, word in enumerate(vocab)}\n",
+    "idx_to_word = {i: word for word, i in word_to_idx.items()}\n",
+    "\n",
+    "print(\"Vocabulario:\", vocab)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "PNZK2ruxpDo4",
+   "metadata": {
+    "id": "PNZK2ruxpDo4"
+   },
+   "source": [
+    "## Paso 3: Crear dataset de secuencias para predecir la siguiente palabra\n",
+    "\n",
+    "Con un tamaño de secuencia fijo (por ejemplo 3), crearemos pares (entrada, objetivo) donde la entrada es una secuencia de palabras y el objetivo es la palabra siguiente."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "id": "YU7iNATepDo4",
+   "metadata": {
+    "id": "YU7iNATepDo4"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Ejemplo de entrada (índices): [4, 6, 1]\n",
+      "Ejemplo de objetivo (índice): 11\n"
+     ]
+    }
+   ],
+   "source": [
+    "def create_sequences(tokens, seq_length):\n",
+    "    inputs = []\n",
+    "    targets = []\n",
+    "    for i in range(len(tokens) - seq_length):\n",
+    "        seq_in = tokens[i:i+seq_length]\n",
+    "        seq_out = tokens[i+seq_length]\n",
+    "        inputs.append([word_to_idx[w] for w in seq_in])\n",
+    "        targets.append(word_to_idx[seq_out])\n",
+    "    return inputs, targets\n",
+    "\n",
+    "seq_length = 3\n",
+    "inputs, targets = create_sequences(tokens, seq_length)\n",
+    "\n",
+    "print(\"Ejemplo de entrada (índices):\", inputs[0])\n",
+    "print(\"Ejemplo de objetivo (índice):\", targets[0])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "HeeOssQNpDo4",
+   "metadata": {
+    "id": "HeeOssQNpDo4"
+   },
+   "source": [
+    "## Paso 4: Definir el modelo LSTM"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "id": "bZqzjashpDo5",
+   "metadata": {
+    "id": "bZqzjashpDo5"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "LSTMWordPredictor(\n",
+      "  (embedding): Embedding(14, 10)\n",
+      "  (lstm): LSTM(10, 20, batch_first=True)\n",
+      "  (linear): Linear(in_features=20, out_features=14, bias=True)\n",
+      ")\n"
+     ]
+    }
+   ],
+   "source": [
+    "import torch\n",
+    "import torch.nn as nn\n",
+    "\n",
+    "class LSTMWordPredictor(nn.Module):\n",
+    "    def __init__(self, vocab_size, embedding_dim, hidden_dim):\n",
+    "        super(LSTMWordPredictor, self).__init__()\n",
+    "        self.embedding = nn.Embedding(vocab_size, embedding_dim)\n",
+    "        self.lstm = nn.LSTM(embedding_dim, hidden_dim, batch_first=True)\n",
+    "        self.linear = nn.Linear(hidden_dim, vocab_size)\n",
+    "\n",
+    "    def forward(self, x):\n",
+    "        embeds = self.embedding(x)  # x shape: (batch, seq_length)\n",
+    "        lstm_out, _ = self.lstm(embeds)  # lstm_out shape: (batch, seq_length, hidden_dim)\n",
+    "        # COMPLETAR: Tomar la salida del último timestep\n",
+    "        last_out = lstm_out[:, -1, :]  # COMPLETAR\n",
+    "        out = self.linear(last_out)  # out shape: (batch, vocab_size)\n",
+    "        return out\n",
+    "\n",
+    "embedding_dim = 10\n",
+    "hidden_dim = 20\n",
+    "vocab_size = len(vocab)\n",
+    "model = LSTMWordPredictor(vocab_size, embedding_dim, hidden_dim)\n",
+    "print(model)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "_ykl-dihpDo5",
+   "metadata": {
+    "id": "_ykl-dihpDo5"
+   },
+   "source": [
+    "## Paso 5: Entrenamiento del modelo"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "id": "AZDWmoBXpDo6",
+   "metadata": {
+    "id": "AZDWmoBXpDo6"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Epoch 20, Loss: 0.0294\n",
+      "Epoch 40, Loss: 0.0075\n",
+      "Epoch 60, Loss: 0.0035\n",
+      "Epoch 80, Loss: 0.0020\n",
+      "Epoch 100, Loss: 0.0013\n"
+     ]
+    }
+   ],
+   "source": [
+    "import torch.optim as optim\n",
+    "\n",
+    "def train(model, inputs, targets, epochs=100, lr=0.01):\n",
+    "    loss_function = nn.CrossEntropyLoss()\n",
+    "    optimizer = optim.Adam(model.parameters(), lr=lr)\n",
+    "\n",
+    "    for epoch in range(epochs):\n",
+    "        total_loss = 0\n",
+    "        for i in range(len(inputs)):\n",
+    "            input_seq = torch.tensor([inputs[i]], dtype=torch.long)  # batch_size=1\n",
+    "            target = torch.tensor([targets[i]], dtype=torch.long)\n",
+    "\n",
+    "            model.zero_grad()\n",
+    "            output = model(input_seq)\n",
+    "            loss = loss_function(output, target)\n",
+    "            loss.backward()\n",
+    "            optimizer.step()\n",
+    "            total_loss += loss.item()\n",
+    "        if (epoch+1) % 20 == 0:\n",
+    "            print(f\"Epoch {epoch+1}, Loss: {total_loss/len(inputs):.4f}\")\n",
+    "\n",
+    "train(model, inputs, targets)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "mMC-G_8bpDo6",
+   "metadata": {
+    "id": "mMC-G_8bpDo6"
+   },
+   "source": [
+    "## Paso 6: Función para predecir la siguiente palabra dada una secuencia"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "r8HT4xnLpDo6",
+   "metadata": {
+    "id": "r8HT4xnLpDo6"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Dada la secuencia ['el', 'gato', 'come'], la siguiente palabra predicha es: pescado\n"
+     ]
+    }
+   ],
+   "source": [
+    "def predict_next_word(model, input_words, word_to_idx, idx_to_word):\n",
+    "    model.eval()\n",
+    "    with torch.no_grad():\n",
+    "        input_idx = torch.tensor([[word_to_idx[w] for w in input_words]], dtype=torch.long)\n",
+    "        output = model(input_idx)\n",
+    "        predicted_idx = output.argmax(dim=1).item()\n",
+    "        return idx_to_word[predicted_idx]\n",
+    "\n",
+    "# Prueba\n",
+    "input_seq = [\"el\", \"gato\", \"come\"]\n",
+    "print(f\"Dada la secuencia {input_seq}, la siguiente palabra predicha es: {predict_next_word(model, input_seq, word_to_idx, idx_to_word)}\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "SxtYrqIErt3w",
+   "metadata": {
+    "id": "SxtYrqIErt3w"
+   },
+   "source": [
+    "## ¡Felicidades!\n",
+    "\n",
+    "Has implementado un modelo LSTM para predecir la siguiente palabra dada una secuencia de palabras"
+   ]
+  }
+ ],
+ "metadata": {
+  "colab": {
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.3"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}
@@ -7,3 +7,4 @@ Repositorio con los laboratorios de la asignatura.
 | # | Laboratorio | Notebook |
 |---|---|---|
 | 1 | Word2Vec desde cero (Skip-gram, CBOW, gensim, PyTorch) | [Lab 1 - Word2Vec](Labs/word2vec_Alejandro_Lembke.ipynb) |
+| 2 | LSTM para predicción de la siguiente palabra (PyTorch) | [Lab 2 - LSTM](Labs/Laboratorio_lstm_Alejandro_Lembke.ipynb) |