{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "FA46Pv6apDow",
   "metadata": {
    "id": "FA46Pv6apDow"
   },
   "source": [
    "# Laboratorio LSTM para Predicción de Palabras en PyTorch\n",
    "\n",
    "En este laboratorio construiremos un modelo LSTM para predecir la siguiente palabra dada una secuencia de palabras.\n",
    "Usaremos un corpus pequeño para facilitar la comprensión.\n",
    "\n",
    "## Objetivos\n",
    "- Preparar datos para modelar secuencias\n",
    "- Implementar un modelo LSTM en PyTorch\n",
    "- Entrenar el modelo para predecir la siguiente palabra\n",
    "- Evaluar el modelo\n",
    "\n",
    "## Instrucciones Generales\n",
    "\n",
    "1. Completar las partes marcadas con `# COMPLETAR`\n",
    "2. Este laboratorio sera realizado de manera individual\n",
    "3. **Formato de Entrega:** Su solución debe subirla en un archivo ZIP enviado por GES y debe contener el archivo .ipynb con sus respuestas a cada uno de los incisos que se necesitan `# COMPLETAR`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdXnzzlXpDo0",
   "metadata": {
    "id": "bdXnzzlXpDo0"
   },
   "source": [
    "## Paso 1: Corpus y tokenización"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7lG9qusMpDo1",
   "metadata": {
    "id": "7lG9qusMpDo1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tokens: ['el', 'gato', 'come', 'pescado', 'el', 'perro', 'come', 'hueso', 'el', 'gato', 'juega', 'con', 'el', 'perro', 'el', 'pez', 'nada', 'en', 'el', 'agua', 'el', 'perro', 'corre', 'rápido']\n"
     ]
    }
   ],
   "source": [
    "# Corpus simple\n",
    "corpus = [\n",
    "    \"el gato come pescado\",\n",
    "    \"el perro come hueso\",\n",
    "    \"el gato juega con el perro\",\n",
    "    \"el pez nada en el agua\",\n",
    "    \"el perro corre rápido\"\n",
    "]\n",
    "\n",
    "def tokenize_corpus(corpus):\n",
    "    tokens = []\n",
    "    for sentence in corpus:\n",
    "        tokens.extend(sentence.lower().split())\n",
    "    return tokens\n",
    "\n",
    "tokens = tokenize_corpus(corpus)\n",
    "print(\"Tokens:\", tokens)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "QdIyZmm7pDo3",
   "metadata": {
    "id": "QdIyZmm7pDo3"
   },
   "source": [
    "## Paso 2: Crear vocabulario e índices"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "Wxl8oFqEpDo3",
   "metadata": {
    "id": "Wxl8oFqEpDo3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vocabulario: ['agua', 'come', 'con', 'corre', 'el', 'en', 'gato', 'hueso', 'juega', 'nada', 'perro', 'pescado', 'pez', 'rápido']\n"
     ]
    }
   ],
   "source": [
    "vocab = sorted(set(tokens))\n",
    "word_to_idx = {word: i for i, word in enumerate(vocab)}\n",
    "idx_to_word = {i: word for word, i in word_to_idx.items()}\n",
    "\n",
    "print(\"Vocabulario:\", vocab)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "PNZK2ruxpDo4",
   "metadata": {
    "id": "PNZK2ruxpDo4"
   },
   "source": [
    "## Paso 3: Crear dataset de secuencias para predecir la siguiente palabra\n",
    "\n",
    "Con un tamaño de secuencia fijo (por ejemplo 3), crearemos pares (entrada, objetivo) donde la entrada es una secuencia de palabras y el objetivo es la palabra siguiente."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "YU7iNATepDo4",
   "metadata": {
    "id": "YU7iNATepDo4"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ejemplo de entrada (índices): [4, 6, 1]\n",
      "Ejemplo de objetivo (índice): 11\n"
     ]
    }
   ],
   "source": [
    "def create_sequences(tokens, seq_length):\n",
    "    inputs = []\n",
    "    targets = []\n",
    "    for i in range(len(tokens) - seq_length):\n",
    "        seq_in = tokens[i:i+seq_length]\n",
    "        seq_out = tokens[i+seq_length]\n",
    "        inputs.append([word_to_idx[w] for w in seq_in])\n",
    "        targets.append(word_to_idx[seq_out])\n",
    "    return inputs, targets\n",
    "\n",
    "seq_length = 3\n",
    "inputs, targets = create_sequences(tokens, seq_length)\n",
    "\n",
    "print(\"Ejemplo de entrada (índices):\", inputs[0])\n",
    "print(\"Ejemplo de objetivo (índice):\", targets[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "HeeOssQNpDo4",
   "metadata": {
    "id": "HeeOssQNpDo4"
   },
   "source": [
    "## Paso 4: Definir el modelo LSTM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "bZqzjashpDo5",
   "metadata": {
    "id": "bZqzjashpDo5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LSTMWordPredictor(\n",
      "  (embedding): Embedding(14, 10)\n",
      "  (lstm): LSTM(10, 20, batch_first=True)\n",
      "  (linear): Linear(in_features=20, out_features=14, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "import torch.nn as nn\n",
    "\n",
    "class LSTMWordPredictor(nn.Module):\n",
    "    def __init__(self, vocab_size, embedding_dim, hidden_dim):\n",
    "        super(LSTMWordPredictor, self).__init__()\n",
    "        self.embedding = nn.Embedding(vocab_size, embedding_dim)\n",
    "        self.lstm = nn.LSTM(embedding_dim, hidden_dim, batch_first=True)\n",
    "        self.linear = nn.Linear(hidden_dim, vocab_size)\n",
    "\n",
    "    def forward(self, x):\n",
    "        embeds = self.embedding(x)  # x shape: (batch, seq_length)\n",
    "        lstm_out, _ = self.lstm(embeds)  # lstm_out shape: (batch, seq_length, hidden_dim)\n",
    "        # COMPLETAR: Tomar la salida del último timestep\n",
    "        last_out = lstm_out[:, -1, :]  # COMPLETAR\n",
    "        out = self.linear(last_out)  # out shape: (batch, vocab_size)\n",
    "        return out\n",
    "\n",
    "embedding_dim = 10\n",
    "hidden_dim = 20\n",
    "vocab_size = len(vocab)\n",
    "model = LSTMWordPredictor(vocab_size, embedding_dim, hidden_dim)\n",
    "print(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "_ykl-dihpDo5",
   "metadata": {
    "id": "_ykl-dihpDo5"
   },
   "source": [
    "## Paso 5: Entrenamiento del modelo"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "AZDWmoBXpDo6",
   "metadata": {
    "id": "AZDWmoBXpDo6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 20, Loss: 0.0294\n",
      "Epoch 40, Loss: 0.0075\n",
      "Epoch 60, Loss: 0.0035\n",
      "Epoch 80, Loss: 0.0020\n",
      "Epoch 100, Loss: 0.0013\n"
     ]
    }
   ],
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "def train(model, inputs, targets, epochs=100, lr=0.01):\n",
    "    loss_function = nn.CrossEntropyLoss()\n",
    "    optimizer = optim.Adam(model.parameters(), lr=lr)\n",
    "\n",
    "    for epoch in range(epochs):\n",
    "        total_loss = 0\n",
    "        for i in range(len(inputs)):\n",
    "            input_seq = torch.tensor([inputs[i]], dtype=torch.long)  # batch_size=1\n",
    "            target = torch.tensor([targets[i]], dtype=torch.long)\n",
    "\n",
    "            model.zero_grad()\n",
    "            output = model(input_seq)\n",
    "            loss = loss_function(output, target)\n",
    "            loss.backward()\n",
    "            optimizer.step()\n",
    "            total_loss += loss.item()\n",
    "        if (epoch+1) % 20 == 0:\n",
    "            print(f\"Epoch {epoch+1}, Loss: {total_loss/len(inputs):.4f}\")\n",
    "\n",
    "train(model, inputs, targets)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "mMC-G_8bpDo6",
   "metadata": {
    "id": "mMC-G_8bpDo6"
   },
   "source": [
    "## Paso 6: Función para predecir la siguiente palabra dada una secuencia"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "r8HT4xnLpDo6",
   "metadata": {
    "id": "r8HT4xnLpDo6"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dada la secuencia ['el', 'gato', 'come'], la siguiente palabra predicha es: pescado\n"
     ]
    }
   ],
   "source": [
    "def predict_next_word(model, input_words, word_to_idx, idx_to_word):\n",
    "    model.eval()\n",
    "    with torch.no_grad():\n",
    "        input_idx = torch.tensor([[word_to_idx[w] for w in input_words]], dtype=torch.long)\n",
    "        output = model(input_idx)\n",
    "        predicted_idx = output.argmax(dim=1).item()\n",
    "        return idx_to_word[predicted_idx]\n",
    "\n",
    "# Prueba\n",
    "input_seq = [\"el\", \"gato\", \"come\"]\n",
    "print(f\"Dada la secuencia {input_seq}, la siguiente palabra predicha es: {predict_next_word(model, input_seq, word_to_idx, idx_to_word)}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "SxtYrqIErt3w",
   "metadata": {
    "id": "SxtYrqIErt3w"
   },
   "source": [
    "## ¡Felicidades!\n",
    "\n",
    "Has implementado un modelo LSTM para predecir la siguiente palabra dada una secuencia de palabras"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}