{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "title-and-goals",
   "metadata": {},
   "source": [
    "# 用 Jupyter + LLM 学懂梯度下降\n",
    "\n",
    "这个 Notebook 不追求把代码写得多复杂，而是走完一次 **问、猜、跑、看、讲、测** 的学习闭环。\n",
    "\n",
    "学习目标：\n",
    "\n",
    "1. 解释梯度下降公式中每个符号的含义；\n",
    "2. 观察单调收敛、振荡收敛和发散；\n",
    "3. 从具体函数推导学习率的收敛范围；\n",
    "4. 可选地调用 OpenAI-compatible LLM API，让 AI 充当追问型学习教练。\n",
    "\n",
    "> 安全提醒：不要把 API key 直接写进 Notebook。代码只从环境变量读取密钥，默认也不会发送任何 API 请求。不要把公司机密、个人隐私、未公开代码或无权交给第三方处理的材料放进提示词。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "setup",
   "metadata": {},
   "source": [
    "## 0. 准备环境\n",
    "\n",
    "如果环境里缺少依赖，可以在一个新 Cell 中运行：\n",
    "\n",
    "```python\n",
    "%pip install numpy matplotlib openai\n",
    "```\n",
    "\n",
    "调用 LLM 前，在启动 Jupyter 的终端中配置环境变量：\n",
    "\n",
    "```bash\n",
    "export LLM_API_KEY=your-api-key\n",
    "export LLM_BASE_URL=https://your-openai-compatible-endpoint/v1\n",
    "export LLM_MODEL=your-model-name\n",
    "```\n",
    "\n",
    "也可以使用 `OPENAI_API_KEY` 和 `OPENAI_BASE_URL`。不要在 Notebook 中打印密钥。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "imports",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "\n",
    "plt.rcParams[\"figure.figsize\"] = (12, 6)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "formula-symbols",
   "metadata": {},
   "source": [
    "## 1. 先看懂公式\n",
    "\n",
    "梯度下降的更新公式是：\n",
    "\n",
    "$$\n",
    "x_{t+1}=x_t-\\eta\\nabla f(x_t)\n",
    "$$\n",
    "\n",
    "- $f(x)$：想要最小化的目标函数或损失函数；\n",
    "- $x_t$：第 $t$ 步的当前参数；\n",
    "- $x_{t+1}$：更新后的参数；\n",
    "- $\\nabla f(x_t)$：当前位置的梯度，指向函数上升最快的方向；\n",
    "- $\\eta$：学习率，控制每次迈多大一步；\n",
    "- 负号：让参数沿梯度的反方向，也就是下坡方向移动。\n",
    "\n",
    "本实验使用：\n",
    "\n",
    "$$\n",
    "f(x)=(x-3)^2+2,\\qquad \\nabla f(x)=2(x-3)\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "prediction",
   "metadata": {},
   "source": [
    "## 2. 先猜，再运行\n",
    "\n",
    "请先填写，不要急着执行后面的图：\n",
    "\n",
    "1. 最低点大约在 $x=$ ______。\n",
    "2. $\\eta=0.1$ 时，轨迹会 ______。\n",
    "3. $\\eta=0.8$ 时，轨迹会 ______。\n",
    "4. $\\eta=1.05$ 时，轨迹会 ______。\n",
    "5. 我目前的信心：______ / 5。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "gradient-functions",
   "metadata": {},
   "outputs": [],
   "source": [
    "def f(x):\n",
    "    return (x - 3) ** 2 + 2\n",
    "\n",
    "\n",
    "def grad(x):\n",
    "    return 2 * (x - 3)\n",
    "\n",
    "\n",
    "def gradient_descent(x0, eta, steps=20):\n",
    "    xs = [float(x0)]\n",
    "    for _ in range(steps):\n",
    "        xs.append(xs[-1] - eta * grad(xs[-1]))\n",
    "    return np.array(xs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "first-step",
   "metadata": {},
   "outputs": [],
   "source": [
    "x0 = -6.0\n",
    "eta = 0.1\n",
    "gradient = grad(x0)\n",
    "x1 = x0 - eta * gradient\n",
    "\n",
    "print(f\"x0 = {x0}\")\n",
    "print(f\"gradient = {gradient}\")\n",
    "print(f\"x1 = {x1}\")\n",
    "assert np.isclose(x1, -4.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "visualize-rates",
   "metadata": {},
   "outputs": [],
   "source": [
    "rates = [0.1, 0.8, 1.05]\n",
    "fig, axes = plt.subplots(2, 3, figsize=(15, 8))\n",
    "\n",
    "for col, eta in enumerate(rates):\n",
    "    xs = gradient_descent(x0=-6, eta=eta)\n",
    "    left = min(-8, xs.min() - 1)\n",
    "    right = max(12, xs.max() + 1)\n",
    "    grid = np.linspace(left, right, 400)\n",
    "\n",
    "    path_ax = axes[0, col]\n",
    "    path_ax.plot(grid, f(grid), color=\"steelblue\")\n",
    "    path_ax.scatter(xs, f(xs), c=np.arange(len(xs)), cmap=\"autumn\", s=28)\n",
    "    path_ax.plot(xs, f(xs), color=\"gray\", alpha=0.5)\n",
    "    path_ax.set_title(f\"eta = {eta}\")\n",
    "    path_ax.set_xlabel(\"x\")\n",
    "    path_ax.grid(alpha=0.25)\n",
    "\n",
    "    error_ax = axes[1, col]\n",
    "    error_ax.plot(np.arange(len(xs)), np.abs(xs - 3), marker=\"o\")\n",
    "    error_ax.set_yscale(\"log\")\n",
    "    error_ax.set_xlabel(\"iteration\")\n",
    "    error_ax.grid(alpha=0.25)\n",
    "\n",
    "axes[0, 0].set_ylabel(\"f(x)\")\n",
    "axes[1, 0].set_ylabel(\"|x - 3|\")\n",
    "plt.tight_layout()\n",
    "plt.show()\n",
    "\n",
    "assert abs(gradient_descent(-6, 0.1)[-1] - 3) < 0.11\n",
    "assert abs(gradient_descent(-6, 0.8)[-1] - 3) < 0.001\n",
    "assert abs(gradient_descent(-6, 1.05)[-1] - 3) > 9"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "observations",
   "metadata": {},
   "source": [
    "## 3. 记录观察和解释\n",
    "\n",
    "先写观察，再写解释，不要把两者混在一起。\n",
    "\n",
    "- **观察**：$\\eta=0.1$ 时，我看到 ______。\n",
    "- **解释**：因为每轮误差会 ______。\n",
    "- **观察**：$\\eta=0.8$ 时，我看到 ______。\n",
    "- **解释**：负号意味着 ______，绝对值小于 1 意味着 ______。\n",
    "- **观察**：$\\eta=1.05$ 时，我看到 ______。\n",
    "- **解释**：它越跑越远，是因为 ______。\n",
    "\n",
    "对这个特定函数：\n",
    "\n",
    "$$\n",
    "x_{t+1}-3=(1-2\\eta)(x_t-3)\n",
    "$$\n",
    "\n",
    "要让误差逐步缩小，需要 $|1-2\\eta|<1$，因此 $0<\\eta<1$。这个范围只属于当前函数和当前假设，不是梯度下降的万能学习率。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "llm-coach-intro",
   "metadata": {},
   "source": [
    "## 4. 可选：让 LLM 充当学习陪练\n",
    "\n",
    "下面使用兼容 OpenAI Chat Completions 的 API。为了避免误调用，`ENABLE_LLM_CALL` 默认为 `False`。\n",
    "\n",
    "启用前请确认：\n",
    "\n",
    "- API 地址、模型名称和计费规则正确；\n",
    "- 提示词中没有秘密、个人数据或受限制材料；\n",
    "- 你允许这些内容被发送给对应的服务提供者。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "llm-client",
   "metadata": {},
   "outputs": [],
   "source": [
    "ENABLE_LLM_CALL = False  # 确认配置和数据边界后，手动改成 True\n",
    "\n",
    "api_key = os.getenv(\"LLM_API_KEY\") or os.getenv(\"OPENAI_API_KEY\")\n",
    "base_url = os.getenv(\"LLM_BASE_URL\") or os.getenv(\"OPENAI_BASE_URL\")\n",
    "model = os.getenv(\"LLM_MODEL\", \"gpt-4o-mini\")\n",
    "client = None\n",
    "\n",
    "if ENABLE_LLM_CALL:\n",
    "    if not api_key:\n",
    "        raise RuntimeError(\"请先设置 LLM_API_KEY 或 OPENAI_API_KEY\")\n",
    "\n",
    "    from openai import OpenAI\n",
    "\n",
    "    client_options = {\"api_key\": api_key, \"timeout\": 30.0}\n",
    "    if base_url:\n",
    "        client_options[\"base_url\"] = base_url\n",
    "    client = OpenAI(**client_options)\n",
    "    print(f\"LLM API enabled; model={model}\")\n",
    "else:\n",
    "    print(\"LLM API disabled; set ENABLE_LLM_CALL=True to enable it.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "llm-coach",
   "metadata": {},
   "outputs": [],
   "source": [
    "def ask_learning_coach(question, my_observation):\n",
    "    if client is None:\n",
    "        return \"LLM API 尚未启用。请先检查配置，再显式打开 ENABLE_LLM_CALL。\"\n",
    "\n",
    "    system_prompt = \"\"\"\n",
    "你是梯度下降学习陪练，不是代答工具。\n",
    "规则：\n",
    "1. 先判断学习者的观察是否具体；\n",
    "2. 一次只追问一个关键问题；\n",
    "3. 不直接给完整答案，优先给最小提示；\n",
    "4. 要求学习者把图像现象和公式中的误差倍率联系起来；\n",
    "5. 如果发现错误，明确指出错误发生在哪一步；\n",
    "6. 回复控制在 200 个汉字以内。\n",
    "\"\"\".strip()\n",
    "\n",
    "    user_prompt = f\"问题：{question}\\n我的观察：{my_observation}\"\n",
    "    response = client.chat.completions.create(\n",
    "        model=model,\n",
    "        messages=[\n",
    "            {\"role\": \"system\", \"content\": system_prompt},\n",
    "            {\"role\": \"user\", \"content\": user_prompt},\n",
    "        ],\n",
    "        temperature=0.2,\n",
    "        max_tokens=400,\n",
    "    )\n",
    "    return response.choices[0].message.content or \"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ask-coach",
   "metadata": {},
   "outputs": [],
   "source": [
    "question = \"为什么 eta=0.8 会左右振荡，却仍然收敛？\"\n",
    "my_observation = (\n",
    "    \"x 在最低点 3 的两边交替出现，\"\n",
    "    \"下排误差曲线持续下降，但我还说不清它与 1-2*eta 的关系。\"\n",
    ")\n",
    "\n",
    "print(ask_learning_coach(question, my_observation))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "closed-book-test",
   "metadata": {},
   "source": [
    "## 5. 闭卷验收\n",
    "\n",
    "关掉上面的答案，考虑新函数：\n",
    "\n",
    "$$\n",
    "g(x)=4(x+2)^2+1\n",
    "$$\n",
    "\n",
    "请先手算，再写代码验证：\n",
    "\n",
    "1. 最低点在哪里？\n",
    "2. 梯度是什么？\n",
    "3. 固定学习率满足什么范围时，误差会逐步缩小？\n",
    "4. 从最低点右侧出发，第一次更新往哪个方向走？\n",
    "5. 设计三个学习率，分别演示单调收敛、振荡收敛和发散。\n",
    "\n",
    "如果能解释边界、预测轨迹并用实验验证，这个公式才算真正开始属于你。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}