← Về thư mục
📄 / / root / ceo / company_kb / notebooklm_auth_guide.md

NotebookLM Auth Guide — SOP & Deep Analysis

File: company_kb/notebooklm_auth_guide.md
Target: notebooklm-mcp-cli integration at /opt/ai-os/products/ceo/integrations/notebooklm-mcp-cli/
Last updated: 2026-07-12

1. Auth Architecture (How It Works)

1.1 Data Flow

User runs `nlm notebook_list`
       │
       ▼
AuthManager.load_profile("default")
       │
       ├── reads ~/.notebooklm-mcp-cli/profiles/default/cookies.json
       ├── reads ~/.notebooklm-mcp-cli/profiles/default/metadata.json
       │
       ▼
BaseClient.__init__(cookies, csrf_token, session_id, build_label)
       │
       ├── _refresh_auth_tokens()
       │     └── fetch https://notebooklm.google.com/ with cookies
       │     └── if redirect to accounts.google.com → AUTH EXPIRED
       │     └── else → extract SNlM0e (CSRF), FdrFJe (session_id), cfb2h (build_label)
       │
       ▼
RPC call (batchexecute) with:
  - Cookie header (SID, HSID, SSID, APISID, SAPISID, etc.)
  - X-Goog-Csrf-Token header
  - ?f.sid=<session_id>&bl=<build_label> in URL

1.2 Storage Layout

~/.notebooklm-mcp-cli/
├── auth.json                         ← Legacy flat cache (auto-migrated to profile)
├── config.toml                       ← CLI config (auth.browser, default_profile)
├── chrome-port-map.json              ← Maps CDP ports → profile names
├── profiles/
│   └── default/
│       ├── cookies.json              ← Cookie dict/list (can be 50+ cookies)
│       └── metadata.json             ← csrf_token, session_id, email, build_label, last_validated
└── chrome-profiles/
    └── default/                      ← Chrome user-data-dir (real browser profile)
        ├── Default/Cookies           ← SQLite cookie DB (36KB+)
        ├── Default/Login Data        ← Saved logins
        └── ...

1.3 Key Components

Component File Role
AuthManager core/auth.py Profile CRUD + cookie serialization
BaseClient core/base.py HTTP client, batchexecute, CSRF auto-refresh
check_auth() core/auth.py Live auth validity check (homepage fetch + RPC probe)
rotate_google_cookies() core/cookie_rotation.py POST to accounts.google.com/RotateCookies to refresh short-lived *PSIDTS cookies
run_headless_auth() utils/cdp.py Launch headless Chrome, navigate to NotebookLM, extract cookies from persisted profile
extract_cookies_via_cdp() utils/cdp.py Full interactive CDP extraction (launches Chrome, waits for login)
extract_cookies_via_existing_cdp() utils/cdp.py Cookie extraction from already-running Chrome endpoint

Google auth uses layered cookies. The long-lived ones (SID, HSID, SSID, APISID, SAPISID) persist for weeks. The short-lived ones (__Secure-1PSIDTS, __Secure-3PSIDTS, SIDCC) rotate hourly/daily.

The cookie_rotation.py module calls the Google RotateCookies endpoint to refresh short-lived cookies before checking if auth is really expired. This is a non-fatal recovery step embedded in _refresh_auth_tokens().

2. Root Cause Analysis: Current Auth Failure

2.1 Observed State

2.2 Why It Happens

  1. The Chrome profile (Default/Cookies SQLite DB) has expired cookies that Google no longer accepts.
  2. When BaseClient._refresh_auth_tokens() fetches https://notebooklm.google.com/, the cookies redirect to accounts.google.com login page.
  3. The short-lived *PSIDTS cookies have rotated and the cached ones are stale.
  4. rotate_google_cookies() was called but failed because the underlying session (SID/HSID) is also stale — rotation only works when long-lived tokens are still valid.
  5. Since headless can't render the interactive login page, and the persisted session is expired, recovery loops fail.

2.3 Triggers for Auth Death

3. Optimal Strategy for Stable VPS Auth

The project supports two auth methods: 1. Browser/profile-based (recommended): Use nlm login to authenticate via Chrome → persisted profile → auto-refresh via headless CDP. 2. Manual cookie import (nlm login --manual): Copy-paste cookie headers. Fragile.

We use method 1 with a persistent Chrome headless daemon.

3.2 Architecture

┌─────────────────────────────────────────────────────────┐
│                    VPS (Headless)                        │
│                                                          │
│  Chrome headless ───port 9223─── CDP WebSocket           │
│  (always running)                 │                      │
│       │                            │                      │
│       ▼                            ▼                      │
│  Chrome profile              extract_cookies_via_cdp()    │
│  ~/.notebooklm-mcp-cli/      → profile cookies.json      │
│    chrome-profiles/default/  → profile metadata.json     │
│       │                                                   │
│       ▼                                                   │
│  nlm / MCP server reads disk profile                      │
│                                                          │
│  Auth Refresh Cron:                                      │
│  ┌─────────────────────────────────────┐                 │
│  │ Every 6h: check_auth(live=True)     │                 │
│  │ If expired:                         │                 │
│  │   1. Kill → relaunch Chrome headless│                 │
│  │   2. Run headless auth extraction   │                 │
│  │   3. If still dead → notify admin   │                 │
│  └─────────────────────────────────────┘                 │
└─────────────────────────────────────────────────────────┘
Action Frequency Who Notes
RotateCookies POST Every request BaseClient._refresh_auth_tokens() Non-fatal, best-effort
Auth validity check Every 6 hours Cron: auth_refresh.sh Homepage fetch + RPC probe via check_auth()
Full re-auth (headless) On auth failure Cron → revive_notebooklm_auth.sh Requires working Chrome profile
Interactive re-auth When headless fails User (via SSH tunnel) See SOP Section 5

3.4 Why This Beats Alternative Approaches

Approach Problem Verdict
Only cookie injection Cookies expire in hours-days, no rotation ❌ Fragile
nlm login every time Interactive, blocks automation ❌ Not automated
Persistent headless Chrome Chrome memory ~250MB but session persists via profile ✅ Best for VPS
Environment variables Can't rotate cookies, env vars lost on restart ❌ Not persistent
Chrome + CDP + profile Combines persistence with automation ✅ WINNER

4. SOP: Setting Up NotebookLM Auth on VPS

4.1 Initial Authentication (One-Time Setup)

Step 1: Verify Chrome is Running

# Check Chrome headless on port 9223
curl -s http://127.0.0.1:9223/json/version | python3 -m json.tool
# Expected: returns JSON with "Browser" and "webSocketDebuggerUrl"

If Chrome is not running, start it:

/opt/ai-os/products/ceo/scripts/revive_notebooklm_auth.sh

Step 2: Tunnel Chrome to Your Local Machine

On your local machine:

ssh -N -L 9223:127.0.0.1:9223 root@<VPS_IP>

Step 3: Authenticate via Local Browser

  1. Open Chrome on your local machine
  2. Go to chrome://inspect
  3. Click "Configure..." and add localhost:9223
  4. Find the NotebookLM tab (or open one in the remote VPS Chrome if needed)
  5. If you see the Google login page, log in with your Google account
  6. Complete any 2FA / verification challenges
  7. Verify you reach https://notebooklm.google.com/ in the remote tab

Important: The browser session is running on the VPS. What you see in chrome://inspect is a mirror. The login happens on the VPS, and cookies are saved to the VPS Chrome profile.

Step 4: Extract and Save Auth Tokens

After successful login, run on the VPS:

cd /opt/ai-os/products/ceo
python3 scripts/notebooklm_refresh.py

This script: 1. Connects to Chrome headless CDP on port 9223 2. Extracts all cookies from notebooklm.google.com domain 3. Extracts CSRF token, session ID, build_label from the page 4. Saves to both profiles/default/ and auth.json 5. Runs nlm login --check to verify

Expected output:

✓ NotebookLM page loaded and authenticated
✓ Cookies extracted: 52 cookies
✓ CSRF token: AABr7s... (present)
✓ Session ID: 898100228... (present)
✓ Build label: boq_labs... (present)
✓ Tokens saved to profile
✓ Auth check: VALID

4.2 Verification

# Fast check (disk heuristic)
nlm login --check --no-live

# Live check (actual HTTP request)
nlm login --check

# Test an API call
nlm notebook list

4.3 Routine Auth Refresh (Cron)

The cron job at /etc/cron.d/notebooklm-auth runs every 6 hours:

0 */6 * * * root /opt/ai-os/products/ceo/scripts/auth_refresh.sh >> /var/log/notebooklm-auth.log 2>&1

This script: 1. Calls check_auth(live=True) via Python 2. If valid, runs rotate_google_cookies() as proactive maintenance 3. If invalid, runs the revive script + re-extraction 4. Logs all results

5. SOP: Auth Recovery (When It Breaks)

5.1 Symptom Detection

nlm login --check
# Output: "Authentication expired" or "No cached tokens found"

Check for detailed state:

# Is Chrome running?
curl -s http://127.0.0.1:9223/json/version > /dev/null && echo "Chrome alive" || echo "Chrome DEAD"

# Are cached cookies present?
ls -la ~/.notebooklm-mcp-cli/profiles/default/cookies.json

# What email is configured?
python3 -c "import json; d=json.load(open('/root/.notebooklm-mcp-cli/profiles/default/metadata.json')); print('Email:', d.get('email','?'))"

5.2 Automated Recovery (First Try)

# One-command recovery
/opt/ai-os/products/ceo/scripts/revive_notebooklm_auth.sh

What the script does: 1. Kills any existing Chrome on port 9223 2. Relaunches Chrome headless with the existing profile 3. Waits 10s for Chrome to initialize 4. Navigates to https://notebooklm.google.com/ 5. Checks login status by examining page URL 6. If logged in → extracts cookies via CDP → saves to profile 7. If not logged in → returns with error (needs interactive login)

5.3 Interactive Recovery (When Automated Fails)

If automated recovery says "not logged in", you need interactive authentication:

# Step 1: Make sure Chrome is running
/opt/ai-os/products/ceo/scripts/revive_notebooklm_auth.sh

# Step 2: On your LOCAL machine, tunnel to VPS
ssh -N -L 9223:127.0.0.1:9223 root@<VPS_IP>

# Step 3: Open chrome://inspect on local machine
# Add localhost:9223
# Find the NotebookLM page, check URL

# Step 4: If on login page → log in interactively
# If on notebooklm.google.com → tokens should work

# Step 5: After login, re-extract tokens
python3 /opt/ai-os/products/ceo/scripts/notebooklm_refresh.py

5.4 Emergency Recovery (Cookies from Another Browser)

If headless Chrome is completely broken, extract cookies from your desktop browser:

  1. Open Chrome on your desktop
  2. Go to https://notebooklm.google.com/ and ensure you're logged in
  3. Open DevTools (F12) → Application → Cookies → notebooklm.google.com
  4. Export cookies as Netscape format, or use an extension like "Get cookies.txt"
  5. Copy the cookie file to the VPS
  6. Run: nlm login --manual /path/to/cookies.txt
  7. Verify: nlm login --check

6. Technical Deep Dive: Auth Code Flow

6.1 check_auth() Flow (core/auth.py:579-680)

check_auth(profile="default", live=True)
│
├── [fast path] profile_exists()? → no → return (valid=False, reason="no_tokens")
├── load_profile() → get cookies dict
│
├── [non-live] heuristic based on last_validated timestamp (7 day window)
│
└── [live] {the authoritative path}
    │
    ├── _fetch_notebooklm_homepage(cookies)
    │     └── httpx GET https://notebooklm.google.com/
    │     └── with browser-like headers (Sec-Fetch-*, User-Agent)
    │     └── follow_redirects=True
    │     │
    │     ├── status=200 AND not accounts.google.com in URL
    │     │   └── ✓ Authenticated! Extract CSRF, save profile, return valid
    │     │
    │     ├── status!=200 AND not accounts.google.com
    │     │   └── ✗ HTTP error, return valid=False
    │     │
    │     └── accounts.google.com in URL (redirect to login)
    │         └── Not definitive! Some valid sessions also bounce.
    │             Need secondary check via RPC...
    │
    └── [secondary check] NotebookLMClient.list_notebooks()
          │
          ├── raises AuthenticationError → ✗ expired
          ├── raises other exception → network error
          └── success → ✓ recovered! Save refreshed CSRF/session_id

6.2 _refresh_auth_tokens() Flow (base.py:1044-1122)

_refresh_auth_tokens()
│
├── rotate_google_cookies(client)  ← non-fatal, best effort
│     └── POST to https://accounts.google.com/RotateCookies
│     └── With Content-Type: application/json
│     └── Body: '[000,"-0000000000000000000"]'
│     └── Rate-limited: 60s between attempts per storage path
│     └── Disabled via NOTEBOOKLM_DISABLE_ROTATE_COOKIES=1
│
├── GET https://notebooklm.google.com/
│     ├── Redirect to accounts.google.com → raise ValueError (expired)
│     └── status=200 → extract from HTML:
│         ├── SNlM0e → csrf_token
│         ├── FdrFJe → session_id
│         └── cfb2h  → build_label
│
└── snapshot_cookie_input() → update cookies with rotated values
└── _update_cached_tokens() → save to auth.json + profile

6.3 run_headless_auth() Flow (cdp.py:1547-1634)

run_headless_auth(port=9223, timeout=30, profile_name="default")
│
├── has_chrome_profile("default")?
│     └── Check Default/Cookies or Default/Network/Cookies exists
│     └── If no profile → return None (can't auth without saved login)
│
├── Try find_existing_nlm_chrome() → reuse existing instance
│     └── If not found → launch_chrome_process(port, headless=True)
│
├── find_or_create_notebooklm_page(port)
│     └── Navigate to https://notebooklm.google.com/
│
├── Poll for login (timeout=30s)
│     └── Check current_url for accounts.google.com redirect
│     └── If logged_in = False after timeout → return None
│
├── _wait_for_page_ready() → poll for DOM tokens (session_id/build_label)
│
├── get_page_cookies(ws_url) → extract cookies via CDP Network.getAllCookies
├── validate_cookies() → check SID, HSID, SSID, APISID, SAPISID present
├── extract_csrf_token(html), extract_session_id(html)
│
└── save_tokens_to_cache(tokens)
└── cleanup_chrome_profile_cache() → remove Cache, Code Cache, etc.

6.4 CDP Communication Pattern (utils/cdp.py)

Connection via Chrome DevTools Protocol WebSocket:
  ws://127.0.0.1:{port}/devtools/browser/{browser_id}

Commands:
  Network.getAllCookies → list[{name, value, domain, path, ...}]
  Runtime.evaluate({expression: "document.documentElement.outerHTML"}) → HTML
  Runtime.evaluate({expression: "window.location.href"}) → current URL
  Page.navigate({url}) → navigate to URL
  DOM.getDocument → get DOM root node

The CDP WebSocket is cached (_cached_ws) per process for reuse.
Proxy env vars are explicitly disabled for CDP connections (httpx_client with trust_env=False).

7. Troubleshooting Reference

7.1 Common Errors

Error Cause Fix
Authentication expired Cookies no longer valid Run revive script → interactive tunnel re-auth
ProfileNotFoundError No auth profile Run nlm login for initial setup
ClientAuthenticationError Backend auth check failed Check Chrome alive + cookies not expired
No supported browser found Chrome/chromium not installed apt install google-chrome-stable
Chrome "Restore Pages" warning Unclean shutdown of headless Chrome Add --disable-extensions to launch args
WebSocket 403 CDP origin blocked Add --remote-allow-origins=* to Chrome args
no_tokens heuristic pass but live check fails Profile metadata stale, actual cookies dead Re-run interactive auth

7.2 Chrome Profile Health Check

# Check if Chrome profile has cookies
ls -la ~/.notebooklm-mcp-cli/chrome-profiles/default/Default/Cookies
# If missing or 0 bytes → profile is empty, need re-auth

# Check if profile is locked by another Chrome instance
ls -la ~/.notebooklm-mcp-cli/chrome-profiles/default/SingletonLock
# If present and Chrome not running → delete it: rm -f SingletonLock

# Check profile size
du -sh ~/.notebooklm-mcp-cli/chrome-profiles/default/
# Should be ~50-100MB for a working profile

7.3 Forcing Clean Slate (Last Resort)

If all else fails, force a clean auth profile:

# Stop Chrome
pkill -f "remote-debugging-port=9223"

# Backup existing profile
mv ~/.notebooklm-mcp-cli ~/.notebooklm-mcp-cli.bak.$(date +%Y%m%d)

# Start fresh Chrome
google-chrome --headless=new --no-sandbox --remote-debugging-port=9223 \
  --user-data-dir=~/.notebooklm-mcp-cli/chrome-profiles/default/ \
  --no-first-run --disable-extensions --disable-gpu &

# Now authenticate via SSH tunnel (see Section 5.3)

8. Monitoring & Alerting

# /etc/cron.d/notebooklm-auth
# Auth check every 6 hours
0 */6 * * * root /opt/ai-os/products/ceo/scripts/auth_refresh.sh >> /var/log/notebooklm-auth.log 2>&1

# Docker restart if Chrome OOM (every hour check)
*/15 * * * * root /opt/ai-os/products/ceo/scripts/check_chrome_alive.sh >> /var/log/notebooklm-chrome.log 2>&1

8.2 Expected Log Output

2026-07-12 04:00:01 [AUTH CHECK] Starting
2026-07-12 04:00:02 [AUTH CHECK] Chrome on 9223: ALIVE
2026-07-12 04:00:03 [AUTH CHECK] Profile: nhivo2504@gmail.com
2026-07-12 04:00:04 [AUTH CHECK] Result: VALID
2026-07-12 04:00:04 [AUTH CHECK] RotateCookies: SUCCESS (200)
2026-07-12 04:00:04 [AUTH CHECK] Completed OK

8.3 Alerting

If auth check fails twice in a row, the system should: 1. Log the error to auth_failures.log 2. Attempt revive automatically 3. If revive also fails → flag for admin intervention

9. Script Reference

Script Path Purpose
revive_notebooklm_auth.sh /opt/ai-os/products/ceo/scripts/revive_notebooklm_auth.sh Kill Chrome → restart → attempt headless auth extraction
notebooklm_refresh.py /opt/ai-os/products/ceo/scripts/notebooklm_refresh.py Extract cookies from running Chrome headless → save to CLI profile
auth_refresh.sh /opt/ai-os/products/ceo/scripts/auth_refresh.sh Cron-friendly: check auth → revive if dead → log results
check_chrome_alive.sh /opt/ai-os/products/ceo/scripts/check_chrome_alive.sh Verify Chrome process + CDP endpoint alive