xps

Managing browser state, cookies, and session persistence across automation sessions

The key to effective authentication automation is preserving session state across browser instances and Claude conversations. Let's explore the mechanisms and implementation strategies.

Cookies are the primary mechanism for maintaining authenticated sessions. Understanding cookie attributes is crucial.

Essential Cookie Properties:

Domain: Which domain can access the cookie (e.g., .university.edu for all subdomains)
Path: URL path scope (e.g., / for entire site)
Expires/Max-Age: When cookie becomes invalid
HttpOnly: Prevents JavaScript access (security feature)
Secure: Only sent over HTTPS
SameSite: CSRF protection (Strict, Lax, None)

Saving and Restoring Cookies

Save Cookies After Authentication

Capture all cookies from the browser context after successful authentication:

// Save cookies after authentication
const cookies = await context.cookies();
await fs.writeFile(
  'state/cookies.json',
  JSON.stringify(cookies, null, 2)
);

Restore Cookies in New Session

Load saved cookies before accessing protected resources:

// Restore cookies in new session
const savedCookies = JSON.parse(
  await fs.readFile('state/cookies.json', 'utf-8')
);
await context.addCookies(savedCookies);

Filter expired cookies before restoration:

function isValidCookie(cookie) {
  if (!cookie.expires) return true; // Session cookie
  const expiryDate = new Date(cookie.expires * 1000);
  return expiryDate > new Date();
}

// Filter expired cookies before restoration
const validCookies = savedCookies.filter(isValidCookie);

Session vs Persistent Cookies

Session Cookies:

No Expires attribute
Cleared when browser closes
Temporary authentication state

Persistent Cookies:

Have Expires or Max-Age attribute
Survive browser restart
Long-term authentication state

For automation, persistent cookies are ideal. Look for "Remember Me" options during login to obtain them.

LocalStorage and SessionStorage

Modern web apps often use Web Storage API for client-side state.

LocalStorage Characteristics

Key-value string storage
Persists across browser sessions
Scoped to origin (protocol + domain + port)
Typically 5-10MB limit per origin

SessionStorage Characteristics

Same API as localStorage
Cleared when tab/window closes
Separate storage per tab
Useful for temporary auth tokens

Extracting and Restoring Storage

Extract LocalStorage After Authentication

Capture all localStorage data after successful login:

// Extract localStorage after authentication
const localStorage = await page.evaluate(() => {
  return JSON.stringify(window.localStorage);
});

await fs.writeFile('state/localStorage.json', localStorage);

Restore LocalStorage Before Accessing App

Inject saved localStorage data before navigating to protected pages:

// Restore localStorage before accessing app
const savedStorage = await fs.readFile('state/localStorage.json', 'utf-8');
await page.evaluate((data) => {
  const items = JSON.parse(data);
  for (const [key, value] of Object.entries(items)) {
    window.localStorage.setItem(key, value);
  }
}, savedStorage);

Common Auth Tokens in Storage

access_token: OAuth/OIDC access token
refresh_token: Token for obtaining new access tokens
id_token: OIDC identity token (JWT)
user: Serialized user object
session_id: Application session identifier

LocalStorage is accessible to any JavaScript on the page. If an XSS vulnerability exists, tokens in localStorage can be stolen. HttpOnly cookies are more secure against XSS.

Browser Profile Reuse

Playwright supports persistent browser contexts that preserve all state automatically.

Persistent Context Approach

import { chromium } from 'playwright';

// Create or reuse persistent context
const userDataDir = './state/browser-profile';
const context = await chromium.launchPersistentContext(userDataDir, {
  headless: false,
  viewport: { width: 1280, height: 720 }
});

const page = await context.newPage();
await page.goto('https://authenticated-site.com');
// All cookies, storage, cache preserved automatically

// Close context (state auto-saved)
await context.close();

Advantages

Automatic state persistence (cookies, storage, cache, service workers)
Browser behaves identically to manual usage
Extensions can be loaded if needed
IndexedDB and other storage mechanisms preserved

Disadvantages

Larger disk footprint
Potential for stale state accumulation
Less granular control over what's saved
Profile corruption risks

Profile Location Strategy

Organize profiles by institution or account:

// Separate profiles per account/institution
const profilePath = `./state/profiles/${institution}-${username}`;
const context = await chromium.launchPersistentContext(profilePath, {
  // options
});

Authentication Token Refresh

Many modern auth systems use short-lived access tokens with long-lived refresh tokens.

Token Refresh Flow

Access Token Expires

Access tokens typically expire after 1-24 hours.

Application Requests New Token

Application uses refresh token to request new access token from auth server.

Auth Server Validates Refresh Token

Server validates the refresh token and checks permissions.

Server Issues New Access Token

Server issues new access token (and optionally new refresh token).

Application Continues

Application continues with new access token without user intervention.

Automated Refresh Implementation

class TokenManager {
  constructor(storageFile) {
    this.storageFile = storageFile;
    this.tokens = null;
  }

  async load() {
    try {
      const data = await fs.readFile(this.storageFile, 'utf-8');
      this.tokens = JSON.parse(data);
    } catch (error) {
      this.tokens = null;
    }
  }

  async save() {
    await fs.writeFile(
      this.storageFile,
      JSON.stringify(this.tokens, null, 2)
    );
  }

  isAccessTokenValid() {
    if (!this.tokens?.access_token) return false;
    if (!this.tokens?.expires_at) return false;

    // Check if token expires within next 5 minutes
    const expiryTime = new Date(this.tokens.expires_at);
    const bufferTime = new Date(Date.now() + 5 * 60 * 1000);

    return expiryTime > bufferTime;
  }

  async refreshAccessToken() {
    if (!this.tokens?.refresh_token) {
      throw new Error('No refresh token available');
    }

    const response = await fetch('https://auth.example.com/token', {
      method: 'POST',
      headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
      body: new URLSearchParams({
        grant_type: 'refresh_token',
        refresh_token: this.tokens.refresh_token,
        client_id: process.env.CLIENT_ID,
        client_secret: process.env.CLIENT_SECRET,
      })
    });

    if (!response.ok) {
      throw new Error('Token refresh failed');
    }

    const data = await response.json();

    this.tokens.access_token = data.access_token;
    this.tokens.expires_at = new Date(
      Date.now() + data.expires_in * 1000
    ).toISOString();

    // Update refresh token if provided
    if (data.refresh_token) {
      this.tokens.refresh_token = data.refresh_token;
    }

    await this.save();
    return this.tokens.access_token;
  }

  async getValidAccessToken() {
    await this.load();

    if (this.isAccessTokenValid()) {
      return this.tokens.access_token;
    }

    return await this.refreshAccessToken();
  }
}

Session Timeout Handling

Authentication sessions eventually expire. Robust automation must handle this gracefully.

Detection Strategies

1. Response Code Detection

async function isSessionExpired(page, response) {
  // Check for redirect to login page
  if (response.url().includes('/login')) return true;

  // Check for 401 Unauthorized
  if (response.status() === 401) return true;

  // Check for 403 Forbidden (sometimes used for expired sessions)
  if (response.status() === 403) {
    const body = await response.text();
    if (body.includes('session expired')) return true;
  }

  return false;
}

2. Page Content Detection

async function checkAuthRequired(page) {
  // Look for login form
  const loginForm = await page.$('form[action*="login"]');
  if (loginForm) return true;

  // Look for "session expired" message
  const expiredText = await page.textContent('body');
  if (expiredText.includes('session has expired')) return true;

  // Look for absence of authenticated user element
  const userElement = await page.$('.user-profile');
  if (!userElement) return true;

  return false;
}

3. Proactive Expiry Tracking

class SessionTracker {
  constructor(sessionDuration = 30 * 60 * 1000) { // 30 minutes default
    this.sessionDuration = sessionDuration;
    this.lastAuthTime = null;
  }

  markAuthenticated() {
    this.lastAuthTime = Date.now();
  }

  isLikelyExpired() {
    if (!this.lastAuthTime) return true;
    return (Date.now() - this.lastAuthTime) > this.sessionDuration;
  }
}

Re-authentication Flow

async function ensureAuthenticated(page, authFunction) {
  // Try to navigate to protected resource
  const response = await page.goto('https://protected.com/resource');

  if (await isSessionExpired(page, response)) {
    console.log('Session expired, re-authenticating...');
    await authFunction(page);

    // Retry original navigation
    await page.goto('https://protected.com/resource');
  }
}

Persistent Session Extension

Some sites offer "keep me logged in" functionality that extends session lifetime significantly:

// During initial authentication
await page.check('input[name="remember_me"]');
await page.click('button[type="submit"]');

// This typically sets a long-lived cookie (days to months)

State Encryption

Storing authentication state in plain text is a security risk. Implement encryption for sensitive data.

Never store credentials, tokens, or session data in plain text. Always use encryption with proper key management.

Encryption Implementation

import crypto from 'crypto';

class EncryptedStateManager {
  constructor(encryptionKey) {
    // Derive encryption key from password/environment variable
    this.key = crypto.scryptSync(encryptionKey, 'salt', 32);
  }

  encrypt(data) {
    const iv = crypto.randomBytes(16);
    const cipher = crypto.createCipheriv('aes-256-cbc', this.key, iv);

    let encrypted = cipher.update(JSON.stringify(data), 'utf8', 'hex');
    encrypted += cipher.final('hex');

    return {
      iv: iv.toString('hex'),
      data: encrypted
    };
  }

  decrypt(encrypted) {
    const iv = Buffer.from(encrypted.iv, 'hex');
    const decipher = crypto.createDecipheriv('aes-256-cbc', this.key, iv);

    let decrypted = decipher.update(encrypted.data, 'hex', 'utf8');
    decrypted += decipher.final('utf8');

    return JSON.parse(decrypted);
  }

  async saveState(filepath, state) {
    const encrypted = this.encrypt(state);
    await fs.writeFile(filepath, JSON.stringify(encrypted));
  }

  async loadState(filepath) {
    const encrypted = JSON.parse(await fs.readFile(filepath, 'utf-8'));
    return this.decrypt(encrypted);
  }
}

Key Management Best Practices

Use Environment Variables

Store encryption keys in environment variables:

const encryptionKey = process.env.STATE_ENCRYPTION_KEY;

State Management