Observability and monitoring

Your AI features work. Users are happy. But something will break eventually—and you want to know before your users tell you. Observability means understanding what's happening in production without staring at logs all day.

Outcome

Set up monitoring for your AI features using AI Gateway analytics, structured logging, and a budget with spend alerts.

Fast track

1. Vercel → AI Gateway Overview → Usage and Requests to review requests, tokens, and cost
2. Add console.log(JSON.stringify({ event, requestId, productSlug, duration, tokens })) to AI functions
3. AI Gateway → API Keys → edit ai-review-summary-key: add a daily budget ($10 works) and turn on email alerts at 50%, 75%, and 100%

Hands-on exercise 3.3

Add observability to your AI features:

Requirements:

1. Review AI Gateway analytics (requests, tokens, costs, errors)
2. Add structured logging to summarizeReviews and getReviewInsights
3. Add a budget with spend alerts to your AI Gateway API key
4. Test logging by generating some AI requests

Implementation hints:

• AI Gateway dashboard shows real-time and historical data
• Log request metadata (product slug, token count, duration)
• Budget spend alerts arrive by email
• Start with conservative thresholds and adjust based on real usage

AI Gateway analytics

AI Gateway tracks everything automatically. No code changes needed.

Find your analytics:

1. Vercel Dashboard → AI Gateway
2. Open the Overview, which has Usage charts and Requests summaries

What you'll see:

Overview (Last 7 days)
────────────────────────────────────────
Total requests:     2,847
Success rate:       99.2%
Avg latency:        1,847ms
Total tokens:       2.1M
Estimated cost:     $6.32

Requests by model:
├─ anthropic/claude-sonnet-4.5    2,614 (91.8%)
├─ anthropic/claude-haiku-4.5      198 (7.0%)
└─ openai/gpt-5-mini                35 (1.2%)

Errors:
├─ 429 Rate Limited:    18
├─ 503 Service Error:    4
└─ Timeout:              1

Key metrics to watch:

Understanding the dashboard

Requests by model: Shows how many requests each model handles over time. Look for:

• Unexpected spikes (bot traffic? viral post?)
• Sudden drops (deployment broke something?)
• Patterns (peak hours, quiet periods)

Latency: The Time to First Token chart shows how long the first token takes on average, and the Requests summaries report P75 duration and P75 TTFT. Look for:

• P75 duration that jumps right after a deploy (something changed in the prompt or the model)
• Time to first token that climbs while traffic stays flat (a slower model or provider)

Input/output token counts: Shows input vs output tokens. Look for:

• Input tokens >> Output tokens (normal for summarization)
• Unexpected token growth (prompts getting longer?)
• Spikes correlating with specific products (long reviews?)

Spend: Shows what you spend over time. Look for:

• Steady growth (normal with traffic)
• Sudden jumps (fallbacks triggering? new feature?)
• Model distribution (are fallbacks firing more than expected?)

Adding structured logging

AI Gateway tracks aggregate metrics. For debugging specific requests, add your own logging.

Update lib/ai-summary.ts to add logging while keeping the "use cache" directive from Lesson 3.1:

import { generateText, Output } from "ai";
import { cacheLife, cacheTag } from "next/cache";
import { Product, ReviewInsights, ReviewInsightsSchema } from "./types";
 
export async function summarizeReviews(product: Product): Promise<string> {
  "use cache";
  cacheLife("hours");
  cacheTag(`product-summary-${product.slug}`);
 
  const startTime = Date.now();
  const requestId = crypto.randomUUID();
 
  console.log(JSON.stringify({
    event: "ai_request_start",
    requestId,
    function: "summarizeReviews",
    productSlug: product.slug,
    reviewCount: product.reviews.length,
    timestamp: new Date().toISOString(),
  }));
 
  const averageRating =
    product.reviews.reduce((acc, review) => acc + review.stars, 0) /
    product.reviews.length;
 
  const prompt = `Write a summary of the reviews for the ${product.name} product...`; // Your existing prompt
 
  try {
    const { text, usage } = await generateText({
      model: "anthropic/claude-sonnet-4.5",
      prompt,
      maxOutputTokens: 1000,
      temperature: 0.75,
    });
 
    const duration = Date.now() - startTime;
 
    console.log(JSON.stringify({
      event: "ai_request_success",
      requestId,
      function: "summarizeReviews",
      productSlug: product.slug,
      duration,
      inputTokens: usage?.inputTokens,
      outputTokens: usage?.outputTokens,
      totalTokens: usage?.totalTokens,
      timestamp: new Date().toISOString(),
    }));
 
    return text.trim();
  } catch (error) {
    const duration = Date.now() - startTime;
 
    console.error(JSON.stringify({
      event: "ai_request_error",
      requestId,
      function: "summarizeReviews",
      productSlug: product.slug,
      duration,
      error: error instanceof Error ? error.message : "Unknown error",
      timestamp: new Date().toISOString(),
    }));
 
    throw new Error("Unable to generate review summary. Please try again.");
  }
}

What this logs:

Request start:

{
  "event": "ai_request_start",
  "requestId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "function": "summarizeReviews",
  "productSlug": "mower",
  "reviewCount": 12,
  "timestamp": "2024-01-15T14:32:01.234Z"
}

Request success:

{
  "event": "ai_request_success",
  "requestId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "function": "summarizeReviews",
  "productSlug": "mower",
  "duration": 2341,
  "inputTokens": 847,
  "outputTokens": 89,
  "totalTokens": 936,
  "timestamp": "2024-01-15T14:32:03.575Z"
}

Request error:

{
  "event": "ai_request_error",
  "requestId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "function": "summarizeReviews",
  "productSlug": "mower",
  "duration": 5023,
  "error": "Rate limit exceeded",
  "timestamp": "2024-01-15T14:32:06.257Z"
}

Why structured logging?

• Searchable — Find all errors for a specific product
• Parseable — Tools like Vercel Logs, Datadog, or Axiom can parse JSON
• Correlatable — Request IDs link start → success/error
• Measurable — Track duration, tokens, and patterns over time

Viewing logs in Vercel

Find your logs:

1. Vercel Dashboard → Your project
2. Click Logs tab
3. Filter by:
   • Level: Error (requests that logged with console.error)
   • Timeline: the past hour, a few days, or a custom range, depending on your plan's log retention
   • Search: ai_request_error or aquaHeat

Example log search:

// Find all AI errors in the selected timeline
ai_request_error

// Find all requests for a specific product
aquaHeat

The search box matches text in the log message and the request path. It doesn't parse your JSON, so a query like duration > 3000 searches for that literal text and finds nothing. To query fields like duration, send your logs to a tool like Datadog or Axiom with a log drain.

Setting up budgets and spend alerts

Picture a scraper that finds /api/summary/mower and hits it all weekend. Every request is a real Claude call. You find out Monday, from the invoice.

AI Gateway handles this with budgets. A budget caps how much a team, project, API key, or team member can spend in a refresh period, and it can email you on the way to that cap.

Our app authenticates with AI_GATEWAY_API_KEY, so let's put the budget on that key. (A project budget only counts requests that use the project's OIDC token, and an API key's spend never counts toward a project.)

1. Vercel Dashboard → AI Gateway → API Keys
2. Find ai-review-summary-key, open its ··· menu, and select Edit key
3. Turn on the budget, enter a limit ($10 works for this course, any amount does), and choose a daily refresh period
4. Under Email ... when usage reaches, select 50%, 75%, and 100%
5. Save

The key now shows its spend against the limit on the API Keys page, something like $0.04 / $10 spent. Here's what each threshold does:

Daily budget: $10 on ai-review-summary-key
├─ $5.00  (50%)  → Email to the key's creator
├─ $7.50  (75%)  → Email to the key's creator
└─ $10.00 (100%) → Email, then AI Gateway rejects further requests with a 402 until midnight UTC

The 50% and 75% emails are warnings only. At 100%, requests start failing with a 402 and quota_for_entity_exceeded, and that's when the fallback from 3.2 earns its keep: shoppers see the plain "Customer Reviews" card instead of an error page. The budget is a soft cap, so the request that crosses the line still completes and you can land slightly over.

Error and deployment notifications (Vercel):

Spend alerts only cover money. For "something broke" emails, each team member picks their own notifications:

1. Select your team, then open Settings in the sidebar
2. Under Account, select My Notifications
3. Turn on:
   • Deployment failures
   • Error anomalies (a high rate of 5xx function invocations)
   • Usage alerts

These settings only change your own notifications, so teammates who want them need to turn them on too. If you're on Pro or Enterprise with Observability Plus, Vercel Alerts can also send error and usage anomalies to Slack or a webhook.

Start conservative: It's better to get too many alerts initially and tune them down than to miss something critical.

Debugging production issues

When something goes wrong, here's how to investigate:

1. Check AI Gateway dashboard

• Error spike? What time did it start?
• Which model? (primary or fallback?)
• What error codes? (429, 503, timeout?)

2. Check Vercel logs

• Search for ai_request_error
• Filter to the timeframe
• Look for patterns (same product? same error?)

3. Correlate with deployments

• Did a deployment happen right before the errors?
• Check deployment logs for build issues

4. Check provider status

• Anthropic Status
• OpenAI Status
• If provider is down, your fallbacks should be handling it

Common issues and causes:

Production monitoring checklist

Before going live, verify:

• AI Gateway analytics accessible: Can you see requests, costs, errors?
• Structured logging added: JSON logs with request IDs and metadata
• Budget with spend alerts set: Get notified before bills surprise you
• Error notifications on: Know when things break
• Fallbacks working: Verified backup models are configured
• Logs searchable: Can find specific requests when debugging

Try it

1. Explore your AI Gateway dashboard:

   • How many requests have you made?
   • What's your average latency?
   • Any errors?

2. Add structured logging:

   • Update summarizeReviews with the logging code
   • Generate a few summaries
   • Check Vercel logs for the JSON output

3. Set up a budget with spend alerts:

   • AI Gateway → API Keys → ai-review-summary-key → Edit key
   • Add a daily budget ($1 makes a quick test, and $1 is the minimum) with alerts at 50%, 75%, and 100%
   • Run rm -rf .next, restart pnpm dev, and open a product page so the cache misses, then check that the key shows its spend against the limit
   • Raise the limit back up when you're done testing, or a full day of traffic will hit the 402

4. Simulate an error (optional):

   • Temporarily break your API key
   • Visit a product page
   • Check that error logs appear correctly
   • Fix the API key

Commit

git add lib/ai-summary.ts
git commit -m "feat(observability): add structured logging to AI functions"
git push

Done-when

• Explored AI Gateway analytics dashboard
• Understand key metrics (requests, latency, tokens, costs)
• Added structured logging to AI functions
• ai-review-summary-key has a budget with at least one spend alert threshold
• My Notifications has deployment failures and error anomalies turned on
• Know how to search logs in Vercel
• Understand debugging workflow for production issues

What's next

Your AI features are observable. You'll know when things break, why they broke, and how much it's costing. Time to wrap up the course and review everything you've built.

Sources:

• AI Gateway observability
• AI Gateway budgets
• Notifications
• Vercel Logs
• Logging Best Practices: An Engineer's Checklist