Introducing the Vercel AI SDK

Note: This post is historical. Product names and runtime guidance may have changed. For new projects, use Vercel Functions with the Node.js runtime and Fluid compute. Use Routing Middleware for request-time routing before a response completes.

Over the past 6 months, AI companies like Scale, Jasper, Perplexity, Runway, Lexica, and Jenni have launched with Next.js and Vercel. Vercel helps accelerate your product development by enabling you to focus on creating value with your AI applications, rather than spending time building and maintaining infrastructure.

Today, we're launching new tools to improve the AI experience on Vercel.

• **Vercel AI SDK: **Easily stream API responses from AI models

• Chat & Prompt Playground: Explore models from OpenAI, Hugging Face, and more

Link to headingThe Vercel AI SDK

The Vercel AI SDK is an open-source library designed to help developers build conversational, streaming, and chat user interfaces in JavaScript and TypeScript. The SDK supports React/Next.js, Svelte/SvelteKit, with support for Nuxt/Vue coming soon.

To install the SDK, enter the following command in your terminal:

npm install ai

You can also view the source code on GitHub.

Link to headingBuilt-in LLM Adapters

Choosing the right LLM for your application is crucial to building a great experience. Each has unique tradeoffs, and can be tuned in different ways to meet your requirements.

Vercel’s AI SDK embraces interoperability, and includes first-class support for OpenAI, LangChain, and Hugging Face Inference. This means that regardless of your preferred AI model provider, you can leverage the Vercel AI SDK to create cutting-edge streaming UI experiences.

1
import { OpenAIStream, StreamingTextResponse } from 'ai'
2
import { Configuration, OpenAIApi } from 'openai-edge'
3

4
// Create an OpenAI API client (that's edge friendly!)
5
const config = new Configuration({
6
  apiKey: process.env.OPENAI_API_KEY
7
})
8
const openai = new OpenAIApi(config)
9
 
10
// IMPORTANT! Set the runtime to edge
11
export const runtime = 'edge'
12
 
13
export async function POST(req: Request) {
14
  // Extract the `messages` from the body of the request
15
  const { messages } = await req.json()
16
 
17
  // Ask OpenAI for a streaming chat completion given the prompt
18
  const response = await openai.createChatCompletion({
19
    model: 'gpt-3.5-turbo',
20
    stream: true,
21
    messages
22
  })
23
  // Convert the response into a friendly text-stream
24
  const stream = OpenAIStream(response)
25
  // Respond with the stream
26
  return new StreamingTextResponse(stream)
27
}

Link to headingStreaming First UI Helpers

The Vercel AI SDK includes React and Svelte hooks for data fetching and rendering streaming text responses. These hooks enable real-time, dynamic data representation in your application, offering an immersive and interactive experience to your users.

Building a rich chat or completion interface now just takes a few lines of code thanks to useChat and useCompletion:

1
'use client'
2

3
import { useChat } from 'ai/react'
4

5
export default function Chat() {
6
  const { messages, input, handleInputChange, handleSubmit } = useChat()
7

8
  return (
9
    <div>
10
      {messages.map(m => (
11
        <div key={m.id}>
12
          {m.role}: {m.content}
13
        </div>
14
      ))}
15

16
      <form onSubmit={handleSubmit}>
17
        <label>
18
          Say something...
19
          <input
20
            value={input}
21
            onChange={handleInputChange}
22
          />
23
        </label>
24
      </form>
25
    </div>
26
  )
27
}

Link to headingStream Helpers and Callbacks

We've also included callbacks for storing completed streaming responses to a database within the same request. This feature allows for efficient data management and streamlines the entire process of handling streaming text responses.

1
export async function POST(req: Request) {
2
  // ... same as above
3
 
4
  // Convert the response into a friendly text-stream
5
  const stream = OpenAIStream(response, {
6
    onStart: async () => {
7
      // This callback is called when the stream starts
8
      // You can use this to save the prompt to your database
9
      await savePromptToDatabase(prompt)
10
    },
11
    onToken: async (token: string) => {
12
      // This callback is called for each token in the stream
13
      // You can use this to debug the stream or save the tokens to your database
14
      console.log(token)
15
    },
16
    onCompletion: async (completion: string) => {
17
      // This callback is called when the stream completes
18
      // You can use this to save the final completion to your database
19
      await saveCompletionToDatabase(completion)
20
    }
21
  })
22
 
23
  // Respond with the stream
24
  return new StreamingTextResponse(stream)
25
}

Link to headingVercel Functions ready

The AI SDK works with Vercel Functions. You can deploy AI applications that scale instantly, stream generated responses, and run efficiently with Fluid compute.

With framework-defined infrastructure, you write application code in frameworks like Next.js and SvelteKit using the AI SDK, and Vercel converts this code into global application infrastructure.

Link to headingChat & Prompt Playground

In late April, we launched an interactive online prompt playground play.vercel.ai with 20 open source and cloud LLMs.

The playground provides a valuable resource for developers to compare various language model results in real-time, tweak parameters, and quickly generate Next.js, Svelte, and Node.js code.

Today, we’ve added a new chat interface to the playground so you can simultaneously compare chat models side-by-side. We’ve also added code generation support for the Vercel AI SDK. You can now go from playground to chat app in just a few clicks.

Link to headingWhat’s Next?

We'll be adding more SDK examples in the coming weeks, as well as new templates built entirely with the AI SDK. Further, as new best practices for building AI applications emerge, we’ll lift them into the SDK based on your feedback.