workers-ai Workers AI

LLM inference with one binding call — source on GitHub 𝕏 Share

src/index.ts

import { Hono } from 'hono'

const app = new Hono<{ Bindings: Env }>()

app.post('/chat', async (c) => {
  const { prompt } = await c.req.json<{ prompt: string }>()
  const result = await c.env.AI.run('@cf/meta/llama-3.2-3b-instruct', { prompt })
  return c.json(result)
})

export default app

Try it — requests go to a Dynamic Worker running this code. State is isolated per browser session and expires after a while.

response will appear here