AI, prompt kits & instructions
Use the user's own AI from your panel or your MCP server — tiers instead of model ids, a daily budget the user controls, streaming and JSON answers — plus prompt kits for the composer and instructions for every chat.
Your app can use the AI the user already has in Chataway. You never hold a key or name a model: you send a prompt and a tier hint, and Chataway runs it on the user's own background model and counts it against a daily limit the user controls.
There are three doors, and they need different things:
| Door | What it is | Needs |
|---|---|---|
| Model calls | Your panel (app.models.*) or your MCP server (sampling) asks the user's model | Grant models |
| Prompt kits | Starters in the user's composer, from contributes.prompts | No grant |
| Instructions | A section in every chat's system prompt, from contributes.instructions | No grant |
All three are app API 4, so the manifest says so:
{
"requires": { "hostApi": 4 },
"grants": ["models"],
"permissions": ["Use your AI to summarize the notes you open (counted in your daily AI limit for this app)"]
}Without requires.hostApi, chataway validate fails with hostapi.required. See App API levels.
#Model calls
import { connect } from '@chataway/apps/bridge';
const app = await connect();
const { text, model, usage } = await app.models.run({
system: 'You write short, plain summaries.',
prompt: `Summarize this note in three bullets:\n\n${note}`,
tier: 'fast',
maxTokens: 300,
});
label.textContent = `${model.label} · ${usage.callsToday} AI calls today`;| Option | Type | |
|---|---|---|
prompt | string | One user message. Optional when you pass messages (then it is added as the newest user turn). |
messages | { role: 'user' | 'assistant', content: string }[] | A conversation, up to 100 messages. The last one must be the user's. |
system | string | Extra instructions. |
images | (ref | rel | Blob)[] | Up to 8 images (png, jpeg, webp, gif; 5 MB each). Refs and rels point into your app's own storage, and the file name's extension gives the type; Chataway uploads a Blob into your storage first. Needs a model that reads images. |
schema | JSON Schema object | Ask for JSON. The parsed answer comes back in json. |
maxTokens | number | Answer budget. Default 1000 (16–16,000). |
tier | 'fast' | 'balanced' | 'best' | A hint. Default balanced. |
The prompt, messages and system text together may be up to 200,000 characters. The result is { text, json?, model: { label }, usage: { callsToday, limit } }. model.label is a name to show ("Claude Haiku"), never an id you can call.
Your app must be on in the project. A call without the models grant fails with denied.
#Tiers
| Tier | What Chataway picks |
|---|---|
balanced (default) | The user's background model — the one they chose in Settings → Providers → Background model — on the provider of the panel's chat (with no chat, the first provider this computer can run). |
fast | The catalog's fast model of that same provider, when it names one. |
best | The catalog's most capable model of that same provider, when it names one. |
The provider never changes: a tier only picks another model of the same provider, and falls back to balanced when there is none. A tier is a hint, so write prompts that work on any of them.
#JSON answers
const { json } = await app.models.run({
prompt: `Give this note a title and up to 3 tags:\n\n${note}`,
schema: {
type: 'object',
properties: { title: { type: 'string' }, tags: { type: 'array', items: { type: 'string' } } },
required: ['title', 'tags'],
},
});From the panel, schema is a JSON Schema object (a zod schema can't cross the bridge — convert it with z.toJSONSchema()). Chataway puts the schema in the instructions and parses the answer. If the answer isn't JSON, it asks once more — that retry is a second call in your daily budget — and then fails. Chataway doesn't check the parsed answer against a JSON Schema, so check the fields you rely on. (Server code that passes a zod schema to ctx.models.run gets the answer validated too.)
#Streaming
const s = app.models.stream({ prompt: 'Draft a reply to this note', tier: 'balanced' });
for await (const delta of s) output.textContent += delta;
const { text, model } = await s.result;app.models.stream() returns at once. Iterate it for text deltas; s.result resolves with { text, model }, and s.text is the text so far. A failure (over budget, no AI access) throws a BridgeError from the loop and rejects s.result. The answer arrives as one delta when the provider can't stream, when you pass a schema (the whole answer is parsed first), and when the user reaches Chataway remotely (the phone), where the connection doesn't stream. s.result has no json — use models.run for JSON. To stop early, call s.cancel() or break out of the loop: the loop ends, s.result rejects with cancelled, and Chataway aborts the model call (it still counts as one call).
Underneath, models.stream answers { streamId }, the host sends model-chunk, model-done and model-error events, and models.cancel({ streamId }) stops it. You only need those if you write your own client.
#The daily budget
Every model request your app makes counts as one call: a panel call or stream, a sampling request from your server, each model turn of an agent run, and server code's ctx.models. A JSON retry is one more call, and a cancelled stream still counts. Chataway keeps a per-app ledger for each day (the user's local day).
| Default limit | 300 calls a day for each app. Chataway's own apps (built in, or By Chataway in the store) have no limit unless the user sets one. |
| Who changes it | The user, on your app's page in Apps (AI use) — any whole number, 0 to turn AI off for your app, or back to the default |
| Over the limit | The call fails with code budget (HTTP 429): "Sketch Notes used today's 300 AI calls — change the limit in Settings → Apps → Sketch Notes". At 0: "AI is off for Sketch Notes — …" |
| Concurrency | 2 calls of one app run at once; the rest wait their turn |
app.models.usage() returns { callsToday, limit, unlimited } (limit is null when there is none). Check it before you start something big:
import { BridgeError } from '@chataway/apps/bridge';
const { callsToday, limit } = await app.models.usage();
if (limit !== null && limit - callsToday < files.length) return showNotice(`That needs ${files.length} AI calls; ${limit - callsToday} are left today.`);
try {
await summarizeAll(files);
} catch (e) {
if (e instanceof BridgeError && e.code === 'budget') return showNotice(e.message);
throw e;
}Use AI for things the user asked for. A background loop that burns the user's limit is a review rejection.
#Sampling from your server
A cloud app's MCP server can ask the user's model too, with MCP sampling (sampling/createMessage). Chataway offers the sampling capability on its connection to your server only when your app has the models grant.
import { createAppServer, SamplingError } from '@chataway/apps/server';
import { z } from 'zod';
const app = createAppServer({ manifest: './plugin.json' });
app.tool('summarize_text', { input: { text: z.string() } }, async ({ text }, ctx) => {
if (!ctx.canSample) return { text: 'AI is not available for this app here.', isError: true };
try {
const { text: summary } = await ctx.sample({ system: 'Be brief.', prompt: `Summarize:\n\n${text}`, tier: 'fast', maxTokens: 300 });
return summary;
} catch (e) {
if (e instanceof SamplingError && e.code === 'budget') return { text: e.message, isError: true };
throw e;
}
});ctx.sample() takes { prompt?, messages?, system?, images?, maxTokens?, tier?, temperature?, stopSequences?, signal?, timeoutMs? } and returns { text, model, stopReason? } (model is the label). Outside a tool handler, sample(server, opts) works on an McpServer or a low-level Server. See Server helpers.
What Chataway does with the request:
| You send | Chataway |
|---|---|
| Text and image content | Runs it on the user's model, like a panel call with messages and images, on the provider of the chat the tool call came from |
systemPrompt, maxTokens | Uses them |
modelPreferences | intelligencePriority ≥ 0.7 → best; else speedPriority or costPriority ≥ 0.7 → fast; else balanced |
temperature, stopSequences, includeContext | Ignores them |
tools / toolChoice, or tool content (tool_use, tool_result) | Refuses it: "tools are not supported in sampling" |
| Audio or other content | Refuses it: text and images only |
No models grant | No sampling capability; a request fails with "this app has no AI access" |
The reply is { role: 'assistant', content: { type: 'text', text }, model: <label>, stopReason: 'endTurn' }. Each request counts in your daily budget.
If you use another MCP library, set modelPreferences yourself. These are the values the helpers send (tierPreferences(tier)):
| Tier | intelligencePriority | speedPriority | costPriority |
|---|---|---|---|
best | 0.9 | 0.2 | 0.2 |
balanced | 0.5 | 0.5 | 0.5 |
fast | 0.2 | 0.9 | 0.8 |
#Prompt kits
Prompt kits are ready-made prompts for the user's composer. They come from your manifest, so they need no code and no grant:
"contributes": {
"prompts": [
{
"id": "outline",
"title": "Outline a note",
"description": "Turn a note into a structured outline",
"text": "Read {{note}} and rewrite it as an outline with at most {{depth}} levels. Keep every fact.",
"vars": [
{ "id": "note", "label": "Note", "kind": "file", "required": true },
{ "id": "depth", "label": "Levels" }
],
"surfaces": ["composer", "panel"]
}
]
}| Field | Rules |
|---|---|
id | A lowercase letter, then lowercase letters, digits or _ (≤ 48). Unique. |
title | ≤ 60 characters. What the user picks. |
description | ≤ 200 characters. |
text | 1–4000 characters. {{var}} placeholders name entries of vars; an undeclared one is the error prompts.unknown-var. |
vars | Up to 8: { id, label (≤ 40), kind?, required? }. kind is text (default), selection (a longer text, prefilled with what the user has selected) or file (a workspace file, written as @path). |
surfaces | composer puts the kit in the composer's Prompts menu and starter chips. Default: ["composer", "panel"]. app.prompts.list() returns every kit either way. |
Up to 20 kits per app.
Where users see them. The composer's ⋯ menu has Prompts: the composer kits of every app that is on in the project (and serves that chat's provider), grouped by app, and the user's own saved prompts. A kit with vars opens a small form first; Add to message stays off until the required vars are filled. An empty chat shows up to four composer kits as starter chips. A kit always fills the composer. The user reads it and sends it. Users can also save what's in their composer as a prompt of their own.
From your panel. app.prompts.list() returns your own kits. app.chat.compose({ promptId, values }) has Chataway fill one into the composer:
await app.chat.compose({ promptId: 'outline', values: { note: 'notes/ideas.md', depth: '3' } });Chataway fills the placeholders. A placeholder without a value stays as it is ({{depth}}), so the user sees what is missing — required isn't enforced here. A file value is a workspace-relative path, written as @notes/ideas.md. For a selection var, pass the text yourself. A promptId that isn't one of your kits fails with denied. See App chats.
#Instructions
contributes.instructions is one text (1–2000 characters) that goes into every chat of a project where your app is on, for every provider: Chataway's chat, Claude Code, Codex, Cursor, Grok and OpenCode. It is also the app part of an agent run's system prompt.
"contributes": {
"instructions": "Sketch Notes keeps the user's notes as Markdown files in the folder they granted it (usually notes/). One topic per file, a # title on the first line. When the user asks about their notes, read those files. Don't rename or delete notes unless the user asks."
}- Say what your app is, where its files are and what a good result looks like. Don't tell the agent to prefer your tools or avoid others.
- It shares the instruction budget with every other app the user has on. Keep it to a few hundred characters.
- Write it provider-neutral: name your own tools, never
Bashormcp__…. - A person reads it in review, in full, every time it changes — see the review guidelines.
Local apps can still register instructions in code with ctx.prompts.register(). Cloud apps have no server code on the Mac, so contributes.instructions is their way in.
#In the mock host
Outside Chataway, the mock host answers model calls with an echo model and counts them against a daily limit like the real one:
const app = await connect({
mock: {
manifest, // your plugin.json: grants, prompt kits
modelLimit: 20, // default 300; null = unlimited; 0 = AI off
model: ({ messages, schema, tier }) => schema ? { title: 'Test', tags: ['a'] } : `(${tier}) ${messages.at(-1)!.content}`,
},
});Without model, a call returns a line that repeats the prompt, and a call with schema returns an example object built from the schema. app.mockHost.composed records what chat.compose would have filled.