Prompt engineering

Prompt engineering in AppEngine boils down to three levers: the system prompt, the context that gets prepended around the user's task, and few-shot examples baked into either. The agent framework gives you a few clean places to set these so you don't end up smuggling instructions through the user message.

The system prompt stack

When the agent runs, the system prompt is assembled in this order:

1. Agent base prompt — defined on the agent class. Establishes role, tone, capabilities.
2. Mode addendum — if aiMode is set, the corresponding mode block is appended.
3. Caller agentInstructions — appended next. Use for tenant-specific guidance.
4. Caller context.systemPrompt — overrides agentInstructions if both are set.
5. Tool catalog — list of available tools is injected automatically.
6. Document context — if files were passed, the extracted text or RAG chunks land here.

Don't try to override the agent base prompt — extend it. Replacing it usually loses framework-level features (memory commands, tool dispatch).

Setting a system prompt

await fetch('/ai/agent/stream', {
  method: 'POST',
  headers: {
    orgid: 'my-org',
    Authorization: `Bearer ${jwt}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    task: 'How do I integrate Stripe?',
    agentInstructions: `
You are an integration assistant for an Acme Inc developer.
Always reply in code-first style with TypeScript snippets.
Reference Acme's internal SDK rather than the raw Stripe SDK when possible.
`,
    conversationId: 'conv-stripe-1',
  }),
});

For a tenant-wide system prompt that applies to every chat, store it on the org config and inject it into your client-side wrapper:

const orgPrompt = await getOrgPrompt(orgId); // your code

await fetch('/ai/agent/stream', {
  method: 'POST',
  headers: { /* ... */ },
  body: JSON.stringify({
    task: userInput,
    agentInstructions: orgPrompt,
    conversationId,
  }),
});

Modes

Modes are pre-baked addenda to the chat agent's base prompt. Pass via context.aiMode:

Mode	Purpose
auto	Default. No addendum.
developer	Code, debugging, architecture.
data-analyst	Data analysis, viz suggestions.
graphic-artist	Image generation enabled. Agent uses generate_image tool freely.
content-writer	Copy, storytelling, written communication.

{ task: 'Write me a brand tagline.', context: { aiMode: 'content-writer' } }

Modes are useful as a low-cost specialisation switch when you don't want to author a full agent. For deeper customisation (different toolset, different memory, different model defaults), register a new agent in the agent factory.

Context construction

context is a free-form object that lives on the agent input and is exposed inside tool calls. Two reserved keys: aiMode and systemPrompt. Everything else is passthrough.

{
  task: 'Find the customer who emailed me last',
  context: {
    callerEmail: '[email protected]',
    callerRole: 'support',
    caseId: 'CASE-1024',
    aiMode: 'auto',
  },
  conversationId: 'conv-...',
}

Inside an MCP tool, the same context is available — useful for permission decisions ("only let support staff see internal notes"). Don't put secrets here; the context is logged when usage tracking is verbose.

Few-shot examples

Two ways to provide examples, depending on whether they should persist across turns or just guide one call.

Persistent — via conversationHistory

Seed the conversation with example turns:

await fetch('/ai/agent/stream', {
  method: 'POST',
  headers: { /* ... */ },
  body: JSON.stringify({
    task: 'Format this address: 123 main st nyc',
    conversationHistory: [
      { role: 'user',      content: 'Format: 99 broadway nyc' },
      { role: 'assistant', content: '99 Broadway, New York, NY' },
      { role: 'user',      content: 'Format: 1 queen st london' },
      { role: 'assistant', content: '1 Queen Street, London' },
    ],
    conversationId: 'conv-formatter-1',
  }),
});

The examples are persisted to chat_message like real turns, so subsequent calls with the same conversationId see them.

Single-call — via the system prompt

For examples that shouldn't pollute the persisted history, embed them in agentInstructions:

{
  task: 'Format: 5 wabash ave chicago',
  agentInstructions: `
Format addresses to: <number> <street>, <city>, <region>

Example:
"99 broadway nyc" → "99 Broadway, New York, NY"
"1 queen st london" → "1 Queen Street, London"
`,
}

Length and cost

Every system-prompt token costs money on every turn. Keep the system prompt under ~300 tokens for chat-style use; longer is fine for specialised agents but check the bill. Use Cost and token control to monitor.

Iterating

The fastest way to iterate a prompt is to keep conversationId stable and edit agentInstructions between turns. The persisted user/assistant turns survive; only the system prompt changes.

For A/B testing, run the same task with two different agentInstructions values into two different conversationIds and compare outputs side by side. Persist runs you like to the prompt_template collection so the rest of the org can re-use them.