Prompt Optimization

ThinkLang includes built-in prompt optimization utilities for improving output quality over time.

Example Store

The ExampleStore records prompt-result pairs and provides them as few-shot examples for future calls.

import { ExampleStore, optimizedThink } from "thinklang";

const store = new ExampleStore();

// First call — no examples, learns from result
const result1 = await optimizedThink({
  jsonSchema: schema,
  prompt: "Classify this email",
  exampleStore: store,
});

// Second call — injects first result as a few-shot example
const result2 = await optimizedThink({
  jsonSchema: schema,
  prompt: "Classify this other email",
  exampleStore: store,
});

Global Example Store

A globalExampleStore singleton is available. optimizedThink() uses it by default.

import { globalExampleStore, optimizedThink } from "thinklang";

// Uses globalExampleStore automatically
const result = await optimizedThink({ jsonSchema: schema, prompt: "..." });

// Check stored examples
console.log(globalExampleStore.count(schema));

Prompt Variation Search

Use optimize() to try multiple prompt variations against an evaluation dataset:

import { optimize } from "thinklang";

const result = await optimize({
  prompts: [
    "Classify the sentiment of this text",
    "What is the emotional tone of this text?",
    "Rate the positivity of this text",
  ],
  evalSet: [
    { input: "I love this!", expected: { label: "positive" } },
    { input: "Terrible.", expected: { label: "negative" } },
  ],
  scorer: (result, expected) => result.label === expected.label ? 1 : 0,
  thinkOptions: { jsonSchema: sentimentSchema },
});

console.log(result.bestPrompt);   // The prompt that scored highest
console.log(result.bestScore);    // Score from 0 to 1

API Reference

ExampleStore

• record(schema, prompt, result, score?) — Store a prompt-result pair
• retrieve(schema, n?) — Get top N examples for a schema
• count(schema) — Number of stored examples
• clear() — Remove all examples

optimizedThink(options)

Like think() but injects few-shot examples and records results. Extra options:

• exampleStore? — Custom store (default: globalExampleStore)
• maxExamples? — Max examples to inject (default: 3)

optimize(options)

• prompts — Array of prompt variations to test
• evalSet — Array of { input, expected } pairs
• scorer — Function (result, expected) => number returning 0-1
• thinkOptions — Base options for think() (prompt is overridden)