Observability

Turn on LangSmith in one block

LangChain auto-wires tracing when these env vars are present in process.env — the runtime never reads them, only declares them in the base env schema so they show up in qiforge-cli env output.

LANGSMITH_TRACING=true
LANGSMITH_API_KEY=ls-...
LANGSMITH_PROJECT=my-oracle
LANGSMITH_ENDPOINT=https://api.smith.langchain.com   # optional, defaults to LangSmith cloud

No code changes needed. Set the vars and every LLM call, tool invocation, and middleware hook produces a span in the LangSmith UI. Unset them to turn it off.

What QiForge gives you out of the box

Three signals — that's the whole surface. Bring your own log aggregator and metrics.

1. LangSmith traces — env-driven, covered above.
2. Plugin status events — app.onPluginStatusChange(handler).
3. Plugin-scoped logger — ctx.logger (boot) and rtCtx.logger (per-request).

Subscribe to plugin status events

app.onPluginStatusChange((event) => {
  Logger.log(
    `[plugin] ${event.plugin} ${event.from} → ${event.to}` +
      (event.reason ? ` (${event.reason})` : ''),
  );
});

app.onPluginStatusChange((event) => {
  if (event.plugin === 'matrix' && event.to === 'loaded') {
    // safe to send messages, mint UCANs, etc.
  }
});

The event shape:

{
  plugin: string;
  from: 'pending' | 'loaded' | 'failed';
  to:   'pending' | 'loaded' | 'failed';
  reason?: string;
}

Capture background errors with onError

app.onError((err, source) => {
  Logger.error(`[runtime] ${source}: ${err.message}`);
});

source is a short label like 'matrix-init'. Use this for background failures (Matrix sync errors, key setup failures). Boot-time errors (env validation, manifest validation, dependency cycles) throw from createOracleApp itself — your try/catch around the call sees them.

Log the boot resolution snapshot

app.plugins.status() returns the loader's resolution snapshot — log it once at boot so operators see what came up.

const status = app.plugins.status();
Logger.log(`[boot] loaded plugins: ${status.loaded.join(', ') || '(none)'}`);
if (status.excluded.length > 0) {
  Logger.log(
    '[boot] excluded plugins:',
    status.excluded.map((e) => `${e.plugin} (${e.reason})`).join(', '),
  );
}

Shape of the snapshot — the PluginStatusReport defined in packages/oracle-runtime/src/bootstrap/create-oracle-app.ts:

{
  loaded: string[];                                  // plugin names that came up
  excluded: Array<{
    plugin: string;
    reason: string;                                  // e.g. "auto-detect precondition not met (COMPOSIO_API_KEY)"
  }>;
  softDepGaps: Array<{ plugin: string; missing: string }>;
}

Use the plugin-scoped logger

Every PluginContext and RuntimeContext carries a logger field auto-prefixed with the plugin's name — see Logger in types.ts.

override getTools(ctx: PluginContext): PluginTool[] {
  ctx.logger.log('Building weather tools');
  // ...
}

handler: async (args, rtCtx: RuntimeContext) => {
  rtCtx.logger.log(`Looking up ${args.city}`);
  // ...
}

The Logger interface:

Use NestJS's default Logger format in development (pnpm dev shows it nicely); pipe to your aggregator in production. Override the global logger with createOracleApp({ logger }) if you want a custom format — see createOracleApp reference.

UI-facing tool-call events

For showing the user what the agent did, the runtime emits typed events via rtCtx.emit. The bundled clients (Portal, Slack, Matrix) consume these and render the right UI — you only emit them yourself when writing a custom client.

rtCtx.emit.toolCall({ /* ... */ });
rtCtx.emit.actionCall({ /* ... */ });
rtCtx.emit.renderComponent({ /* ... */ });
rtCtx.emit.reasoning({ /* ... */ });
rtCtx.emit.browserToolCall({ /* ... */ });
rtCtx.emit.router({ /* ... */ });
rtCtx.emit.messageCacheInvalidation({ /* ... */ });

The seven event types are defined on RuntimeContext.emit in types.ts.

Where to read next