Langfuse Features: Prompts, Tracing, Scores, Usage

A comprehensive guide to implementing Langfuse features for production-ready AI applications, covering prompt management, tracing, evaluation, and observability.

Overview

This guide covers:

• Prompt management with caching and versioning
• Distributed tracing with OpenTelemetry
• User feedback and scoring
• Usage tracking and analytics
• A/B testing and experimentation

Architecture Overview

Environment Setup

We use only three core Langfuse environment variables:

LANGFUSE_SECRET_KEY="sk-lf-..."
LANGFUSE_PUBLIC_KEY="pk-lf-..."
LANGFUSE_BASE_URL="https://cloud.langfuse.com"

Core Features

1. Prompt Management

1.1 Singleton Client Pattern

// src/langfuse/index.ts
let singleton: LangfuseClient | null = null;

export function getLangfuseClient(): LangfuseClient {
  if (!singleton) {
    singleton = new LangfuseClient({
      secretKey: LANGFUSE_SECRET_KEY,
      publicKey: LANGFUSE_PUBLIC_KEY,
      baseUrl: LANGFUSE_BASE_URL,
    });
  }
  return singleton;
}

Benefits:

• Single connection reuse
• Optimal connection pooling
• Consistent configuration

1.2 Prompt Fetching with Caching

export async function fetchLangfusePrompt(
  name: string,
  options: PromptFetchOptions = {},
) {
  const langfuse = getLangfuseClient();
  
  return await langfuse.prompt.get(name, {
    type: options.type,
    label: options.label,
    version: options.version,
    cacheTtlSeconds: options.cacheTtlSeconds ?? defaultCacheTtlSeconds(),
    fallback: options.fallback,
  });
}

Cache Strategy:

Environment	TTL	Behavior
Production	300s	Cached for 5 minutes
Development	0s	Always fetch latest

Reliability Features:

• Fallback prompts for first-fetch failures
• Stale-while-revalidate pattern
• Optional prewarming on startup

// Prewarm critical prompts
export async function prewarmPrompts(names: string[]) {
  await Promise.all(names.map((n) => fetchLangfusePrompt(n)));
}

2. Prompt Organization & Access Control

2.1 Folder-Style Naming

// Convert: "my-prompt" to "users/alice-example-com/my-prompt"
export function toUserPromptName(
  userIdOrEmail: string,
  shortName: string,
): string {
  const safe = userIdOrEmail
    .trim()
    .toLowerCase()
    .replace(/[^a-z0-9@._-]+/g, "-")
    .replace(/@/g, "-at-");
  return `users/${safe}/${shortName}`;
}

Access Control Rules:

1. User owns: users/{their-email}/*
2. Shared access: shared/*, public/*
3. Everything else: denied

export function assertPromptAccess(
  promptName: string,
  userIdOrEmail: string,
  allowedSharedPrefixes: string[] = ["shared/", "public/"],
) {
  const userPrefix = toUserPromptName(userIdOrEmail, "").replace(/\/$/, "");
  const isUserOwned = promptName.startsWith(userPrefix + "/");
  const isShared = allowedSharedPrefixes.some((p) => promptName.startsWith(p));

  if (!isUserOwned && !isShared) {
    throw new Error(`Access denied to prompt: ${promptName}`);
  }
}

3. Prompt Composability

Reuse prompt snippets across multiple prompts to maintain DRY principles.

Reference Format:

@@@langfusePrompt:name=PromptName|version=1@@@
@@@langfusePrompt:name=PromptName|label=production@@@

Helper Function:

export function composePromptRef(
  name: string,
  options: { version?: number; label?: string } = {},
): string {
  let ref = `@@@langfusePrompt:name=${name}`;
  if (options.version !== undefined) {
    ref += `|version=${options.version}`;
  }
  if (options.label) {
    ref += `|label=${options.label}`;
  }
  ref += "@@@";
  return ref;
}

Example Usage:

// Base prompt: "shared/system-instructions"
const systemRef = composePromptRef("shared/system-instructions", { 
  label: "production" 
});

// Composed prompt
const prompt = `
${systemRef}

Your task: Review code for bugs and performance issues.
`;

// Automatically resolved when fetched
const resolved = await resolveComposedPrompt(prompt);

4. Variables & Placeholders

4.1 Variables (Simple String Substitution)

const prompt = "Hello {{name}}, welcome to {{app}}!";

compilePrompt(prompt, {
  variables: {
    name: "Alice",
    app: "Nomadically"
  }
});
// Result: "Hello Alice, welcome to Nomadically!"

4.2 Message Placeholders (Chat Prompts)

// Prompt with placeholder
const chatPrompt = [
  { role: "system", content: "You are a helpful assistant." },
  { type: "placeholder", name: "conversation_history" },
  { role: "user", content: "{{user_question}}" }
];

compilePrompt(chatPrompt, {
  variables: { user_question: "What is TypeScript?" },
  placeholders: {
    conversation_history: [
      { role: "user", content: "Hi!" },
      { role: "assistant", content: "Hello! How can I help?" }
    ]
  }
});

5. Prompt Config

Store model parameters, tools, and schemas directly with prompt versions.

DeepSeek-Focused Config:

export type PromptConfig = {
  model?: string;
  temperature?: number;
  top_p?: number;
  max_tokens?: number;
  presence_penalty?: number;
  frequency_penalty?: number;
  response_format?: unknown;
  tools?: unknown[];
  tool_choice?: unknown;
  stop?: string[];
} & Record<string, unknown>;

Config Extraction:

export function extractPromptConfig(config: unknown): PromptConfig {
  // Validates and normalizes config
  // Enforces DeepSeek models only
  // Preserves custom keys
}

6. A/B Testing

Deterministic hash-based routing for consistent experiment assignment.

export function pickAbLabel(params: {
  seed: string;            // stable userId/sessionId
  labelA: string;          // "prod-a"
  labelB: string;          // "prod-b"
  splitA?: number;         // default 0.5
}): string {
  const u = hashToUnit(params.seed);
  return u < (params.splitA ?? 0.5) ? params.labelA : params.labelB;
}

Usage:

const label = pickAbLabel({
  seed: userId,     // Same user always gets same variant
  labelA: "prod-a",
  labelB: "prod-b",
  splitA: 0.5       // 50/50 split
});

const prompt = await fetchLangfusePrompt("shared/support-agent", { label });

7. DeepSeek Integration with Tracing

Complete OpenAI-compatible integration with full observability.

Key Features:

• Automatic prompt-to-trace linking
• User/session attribution
• Tag-based filtering
• Model parameter extraction from prompt config

export async function generateDeepSeekWithLangfuse(
  input: GenerateInput,
): Promise<string> {
  await initOtel();

  const langfusePrompt = await fetchLangfusePrompt(input.promptName, {
    type: input.promptType,
    label: input.label,
    cacheTtlSeconds: defaultCacheTtlSeconds(),
    fallback: /* ... */,
  });

  const compiled = compilePrompt(langfusePrompt, {
    variables: input.variables,
    placeholders: input.placeholders,
  });

  const cfg = extractPromptConfig(langfusePrompt.config);

  const traced = observeOpenAI(getDeepSeekClient(), {
    langfusePrompt,        // Links to prompt version
    userId: input.userId,
    sessionId: input.sessionId,
    tags: input.tags,
  });

  const res = await traced.chat.completions.create({
    model: cfg.model ?? "deepseek-chat",
    messages: compiled,
    temperature: cfg.temperature,
    // ... other params from config
  });

  return res.choices?.[0]?.message?.content ?? "";
}

8. Scores & Feedback

Capture user feedback and evaluation metrics.

export async function createScore(input: {
  traceId: string;
  observationId?: string;
  sessionId?: string;
  name: string;              // e.g. "helpfulness"
  value: number | string;    // boolean => 0/1
  dataType?: ScoreDataType;
  comment?: string;
  id?: string;               // idempotency key
}) {
  const langfuse = getLangfuseClient();

  langfuse.score.create({ /* ... */ });
  
  await langfuse.flush(); // Important for serverless
}

Use Cases:

• Thumbs up/down feedback
• Star ratings (1-5)
• Correctness evaluation
• Guardrail checks
• Custom metrics

9. Usage Tracking via Observations API

Replace in-memory logs with real production data.

export async function getRecentGenerationsForUser(params: {
  userId: string;
  limit?: number;
  environment?: string;
}): Promise<ObservationUsageItem[]> {
  const url = new URL(`${LANGFUSE_BASE_URL}/api/public/v2/observations`);
  
  url.searchParams.set("type", "GENERATION");
  url.searchParams.set("userId", params.userId);
  url.searchParams.set("limit", String(limit));
  url.searchParams.set("fields", "core,basic,prompt,time");

  const res = await fetch(url.toString(), {
    headers: {
      Authorization: `Basic ${Buffer.from(
        `${LANGFUSE_PUBLIC_KEY}:${LANGFUSE_SECRET_KEY}`
      ).toString("base64")}`,
    },
  });

  return parseObservations(await res.json());
}

Benefits:

• Real production data (no fake logs)
• Filtered by user, environment, time
• Includes prompt name & version
• Ready for billing, quotas, analytics

10. OpenTelemetry Setup

Required for @langfuse/openai integration.

Setup:

// src/otel/initOtel.ts
import { NodeSDK } from "@opentelemetry/sdk-node";
import { LangfuseSpanProcessor } from "@langfuse/otel";

let started = false;

export async function initOtel() {
  if (started) return;
  
  const sdk = new NodeSDK({
    spanProcessors: [new LangfuseSpanProcessor()],
  });
  
  await sdk.start();
  started = true;
}

// instrumentation.ts (Next.js hook)
export async function register() {
  if (process.env.NEXT_RUNTIME === "nodejs") {
    await initOtel();
  }
}

GraphQL API

All Langfuse features are accessible via GraphQL.

Query Prompts

query GetPrompt($name: String!, $label: String) {
  prompt(name: $name, label: $label, resolveComposition: true) {
    name
    version
    type
    chatMessages
    config
    labels
    tags
  }
}

Track Usage

query MyUsage($limit: Int) {
  myPromptUsage(limit: $limit) {
    promptName
    version
    usedAt
    traceId
  }
}

Create Prompts

mutation CreatePrompt($input: CreatePromptInput!) {
  createPrompt(input: $input) {
    name
    version
    config
  }
}

Best Practices

Recommended Practices

• Caching: Always use caching in production (300s TTL)
• Fallbacks: Provide fallback prompts for critical operations
• Prewarming: Prewarm prompts on server startup
• Tagging: Tag all generations with userId and sessionId
• A/B Testing: Use labels for experiments (prod-a/prod-b)
• Organization: Use folder-style naming convention
• DRY Principle: Compose prompts to avoid duplication
• Configuration: Extract config from prompts (avoid hardcoding)
• Serverless: Flush scores in serverless environments
• Production Data: Use Observations API for real usage tracking

Common Pitfalls to Avoid

• Bypassing caching in production
• Hardcoding model parameters
• Creating circular prompt references
• Skipping ACL checks
• Using in-memory logs for usage tracking
• Forgetting to call initOtel() before tracing
• Mixing manual prompt names with namespace convention

Performance Characteristics

Feature	Latency	Cache Hit Rate	Notes
Prompt Fetch (cached)	~1ms	95%+	In-process cache
Prompt Fetch (miss)	~50-100ms	-	Network + DB
Prompt Compilation	<1ms	-	Pure computation
Score Creation	~20-50ms	-	Async, buffered
Observations API	~100-200ms	-	Paginated queries
Composed Prompt (3 refs)	~150ms	80%+	Parallel fetches

Security Considerations

• Credentials: Never expose LANGFUSE_SECRET_KEY to client-side code
• Access Control: Always validate access with assertPromptAccess()
• User Isolation: Folder-style naming prevents cross-user data leaks
• Rate Limits: Observations API has a limit of 1000 records per query
• Idempotency: Use score IDs to prevent duplicate feedback submission

Monitoring & Debugging

In Langfuse Dashboard

1. Traces - Filter by userId, sessionId, tags
2. Prompts - View usage per version/label
3. Scores - Aggregate feedback by name
4. Sessions - Track multi-turn conversations
5. Datasets - Export for evals (future)

Local Development

# Disable caching for instant updates
NODE_ENV=development

# Check OTel initialization
# Look for "LangfuseSpanProcessor initialized" in logs

# Verify prompt fetches
# Check Network tab for Langfuse API calls

# Test composability
# Use resolveComposedPrompt() directly

Migration Guide

Upgrading from a previous implementation:

Step 1: Install Dependencies

pnpm add @langfuse/openai @langfuse/otel

Step 2: Setup OpenTelemetry

• Add instrumentation.ts for OTel initialization
• Configure LangfuseSpanProcessor

Step 3: Update Client

• Replace Langfuse with LangfuseClient
• Update prompt fetching to use the caching API

Step 4: Refactor Code

• Switch to folder-style naming convention
• Replace in-memory usage tracking with Observations API
• Add langfusePrompt parameter to all observeOpenAI calls

Resources

• Langfuse Prompt Management
• Langfuse OpenAI Integration
• Observations API
• DeepSeek API Docs
• A/B Testing Guide
• Prompt Composability

Troubleshooting

Common issues and solutions:

Build Errors

• Check build logs for TypeScript errors
• Verify all required dependencies are installed

Configuration Issues

• Verify environment variables are set correctly
• Confirm LANGFUSE_SECRET_KEY and LANGFUSE_PUBLIC_KEY are valid

Runtime Issues

• Confirm OTel is initialized before first LLM call
• Check that initOtel() is called in instrumentation.ts

Monitoring

• Review Langfuse dashboard for traces and errors
• Check browser Network tab for API errors
• Verify API responses and status codes