Migrate Langfuse v3 to v4

Langfuse v4 moves to an observations-first data model built on a new, wide, and (mostly) immutable ClickHouse table. This eliminates joins and deduplication at read time and optimizes for the most performant ClickHouse access patterns. Initial table loads for large amounts of data go from seconds to milliseconds, and dashboard load times for large projects improve by 10x or more over longer time ranges.

The new data model also unlocks a set of new features for self-hosted deployments: full-text search across inputs, outputs, and metadata, the filter search bar, monitors and alerts, and the significantly faster Observations API v2 and Metrics API v2.

Langfuse Cloud has been running on this data model since early 2026. Read the technical deep dive and the overview of the new experience for more background.

Migration overview

The migration consists of three independent steps. Each step leaves your deployment in a stable state, so you can schedule them separately, pause in between, and retain your current behavior as long as you need it:

1. Upgrade your infrastructure: bring ClickHouse to a v4-compatible version while still on Langfuse v3.
2. Upgrade the server to Langfuse v4: retain your current ingestion behavior with the legacy or dual write mode. All schema migrations are applied automatically.
3. Move to the new data model: upgrade your SDKs, migrate API consumers, evaluators, and exports, bring historic data into the new data model, and cut over to the v4 defaults.

For historic data, you choose between two options in step 3: an automated backfill rewrites all existing data into the new tables in the background (plan for roughly 3x disk headroom in ClickHouse), or a retention-based rollover skips the backfill entirely by keeping the dual write active until your data retention window has rolled over.

New deployments skip all of this; see new deployments.

What changes

Breaking changes at a glance

The feature availability matrix is the canonical overview of what each server version supports. In summary, once your deployment runs the v4 defaults (write mode events_only), the following v3 surfaces stop working:

• SDKs: Python SDK v2 and older and JS/TS SDK v3 and older are rejected at ingestion; see SDK upgrade.
• Ingestion APIs: the legacy batch ingestion endpoints are replaced by OpenTelemetry-based ingestion; see API changes.
• Read APIs: the deprecated read endpoints for traces, observations, sessions, scores, metrics, and dataset runs are replaced by newer APIs; see API changes.
• Evaluators: trace-level and legacy-dataset LLM-as-a-Judge evaluators stop running; see evaluations.
• Exports: the "Traces and observations (legacy)" export source stops producing data; see batch exports and integrations.

All of these keep working while you run the legacy or dual write mode, so you decide when the breaking changes take effect during your migration.

The new data model

Langfuse v4 introduces two new ClickHouse tables that replace traces and observations as the source for all reads:

• events_full: the immutable, full-fidelity event table. Every LLM call, tool execution, and agent step is one row, including all trace-level attributes (user, session, tags, release, version) directly on the row.
• events_core: a lightweight, query-optimized projection of events_full with truncated input/output/metadata, populated automatically via a materialized view. It serves table and chart queries.

Traces are represented as root spans within the same table instead of a separate entity. Existing traces are converted into virtual root spans (type SPAN, with span ID t-<trace_id>) so that historic data remains browsable.

The table schemas and all data migrations ship as regular migrations with Langfuse v4; there is no manual schema work.

What does not change

The infrastructure architecture is unchanged. Langfuse v4 runs on the same components as v3: web and worker containers, PostgreSQL, ClickHouse, Redis, and S3/blob storage. No new services are introduced, and all data in PostgreSQL (projects, users, prompts, datasets, scores configuration, etc.) is unaffected. This is a change on the data-model level, not the infrastructure level.

UI changes

The main exploration surface is a single Observations view, the same experience that is live on Langfuse Cloud. Traces do not go away: every trace is represented by its root observation, and the table opens filtered to Is Root Observation = true by default, which shows exactly one row per trace like the previous traces table. Remove the filter to explore all observations, and use trace_id like any other filter column (like session_id or user_id) to group related observations. See working with observations for common views and workflows, and dashboard changes for how charts behave on the new model.

Scores and comments that were previously attached to a trace are shown on the corresponding root observation.

API changes

Langfuse v4 serves reads from the new APIs that were introduced alongside the data model. The following legacy endpoints are unavailable on deployments that run the default write mode (events_only), that is, all new deployments and migrated deployments after the cutover. While you run the legacy or dual write mode during your migration, these endpoints keep working as before.

Ingestion, replaced by OpenTelemetry-based ingestion:

Reads, replaced by the Observations API v2:

Scores, replaced by the Scores API v3:

Metrics, replaced by the Metrics API v2:

Dataset runs, replaced by the experiments API (GET /api/public/experiments, GET /api/public/experiment-items) for reads and the Experiment runner SDK for writes. The legacy POST /api/public/dataset-run-items endpoint is deprecated and must not be used with Langfuse v4.

Create experiment data with the Experiment runner SDK (Python and JS/TS), which applies the required experiment semantics automatically; from other languages, send experiment traces to POST /api/public/otel/v1/traces with the experiment attributes set.

Evaluations

Trace-based and legacy-dataset-based LLM-as-a-Judge evaluators perform lookups on the old tables. They are marked as Legacy in the UI, keep working during the legacy and dual write phases, and stop running after the cutover to events_only. Migrate them to observation-level evaluators before the cutover; see the evaluator upgrade guide.

Batch exports and integrations

The blob storage, PostHog, and Mixpanel integrations read their data through an export source that is configured per integration in the project settings. The source "Traces and observations (legacy)" (LEGACY_TRACES_OBSERVATIONS) exports from the old tables and stops producing data after the cutover to events_only. Switch each integration to the "Enriched observations" source before the cutover; see the export upgrade path and the changelog.

Step 1: Upgrade your infrastructure

Langfuse v4 raises the minimum infrastructure versions:

Upgrade ClickHouse before the server upgrade. Langfuse v3 releases are fully compatible with current ClickHouse versions, so you can perform the ClickHouse upgrade on your existing v3 deployment and operate it as long as you like before continuing. See the ClickHouse deployment guide for version and operations guidance.

If you plan to use the automated backfill for historic data in step 3, also make sure the ClickHouse disks can handle roughly 3x the current data volume: data is copied into the events tables and, for the observations table, additionally into an intermediate format that is cleaned up at the end.

Step 2: Upgrade the server to Langfuse v4

Perform a regular server upgrade to the latest Langfuse v4 release. All ClickHouse schema migrations (the new events tables) are applied automatically on startup. Deployments that ran the Langfuse v4 preview (v3-tagged images with the preview environment variables) are handled as well: existing tables are detected and skipped.

The only decision to make is the write mode you start with. It controls which tables ingestion writes to, and thereby how much of the v3 behavior you retain. The legacy and dual modes are migration utilities, not permanent operating modes: they will be removed in an upcoming major version, so plan to complete the cutover to events_only as part of your migration.

LANGFUSE_MIGRATION_V4_WRITE_MODE=dual
# Only if native OTel producers rely on server-side attribute propagation:
LANGFUSE_MIGRATION_V4_NATIVE_OTEL_BEHAVIOUR=dual_write
# Recommended until dual_write behaviour is confirmed to be working
# (see step 3):
LANGFUSE_BACKGROUND_MIGRATION_V4_ENABLE_HISTORIC_BACKFILL=false
# Optional: set false to keep the v3 read experience for all users while
# dual-writing (hides the toggle, disables the v2 APIs); defaults to true.
# Flip to true or remove overwrite, if you want to allow users to opt-in to v4 experience.
LANGFUSE_MIGRATION_V4_ALLOW_PREVIEW_OPT_IN=false

LANGFUSE_MIGRATION_V4_WRITE_MODE=legacy
LANGFUSE_MIGRATION_V4_NATIVE_OTEL_BEHAVIOUR=dual_write
# Keep the v4 read paths (UI toggle, v2 APIs) off; the events tables
# are not written in legacy mode:
LANGFUSE_MIGRATION_V4_ALLOW_PREVIEW_OPT_IN=false
LANGFUSE_BACKGROUND_MIGRATION_V4_ENABLE_HISTORIC_BACKFILL=false

Step 3: Move to the new data model

If you started with the dual or legacy write mode, complete the migration with the following steps, at your own pace.

# Remove these overrides (or set them explicitly):
# LANGFUSE_MIGRATION_V4_WRITE_MODE=events_only
# LANGFUSE_MIGRATION_V4_NATIVE_OTEL_BEHAVIOUR=direct
# LANGFUSE_MIGRATION_V4_ALLOW_PREVIEW_OPT_IN=true

TRUNCATE TABLE traces;
TRUNCATE TABLE observations;

-- On clustered ClickHouse deployments:
TRUNCATE TABLE traces ON CLUSTER default;
TRUNCATE TABLE observations ON CLUSTER default;

New deployments

Fresh installs need none of the above: deploy Langfuse v4 following the regular guides (Docker Compose, Kubernetes (Helm)) on the required infrastructure versions, and instrument with the compatible SDKs. The defaults apply, and there is no historic data to migrate.

Configuration reference

On Langfuse v4, the new behavior is the default. These variables exist to retain v3 behaviors while your migration is in flight; they will be removed in an upcoming major version:

How the dual write works

The dual write bridges older SDKs onto the new data model:

• Incoming events from older SDKs are written to traces/observations as before, plus into a staging table (observations_batch_staging) partitioned into 3-minute windows.
• A worker job periodically picks up completed partitions, joins them with the traces table to propagate trace-level attributes (user, session, tags, release, version, metadata), and inserts the result into events_full. This is why data from older SDKs appears with a ~10 minute delay in the UI and APIs.
• Compatible SDKs (Python ≥ 4.7.0, JS/TS ≥ 5.4.0) propagate attributes client-side and write into the new tables directly, without delay, which also reduces the ClickHouse load of the dual-write phase. Python SDK 4.0.0–4.6.x and JS/TS SDK 5.0.0–5.3.x still ingest on v4 but can appear with the same ~10 minute delay until upgraded.
• Native OpenTelemetry spans follow LANGFUSE_MIGRATION_V4_NATIVE_OTEL_BEHAVIOUR: with dual_write, spans go through the staging pipeline unless they carry the x-langfuse-ingestion-version: 4 header; with direct (the v4 default), all OTel spans are written directly, assuming client-side propagation.

Staging partitions are kept for 48 hours, which gives you a multi-day grace period to recover the propagation job after an incident without losing staging data. You can monitor the propagation pipeline via the worker health endpoint: GET /api/health?failIfEventPropagationStuck=true (worker container, port 3030) returns 503 if the job has not made progress within LANGFUSE_EVENT_PROPAGATION_STUCK_THRESHOLD_MINUTES (default 15). The check passes on deployments where the dual write does not run, so it is safe to keep configured after the cutover. See health and readiness endpoints for probe configuration.

How the historic backfill works

The historic backfill rewrites existing traces and observations (and dataset run items) into the new events tables so that data ingested before the dual write became active is visible alongside new data. It runs as a chain of background migrations on the worker, without downtime and without impact on new ingestion.

Enable it only after the dual write is active and confirmed healthy: the backfill runs once, and anything ingested after its cutoff but before the dual write started would be permanently missing from the new tables.

What runs, and in what order

The backfill is an ordered chain. Each step only starts after its predecessor finished successfully; a failure halts the rest of the chain rather than running on partial data.

What you will observe

• Historic data appears gradually, newest first, as each batch completes, not all at once. For a period you may see only recent (dual-written) data while older data is still being rewritten.
• There is no downtime and no impact on new ingestion, though a large backfill adds load and temporary storage (the scratch table, up to ~3x data volume) to your ClickHouse cluster until step 5 cleans it up.
• The migrations are resumable: they can be interrupted (e.g. by a deployment) and continue where they left off.
• Because the new tables use a ReplacingMergeTree, re-runs and overlaps with the dual write are de-duplicated; the most recently written version wins.

Track progress on the background migrations page in the UI; see background migrations for monitoring and troubleshooting.

Rollback and safety

• The v4 schema migrations are additive (new tables only). While you run the legacy or dual write mode, the old traces/observations tables keep receiving all data, so you can roll the server back to the latest v3 release without data loss.
• The historic backfill never modifies the old tables; it only reads from them.
• Once you complete the cutover to events_only, new data lands only in the new tables. Rolling back to a v3 read path from that point would miss the data written since the switch; treat the cutover as the point of commitment.

FAQ

Do I need to do anything special if I ran the Langfuse v4 preview on a v3-tagged image? No. If you adopted the events tables early by running the Langfuse v4 preview (a v3-tagged image with the preview environment variables), the v4 migrations detect the existing tables and skip their creation. Your deployment upgrades like any other; keep the environment variables you set for the preview until you complete the cutover.

Is it safe to upgrade the SDKs before the server? Yes. Python SDK v4 and JS/TS SDK v5 are fully compatible with Langfuse v3 servers. Two caveats: client.api.observations.* and client.api.metrics.* point to the new v2 endpoints which are not available on v3 servers (use client.api.legacy.* in the meantime), and trace input/output must be set explicitly if you depart from the default root-span behavior.

Can I stay on the legacy or dual write mode long-term? No. They are transition modes that will be removed in an upcoming major version: legacy forgoes all v4 performance improvements, and dual doubles the write load and storage costs. Complete the cutover once your producers are migrated.

Do I need new infrastructure components? No. The infrastructure is identical to v3 (web/worker, PostgreSQL, ClickHouse, Redis, S3); only the minimum/recommended versions changed, see step 1.

How do I fetch a trace by ID now? Use the Observations API v2 with a traceId filter. Providing a time range in addition is recommended for performance, but not required.

What happens to trace-level scores and comments? They are attached to the root observation of the trace and remain visible in the UI and APIs.

Why is historic data missing right after enabling the backfill? The historic backfill rewrites data gradually, newest first. Depending on your data volume, it can take from minutes to days until all historic data is visible.

Why does data from some SDKs appear with a ~10 minute delay? Data from SDKs older than Python 4.7.0 / JS/TS 5.4.0 (and OTel exporters without x-langfuse-ingestion-version: 4) flows through the dual-write pipeline, which processes micro-batches with a delay. Upgrading to those minimums (or setting the header) removes the delay.

Langfuse does not start after I set the migration variables. Langfuse validates the migration configuration on startup and exits on invalid combinations (for example, events_only write mode together with dual_write OTel behavior or with LANGFUSE_MIGRATION_V4_ALLOW_PREVIEW_OPT_IN=false). Double-check values and exact variable names, including the uppercase V4.

Support

If you experience any issues during the migration, please:

1. Ask in the v4 GitHub Discussion; we use it as the central thread for the v4 release and answer questions there.
2. Create a GitHub Issue for bugs.
3. Enterprise customers can reach out to support directly.

If you experience any issues when self-hosting Langfuse, please:

1. Check out Troubleshooting & FAQ page.
2. Use Ask AI to get instant answers to your questions.
3. Ask the maintainers on GitHub Discussions.
4. Create a bug report or feature request on GitHub.