Skip to content

docs: add StatsD configuration details for worker container scaling#3261

Open
CoderSufiyan wants to merge 2 commits into
langfuse:mainfrom
CoderSufiyan:docs/statsd-configuration
Open

docs: add StatsD configuration details for worker container scaling#3261
CoderSufiyan wants to merge 2 commits into
langfuse:mainfrom
CoderSufiyan:docs/statsd-configuration

Conversation

@CoderSufiyan

@CoderSufiyan CoderSufiyan commented Jul 8, 2026

Copy link
Copy Markdown

Closes #12645

Adds documentation for the StatsD/DogStatsD environment variables needed to configure queue metrics publishing from worker containers.

Changes

  • Clarified that StatsD uses DogStatsD format via dd-trace
  • Added table of standard DataDog environment variables (DD_AGENT_HOST, DD_DOGSTATSD_PORT, DD_ENV, DD_SERVICE)
  • Added table of Langfuse-specific queue metric env vars (LANGFUSE_QUEUE_METRICS_ENABLED, LANGFUSE_QUEUE_METRICS_INTERVAL_MS, LANGFUSE_QUEUE_METRICS_SAMPLE_RATE)
  • Noted that DD_* vars apply to both web and worker, while queue metric vars are worker-only
  • Kept existing note about ENABLE_AWS_CLOUDWATCH_METRIC_PUBLISHING

Related code:

  • packages/shared/src/server/instrumentation/index.ts - uses dd-trace DogStatsD client
  • worker/src/env.ts - defines LANGFUSE_QUEUE_METRICS_* env vars

Greptile Summary

This PR adds a new "Configuring StatsD" sub-section to the worker scaling documentation, explaining how Langfuse's dd-trace-based DogStatsD client is configured via environment variables.

  • Introduces a table of standard DataDog environment variables (DD_AGENT_HOST, DD_DOGSTATSD_PORT, DD_ENV, DD_SERVICE) and notes they apply to both web and worker containers.
  • Adds a second table of Langfuse-specific queue-metric variables (LANGFUSE_QUEUE_METRICS_ENABLED, LANGFUSE_QUEUE_METRICS_INTERVAL_MS, LANGFUSE_QUEUE_METRICS_SAMPLE_RATE), scoped to the worker container only.

Confidence Score: 4/5

Documentation-only change that adds helpful configuration details; no runtime code is affected.

The added tables and prose are accurate. Two small wording issues exist: the dd-trace hyperlink points at the APM instrumentation page rather than the DogStatsD-specific docs, and the DD_AGENT_HOST description says 'StatsD agent' where 'DataDog Agent' is the correct term. Neither would break anything at runtime, but they could send operators down the wrong path when configuring the DataDog Agent.

content/self-hosting/configuration/scaling.mdx — the dd-trace link target and the DD_AGENT_HOST description wording are the two spots worth a quick fix.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    WC[Worker Container\nLANGFUSE_QUEUE_METRICS_*] -->|DogStatsD UDP| DA[DataDog Agent\nDD_AGENT_HOST:DD_DOGSTATSD_PORT]
    WEB[Web Container\nDD_AGENT_HOST / DD_DOGSTATSD_PORT] -->|APM traces| DA
    DA -->|Metrics & Traces| DD[(DataDog Platform)]
    DA -->|langfuse.queue.*| AS[Auto-Scaler\ne.g. KEDA / ECS ASG]
    AWS[AWS CloudWatch\nENABLE_AWS_CLOUDWATCH_METRIC_PUBLISHING] -->|Queue metrics| AS
    WC -->|Optional| AWS
    DD -->|Dashboard / Alerts| OPS[Operator]
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    WC[Worker Container\nLANGFUSE_QUEUE_METRICS_*] -->|DogStatsD UDP| DA[DataDog Agent\nDD_AGENT_HOST:DD_DOGSTATSD_PORT]
    WEB[Web Container\nDD_AGENT_HOST / DD_DOGSTATSD_PORT] -->|APM traces| DA
    DA -->|Metrics & Traces| DD[(DataDog Platform)]
    DA -->|langfuse.queue.*| AS[Auto-Scaler\ne.g. KEDA / ECS ASG]
    AWS[AWS CloudWatch\nENABLE_AWS_CLOUDWATCH_METRIC_PUBLISHING] -->|Queue metrics| AS
    WC -->|Optional| AWS
    DD -->|Dashboard / Alerts| OPS[Operator]
Loading
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
content/self-hosting/configuration/scaling.mdx:42
The description for `DD_AGENT_HOST` says "StatsD agent hostname or IP", but since Langfuse uses DogStatsD format (not vanilla StatsD), this host must point to a **DataDog Agent** running a DogStatsD listener — a plain StatsD server would not understand the DogStatsD wire format and metrics would be silently dropped. Calling it a "StatsD agent" could lead operators to point it at a generic StatsD endpoint and wonder why nothing shows up in DataDog.

```suggestion
| `DD_AGENT_HOST`         | `localhost`   | DataDog Agent hostname or IP (must run a DogStatsD listener) |
```

### Issue 2 of 2
content/self-hosting/configuration/scaling.mdx:38
The link targets DataDog's APM/distributed-tracing instrumentation page, which focuses on trace collection rather than StatsD/metrics configuration. Readers following this link to learn how to configure `DD_AGENT_HOST` or `DD_DOGSTATSD_PORT` will land on unrelated APM setup content. The DataDog DogStatsD docs (`https://docs.datadoghq.com/developers/dogstatsd/`) more directly describe the DogStatsD client and its environment variables.

```suggestion
Langfuse uses the [`dd-trace`](https://docs.datadoghq.com/developers/dogstatsd/?tab=nodejs) package (DataDog's tracing library) which includes a DogStatsD client. The client auto-configures using standard DataDog environment variables:
```

Reviews (1): Last reviewed commit: "docs: add StatsD configuration details f..." | Re-trigger Greptile

Greptile also left 2 inline comments on this PR.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@vercel

vercel Bot commented Jul 8, 2026

Copy link
Copy Markdown

@CoderSufiyan is attempting to deploy a commit to the langfuse Team on Vercel.

A member of the Team first needs to authorize it.

@dosubot dosubot Bot added the size:S This PR changes 10-29 lines, ignoring generated files. label Jul 8, 2026
@CLAassistant

CLAassistant commented Jul 8, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@dosubot dosubot Bot added the documentation Improvements or additions to documentation label Jul 8, 2026

| Environment Variable | Default | Description |
| ----------------------- | ------------- | ------------------------------------------ |
| `DD_AGENT_HOST` | `localhost` | StatsD agent hostname or IP |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The description for DD_AGENT_HOST says "StatsD agent hostname or IP", but since Langfuse uses DogStatsD format (not vanilla StatsD), this host must point to a DataDog Agent running a DogStatsD listener — a plain StatsD server would not understand the DogStatsD wire format and metrics would be silently dropped. Calling it a "StatsD agent" could lead operators to point it at a generic StatsD endpoint and wonder why nothing shows up in DataDog.

Suggested change
| `DD_AGENT_HOST` | `localhost` | StatsD agent hostname or IP |
| `DD_AGENT_HOST` | `localhost` | DataDog Agent hostname or IP (must run a DogStatsD listener) |
Prompt To Fix With AI
This is a comment left during a code review.
Path: content/self-hosting/configuration/scaling.mdx
Line: 42

Comment:
The description for `DD_AGENT_HOST` says "StatsD agent hostname or IP", but since Langfuse uses DogStatsD format (not vanilla StatsD), this host must point to a **DataDog Agent** running a DogStatsD listener — a plain StatsD server would not understand the DogStatsD wire format and metrics would be silently dropped. Calling it a "StatsD agent" could lead operators to point it at a generic StatsD endpoint and wonder why nothing shows up in DataDog.

```suggestion
| `DD_AGENT_HOST`         | `localhost`   | DataDog Agent hostname or IP (must run a DogStatsD listener) |
```

How can I resolve this? If you propose a fix, please make it concise.


#### Configuring StatsD

Langfuse uses the [`dd-trace`](https://docs.datadoghq.com/tracing/trace_collection/automatic_instrumentation/dd_libraries/nodejs/) package (DataDog's tracing library) which includes a DogStatsD client. The client auto-configures using standard DataDog environment variables:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The link targets DataDog's APM/distributed-tracing instrumentation page, which focuses on trace collection rather than StatsD/metrics configuration. Readers following this link to learn how to configure DD_AGENT_HOST or DD_DOGSTATSD_PORT will land on unrelated APM setup content. The DataDog DogStatsD docs (https://docs.datadoghq.com/developers/dogstatsd/) more directly describe the DogStatsD client and its environment variables.

Suggested change
Langfuse uses the [`dd-trace`](https://docs.datadoghq.com/tracing/trace_collection/automatic_instrumentation/dd_libraries/nodejs/) package (DataDog's tracing library) which includes a DogStatsD client. The client auto-configures using standard DataDog environment variables:
Langfuse uses the [`dd-trace`](https://docs.datadoghq.com/developers/dogstatsd/?tab=nodejs) package (DataDog's tracing library) which includes a DogStatsD client. The client auto-configures using standard DataDog environment variables:
Prompt To Fix With AI
This is a comment left during a code review.
Path: content/self-hosting/configuration/scaling.mdx
Line: 38

Comment:
The link targets DataDog's APM/distributed-tracing instrumentation page, which focuses on trace collection rather than StatsD/metrics configuration. Readers following this link to learn how to configure `DD_AGENT_HOST` or `DD_DOGSTATSD_PORT` will land on unrelated APM setup content. The DataDog DogStatsD docs (`https://docs.datadoghq.com/developers/dogstatsd/`) more directly describe the DogStatsD client and its environment variables.

```suggestion
Langfuse uses the [`dd-trace`](https://docs.datadoghq.com/developers/dogstatsd/?tab=nodejs) package (DataDog's tracing library) which includes a DogStatsD client. The client auto-configures using standard DataDog environment variables:
```

How can I resolve this? If you propose a fix, please make it concise.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants