docs: add StatsD configuration details for worker container scaling#3261
docs: add StatsD configuration details for worker container scaling#3261CoderSufiyan wants to merge 2 commits into
Conversation
|
@CoderSufiyan is attempting to deploy a commit to the langfuse Team on Vercel. A member of the Team first needs to authorize it. |
|
|
||
| | Environment Variable | Default | Description | | ||
| | ----------------------- | ------------- | ------------------------------------------ | | ||
| | `DD_AGENT_HOST` | `localhost` | StatsD agent hostname or IP | |
There was a problem hiding this comment.
The description for
DD_AGENT_HOST says "StatsD agent hostname or IP", but since Langfuse uses DogStatsD format (not vanilla StatsD), this host must point to a DataDog Agent running a DogStatsD listener — a plain StatsD server would not understand the DogStatsD wire format and metrics would be silently dropped. Calling it a "StatsD agent" could lead operators to point it at a generic StatsD endpoint and wonder why nothing shows up in DataDog.
| | `DD_AGENT_HOST` | `localhost` | StatsD agent hostname or IP | | |
| | `DD_AGENT_HOST` | `localhost` | DataDog Agent hostname or IP (must run a DogStatsD listener) | |
Prompt To Fix With AI
This is a comment left during a code review.
Path: content/self-hosting/configuration/scaling.mdx
Line: 42
Comment:
The description for `DD_AGENT_HOST` says "StatsD agent hostname or IP", but since Langfuse uses DogStatsD format (not vanilla StatsD), this host must point to a **DataDog Agent** running a DogStatsD listener — a plain StatsD server would not understand the DogStatsD wire format and metrics would be silently dropped. Calling it a "StatsD agent" could lead operators to point it at a generic StatsD endpoint and wonder why nothing shows up in DataDog.
```suggestion
| `DD_AGENT_HOST` | `localhost` | DataDog Agent hostname or IP (must run a DogStatsD listener) |
```
How can I resolve this? If you propose a fix, please make it concise.|
|
||
| #### Configuring StatsD | ||
|
|
||
| Langfuse uses the [`dd-trace`](https://docs.datadoghq.com/tracing/trace_collection/automatic_instrumentation/dd_libraries/nodejs/) package (DataDog's tracing library) which includes a DogStatsD client. The client auto-configures using standard DataDog environment variables: |
There was a problem hiding this comment.
The link targets DataDog's APM/distributed-tracing instrumentation page, which focuses on trace collection rather than StatsD/metrics configuration. Readers following this link to learn how to configure
DD_AGENT_HOST or DD_DOGSTATSD_PORT will land on unrelated APM setup content. The DataDog DogStatsD docs (https://docs.datadoghq.com/developers/dogstatsd/) more directly describe the DogStatsD client and its environment variables.
| Langfuse uses the [`dd-trace`](https://docs.datadoghq.com/tracing/trace_collection/automatic_instrumentation/dd_libraries/nodejs/) package (DataDog's tracing library) which includes a DogStatsD client. The client auto-configures using standard DataDog environment variables: | |
| Langfuse uses the [`dd-trace`](https://docs.datadoghq.com/developers/dogstatsd/?tab=nodejs) package (DataDog's tracing library) which includes a DogStatsD client. The client auto-configures using standard DataDog environment variables: |
Prompt To Fix With AI
This is a comment left during a code review.
Path: content/self-hosting/configuration/scaling.mdx
Line: 38
Comment:
The link targets DataDog's APM/distributed-tracing instrumentation page, which focuses on trace collection rather than StatsD/metrics configuration. Readers following this link to learn how to configure `DD_AGENT_HOST` or `DD_DOGSTATSD_PORT` will land on unrelated APM setup content. The DataDog DogStatsD docs (`https://docs.datadoghq.com/developers/dogstatsd/`) more directly describe the DogStatsD client and its environment variables.
```suggestion
Langfuse uses the [`dd-trace`](https://docs.datadoghq.com/developers/dogstatsd/?tab=nodejs) package (DataDog's tracing library) which includes a DogStatsD client. The client auto-configures using standard DataDog environment variables:
```
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Closes #12645
Adds documentation for the StatsD/DogStatsD environment variables needed to configure queue metrics publishing from worker containers.
Changes
DD_AGENT_HOST,DD_DOGSTATSD_PORT,DD_ENV,DD_SERVICE)LANGFUSE_QUEUE_METRICS_ENABLED,LANGFUSE_QUEUE_METRICS_INTERVAL_MS,LANGFUSE_QUEUE_METRICS_SAMPLE_RATE)ENABLE_AWS_CLOUDWATCH_METRIC_PUBLISHINGRelated code:
packages/shared/src/server/instrumentation/index.ts- uses dd-trace DogStatsD clientworker/src/env.ts- defines LANGFUSE_QUEUE_METRICS_* env varsGreptile Summary
This PR adds a new "Configuring StatsD" sub-section to the worker scaling documentation, explaining how Langfuse's
dd-trace-based DogStatsD client is configured via environment variables.DD_AGENT_HOST,DD_DOGSTATSD_PORT,DD_ENV,DD_SERVICE) and notes they apply to both web and worker containers.LANGFUSE_QUEUE_METRICS_ENABLED,LANGFUSE_QUEUE_METRICS_INTERVAL_MS,LANGFUSE_QUEUE_METRICS_SAMPLE_RATE), scoped to the worker container only.Confidence Score: 4/5
Documentation-only change that adds helpful configuration details; no runtime code is affected.
The added tables and prose are accurate. Two small wording issues exist: the dd-trace hyperlink points at the APM instrumentation page rather than the DogStatsD-specific docs, and the DD_AGENT_HOST description says 'StatsD agent' where 'DataDog Agent' is the correct term. Neither would break anything at runtime, but they could send operators down the wrong path when configuring the DataDog Agent.
content/self-hosting/configuration/scaling.mdx — the dd-trace link target and the DD_AGENT_HOST description wording are the two spots worth a quick fix.
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD WC[Worker Container\nLANGFUSE_QUEUE_METRICS_*] -->|DogStatsD UDP| DA[DataDog Agent\nDD_AGENT_HOST:DD_DOGSTATSD_PORT] WEB[Web Container\nDD_AGENT_HOST / DD_DOGSTATSD_PORT] -->|APM traces| DA DA -->|Metrics & Traces| DD[(DataDog Platform)] DA -->|langfuse.queue.*| AS[Auto-Scaler\ne.g. KEDA / ECS ASG] AWS[AWS CloudWatch\nENABLE_AWS_CLOUDWATCH_METRIC_PUBLISHING] -->|Queue metrics| AS WC -->|Optional| AWS DD -->|Dashboard / Alerts| OPS[Operator]%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% flowchart TD WC[Worker Container\nLANGFUSE_QUEUE_METRICS_*] -->|DogStatsD UDP| DA[DataDog Agent\nDD_AGENT_HOST:DD_DOGSTATSD_PORT] WEB[Web Container\nDD_AGENT_HOST / DD_DOGSTATSD_PORT] -->|APM traces| DA DA -->|Metrics & Traces| DD[(DataDog Platform)] DA -->|langfuse.queue.*| AS[Auto-Scaler\ne.g. KEDA / ECS ASG] AWS[AWS CloudWatch\nENABLE_AWS_CLOUDWATCH_METRIC_PUBLISHING] -->|Queue metrics| AS WC -->|Optional| AWS DD -->|Dashboard / Alerts| OPS[Operator]Prompt To Fix All With AI
Reviews (1): Last reviewed commit: "docs: add StatsD configuration details f..." | Re-trigger Greptile