feat: add optional prometheus metrics endpoint.

[pantierra]

Adds opt-in Prometheus metrics support via prometheus-fastapi-instrumentator, controlled by ENABLE_METRICS / enable_metrics. Metrics remain disabled by default and the dependency is available through the metrics extra.

[codecov]

Codecov Report

❌ Patch coverage is 83.87097% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.34%. Comparing base (c89f650) to head (dca2c8b).

Files with missing lines	Patch %	Lines
src/stac_auth_proxy/app.py	75.00%	2 Missing and 1 partial ⚠️
src/stac_auth_proxy/metrics.py	89.47%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #186      +/-   ##
==========================================
- Coverage   89.39%   89.34%   -0.06%     
==========================================
  Files          29       30       +1     
  Lines        1273     1304      +31     
  Branches      170      174       +4     
==========================================
+ Hits         1138     1165      +27     
- Misses         92       96       +4     
  Partials       43       43              

Flag	Coverage Δ
unittests	89.34% <83.87%> (-0.06%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[alukach]

Left some thoughts.

As stated in developmentseed/eoapi-k8s#541 (comment), I'm generally in favor of this idea.

Before we merge this, we will want to ensure that the PR includes an update to our documentation. At the very least, we should update the config docs. We may also want to include something long-form within /tips.

One thing that stands out to me is how I don't really understand how this works in various deployment scenarios. Based on the fact that we're not configuring any datastore with this plugin, I assume these metrics are held within memory. So how would that work in a multi-node environment? Or what about a serverless environment where an AWS Lambda may or may not be warmed?

[alukach]

Currently, we just auto-include optional endpoints in the default public setting:

stac-auth-proxy/src/stac_auth_proxy/config.py

Lines 101 to 110 in c89f650

	public_endpoints: EndpointMethods = {
	r"^/$": ["GET"],
	r"^/api.html$": ["GET"],
	r"^/api$": ["GET"],
	r"^/conformance$": ["GET"],
	r"^/docs/oauth2-redirect": ["GET"],
	r"^/healthz": ["GET"],
	r"^/_mgmt/ping": ["GET"],
	r"^/_mgmt/health": ["GET"],
	}

Any reason to not do this?

[pantierra]

I added it this way to handle the dependency as conditional. But If we move it to a normal dependency it would be much simpler. I like this more. Now it would be enabled by default, is this ok?

[alukach]

It's interesting that /metrics path is never specified. Is this a hard requirement? I assume it's a common convention for prometheus? It would be nice to tuck it behind /_mgmt so it can sit alongside the other meta-endpoints, but if that's super atypical then it's not a big deal to leave it as it currently is written.

[pantierra]

It is the normal convention, but no problem moving it under /_mgmt as there should always be a configuration option when scraping the metrics.

[pantierra]

Can you have a look again, please? If you agree with this approach (enabled by default, dependency added in main bulk) I am happy to add the documentation.

[alukach]

@pantierra sorry for the back-and-forth.

• serving metrics from /_mgmt/metrics: 👍
• having /_mgmt/metrics in default public_endpoints: 👍

However, I do still think this should be an optional dependency

[pantierra]

I wasn't sure how you want to do the conditional. Inspired from stac-fastapi-elasticsearch-opensearch I made it to add the metrics endpoint if the dependency extra metrics is installed. And we somehow have to deal with adding/removing it from the public_endpoints depending on the conditional.

I gave it a try. Feel free to add over this branch if you prefer to change it. Or just let me know, I am fine to change it to you think is best.

[pantierra]

One thing that stands out to me is how I don't really understand how this works in various deployment scenarios. Based on the fact that we're not configuring any datastore with this plugin, I assume these metrics are held within memory. So how would that work in a multi-node environment? Or what about a serverless environment where an AWS Lambda may or may not be warmed?

Yes, metrics are held in memory. In a multi-node setup, Prometheus scrapes each pod separately with possible aggregation when querying them. Serverless is trickier as the metrics need to be pushed. Afaik there are several approaches, but i don't have experience with any.

[pantierra]

Based on input developmentseed/eoAPI#193 (comment), adjusted the structure of the metrics to be more STACish.