-
Notifications
You must be signed in to change notification settings - Fork 20
fix(libdd-data-pipeline): cargo clippy fix with all lints #2013
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
Eldolfin
wants to merge
13
commits into
main
Choose a base branch
from
oscarld/restrictive-clippy-on-data-pipeline
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
db1433f
fix(libdd-data-pipeline): add workspace lints + cargo clippy --fix
Eldolfin 695a164
fix(libdd-data-pipeline): clippy match_same_arms, missing_fields_in_d…
Eldolfin ae2e333
fix(libdd-data-pipeline): clippy assigning_clones, or_fun_call, redun…
Eldolfin 1529bd9
fix: fmt and add + Symc for AgentInfoFetcher<C>
Eldolfin 8c6c222
fix(libdd-data-pipeline): clippy unused_async, unnecessary_wraps, nee…
Eldolfin f06d577
fix(libdd-data-pipeline): clippy allow deferred lints with reasons; f…
Eldolfin abb7b5a
fix(libdd-data-pipeline): remove redundant cfg_attr deny lints supers…
Eldolfin eb2ad3f
fix(libdd-data-pipeline): clippy MSRV 1.84.1 — future_not_send allows…
Eldolfin 4266ae7
fix(libdd-data-pipeline): clarify future_not_send allow reasons with …
Eldolfin d775896
fix(libdd-data-pipeline): update future_not_send FIXME — regression s…
Eldolfin 2d86776
fix: weird lints
Eldolfin 21aff79
fix: fmt
Eldolfin 6babd5f
fix: missing Eq in SqlObfuscationMode
Eldolfin File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -108,3 +108,6 @@ fips = [ | |
| ] | ||
| test-utils = [] | ||
| regex-lite = ["libdd-common/regex-lite"] | ||
|
|
||
| [lints] | ||
| workspace = true | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -34,6 +34,10 @@ pub enum FetchInfoStatus { | |
| /// If either the agent state hash or container tags hash is different from the current one: | ||
| /// - Return a `FetchInfoStatus::NewState` of the info struct | ||
| /// - Else return `FetchInfoStatus::SameState` | ||
| #[allow( | ||
| clippy::future_not_send, | ||
| reason = "FIXME: remove when MSRV > 1.84.1 — regression in 1.84.1 patch: future_not_send fires spuriously on async fns holding non-Send/Sync generics across .await even when the future is never required to be Send; fixed in 1.85.0" | ||
| )] | ||
| async fn fetch_info_with_state_and_container_tags<C: HttpClientCapability + SleepCapability>( | ||
| info_endpoint: &Endpoint, | ||
| current_state_hash: Option<&str>, | ||
|
|
@@ -59,11 +63,15 @@ async fn fetch_info_with_state_and_container_tags<C: HttpClientCapability + Slee | |
| Ok(FetchInfoStatus::NewState(info)) | ||
| } | ||
|
|
||
| /// Fetch info from the given info_endpoint and compare its state to the current state hash. | ||
| /// Fetch info from the given `info_endpoint` and compare its state to the current state hash. | ||
| /// | ||
| /// If the state hash is different from the current one: | ||
| /// - Return a `FetchInfoStatus::NewState` of the info struct | ||
| /// - Else return `FetchInfoStatus::SameState` | ||
| #[allow( | ||
| clippy::future_not_send, | ||
| reason = "FIXME: remove when MSRV > 1.84.1 — regression in 1.84.1 patch: future_not_send fires spuriously on async fns holding non-Send/Sync generics across .await even when the future is never required to be Send; fixed in 1.85.0" | ||
| )] | ||
| pub async fn fetch_info_with_state<C: HttpClientCapability + SleepCapability>( | ||
| info_endpoint: &Endpoint, | ||
| current_state_hash: Option<&str>, | ||
|
|
@@ -92,6 +100,10 @@ pub async fn fetch_info_with_state<C: HttpClientCapability + SleepCapability>( | |
| /// # Ok(()) | ||
| /// # } | ||
| /// ``` | ||
| #[allow( | ||
| clippy::future_not_send, | ||
| reason = "FIXME: remove when MSRV > 1.84.1 — regression in 1.84.1 patch: future_not_send fires spuriously on async fns holding non-Send/Sync generics across .await even when the future is never required to be Send; fixed in 1.85.0" | ||
| )] | ||
| pub async fn fetch_info<C: HttpClientCapability + SleepCapability>( | ||
| info_endpoint: &Endpoint, | ||
| ) -> Result<Box<AgentInfo>> { | ||
|
|
@@ -104,8 +116,12 @@ pub async fn fetch_info<C: HttpClientCapability + SleepCapability>( | |
|
|
||
| /// Fetch and hash the response from the agent info endpoint. | ||
| /// | ||
| /// Returns a tuple of (state_hash, response_body_bytes, container_tags_hash). | ||
| /// Returns a tuple of (`state_hash`, `response_body_bytes`, `container_tags_hash`). | ||
| /// The hash is calculated using SHA256 to match the agent's calculation method. | ||
| #[allow( | ||
| clippy::future_not_send, | ||
| reason = "FIXME: remove when MSRV > 1.84.1 — regression in 1.84.1 patch: future_not_send fires spuriously on async fns holding non-Send/Sync generics across .await even when the future is never required to be Send; fixed in 1.85.0" | ||
| )] | ||
| async fn fetch_and_hash_response<C: HttpClientCapability + SleepCapability>( | ||
| info_endpoint: &Endpoint, | ||
| ) -> Result<(String, bytes::Bytes, Option<String>)> { | ||
|
|
@@ -120,28 +136,28 @@ async fn fetch_and_hash_response<C: HttpClientCapability + SleepCapability>( | |
| // Runtime-agnostic timeout: race the request against a capability-driven | ||
| // sleep instead of `tokio::time::timeout`, which requires a tokio reactor | ||
| // (not available on wasm where we run on the JS event loop). | ||
| let res = tokio::select! { | ||
| let response = tokio::select! { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it really clippy or unrelated claude changes ? 😛 |
||
| biased; | ||
| result = client.request(req) => result?, | ||
| _ = sleeper.sleep(timeout) => { | ||
| () = sleeper.sleep(timeout) => { | ||
| return Err(anyhow!("Request to /info timed out after {:?}", timeout)); | ||
| } | ||
| }; | ||
|
|
||
| // Extract the Datadog-Container-Tags-Hash header | ||
| let container_tags_hash = res | ||
| let container_tags_hash = response | ||
| .headers() | ||
| .get("Datadog-Container-Tags-Hash") | ||
| .and_then(|v| v.to_str().ok()) | ||
| .map(|s| s.to_string()); | ||
| .map(std::string::ToString::to_string); | ||
|
|
||
| let body_data = res.into_body(); | ||
| let body_data = response.into_body(); | ||
| let hash = format!("{:x}", Sha256::digest(&body_data)); | ||
|
|
||
| Ok((hash, body_data, container_tags_hash)) | ||
| } | ||
|
|
||
| /// Fetch the info endpoint and update an ArcSwap keeping it up-to-date. | ||
| /// Fetch the info endpoint and update an `ArcSwap` keeping it up-to-date. | ||
| /// | ||
| /// This type implements [`libdd_shared_runtime::Worker`] and is intended to be driven by a worker | ||
| /// runner such as [`libdd_shared_runtime::SharedRuntime`]. | ||
|
|
@@ -191,7 +207,7 @@ async fn fetch_and_hash_response<C: HttpClientCapability + SleepCapability>( | |
| /// `C` is the capability bundle, see [`HttpClientCapability`] and [`SleepCapability`]. | ||
| /// Leaf crates pin it to a concrete type. | ||
| #[derive(Debug)] | ||
| pub struct AgentInfoFetcher<C: HttpClientCapability + SleepCapability> { | ||
| pub struct AgentInfoFetcher<C: HttpClientCapability + SleepCapability + Sync> { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @Aaalibaba42 is that really ok to change a sync? This doesn't look like a clippy trivial fix. |
||
| info_endpoint: Endpoint, | ||
| refresh_interval: Duration, | ||
| trigger_rx: Option<mpsc::Receiver<()>>, | ||
|
|
@@ -201,13 +217,13 @@ pub struct AgentInfoFetcher<C: HttpClientCapability + SleepCapability> { | |
| _phantom: PhantomData<C>, | ||
| } | ||
|
|
||
| impl<C: HttpClientCapability + SleepCapability> AgentInfoFetcher<C> { | ||
| impl<C: HttpClientCapability + SleepCapability + Sync> AgentInfoFetcher<C> { | ||
| /// Return a new `AgentInfoFetcher` fetching the `info_endpoint` on each `refresh_interval` | ||
| /// and updating the stored info. | ||
| /// | ||
| /// Returns a tuple of (fetcher, trigger_component) where: | ||
| /// - `fetcher`: The AgentInfoFetcher to be run in a background task | ||
| /// - `response_observer`: The ResponseObserver component for checking HTTP responses | ||
| /// Returns a tuple of (fetcher, `trigger_component`) where: | ||
| /// - `fetcher`: The `AgentInfoFetcher` to be run in a background task | ||
| /// - `response_observer`: The `ResponseObserver` component for checking HTTP responses | ||
| pub fn new(info_endpoint: Endpoint, refresh_interval: Duration) -> (Self, ResponseObserver) { | ||
| // The trigger channel stores a single message to avoid multiple triggers. | ||
| let (trigger_tx, trigger_rx) = mpsc::channel(1); | ||
|
|
@@ -244,7 +260,7 @@ impl<C: HttpClientCapability + SleepCapability + MaybeSend + Sync + 'static> Wor | |
| if AGENT_INFO_CACHE.load().is_none() { | ||
| return; | ||
| } | ||
| self.trigger().await | ||
| self.trigger().await; | ||
| } | ||
|
|
||
| async fn trigger(&mut self) { | ||
|
|
@@ -261,7 +277,7 @@ impl<C: HttpClientCapability + SleepCapability + MaybeSend + Sync + 'static> Wor | |
| } | ||
| } | ||
| // Regular periodic fetch timer | ||
| _ = sleeper.sleep(self.refresh_interval) => {} | ||
| () = sleeper.sleep(self.refresh_interval) => {} | ||
| } | ||
| } | ||
| None => { | ||
|
|
@@ -275,19 +291,27 @@ impl<C: HttpClientCapability + SleepCapability + MaybeSend + Sync + 'static> Wor | |
| // Release the IoStack waker stored in trigger_rx by waking the channel and drain the | ||
| // message to avoid a spurious fetch on restart. If the channel is not empty then it has | ||
| // already been waked. | ||
| if self.trigger_rx.as_ref().is_some_and(|rx| rx.is_empty()) { | ||
| if self | ||
| .trigger_rx | ||
| .as_ref() | ||
| .is_some_and(tokio::sync::mpsc::Receiver::is_empty) | ||
| { | ||
| let _ = self.trigger_tx.try_send(()); | ||
| self.drain(); | ||
| }; | ||
| } | ||
| } | ||
|
|
||
| async fn run(&mut self) { | ||
| self.fetch_and_update().await; | ||
| } | ||
| } | ||
|
|
||
| impl<C: HttpClientCapability + SleepCapability> AgentInfoFetcher<C> { | ||
| impl<C: HttpClientCapability + SleepCapability + Sync> AgentInfoFetcher<C> { | ||
| /// Fetch agent info and update cache if needed | ||
| #[allow( | ||
| clippy::future_not_send, | ||
| reason = "FIXME: remove when MSRV > 1.84.1 — regression in 1.84.1 patch: future_not_send fires spuriously on async fns holding non-Send/Sync generics across .await even when the future is never required to be Send; fixed in 1.85.0" | ||
| )] | ||
| async fn fetch_and_update(&self) { | ||
| let current_info = AGENT_INFO_CACHE.load(); | ||
| let current_hash = current_info.as_ref().map(|info| info.state_hash.as_str()); | ||
|
|
@@ -306,7 +330,7 @@ impl<C: HttpClientCapability + SleepCapability> AgentInfoFetcher<C> { | |
| AGENT_INFO_CACHE.store(Some(Arc::new(*new_info))); | ||
| } | ||
| Ok(FetchInfoStatus::SameState) => { | ||
| debug!("Agent info is up-to-date") | ||
| debug!("Agent info is up-to-date"); | ||
| } | ||
| Err(err) => { | ||
| warn!(?err, "Error while fetching /info"); | ||
|
|
@@ -325,8 +349,9 @@ pub struct ResponseObserver { | |
| } | ||
|
|
||
| impl ResponseObserver { | ||
| /// Create a new ResponseObserver with the given channel sender. | ||
| pub fn new(trigger_tx: mpsc::Sender<()>) -> Self { | ||
| /// Create a new `ResponseObserver` with the given channel sender. | ||
| #[must_use] | ||
| pub const fn new(trigger_tx: mpsc::Sender<()>) -> Self { | ||
| Self { trigger_tx } | ||
| } | ||
|
|
||
|
|
@@ -344,11 +369,11 @@ impl ResponseObserver { | |
| let current_state = AGENT_INFO_CACHE.load(); | ||
| if current_state.as_ref().map(|s| s.state_hash.as_str()) != Some(state_str) { | ||
| match self.trigger_tx.try_send(()) { | ||
| Ok(_) => {} | ||
| Err(mpsc::error::TrySendError::Full(_)) => { | ||
| Ok(()) => {} | ||
| Err(mpsc::error::TrySendError::Full(())) => { | ||
| debug!("Response observer channel full, fetch has already been triggered"); | ||
| } | ||
| Err(mpsc::error::TrySendError::Closed(_)) => { | ||
| Err(mpsc::error::TrySendError::Closed(())) => { | ||
| debug!("Agent info fetcher channel closed, unable to trigger refresh"); | ||
| } | ||
| } | ||
|
|
@@ -583,10 +608,8 @@ mod single_threaded_tests { | |
| .body(r#"{"version":"1"}"#); | ||
| }); | ||
| let endpoint = Endpoint::from_url(server.url("/info").parse().unwrap()); | ||
| let (fetcher, _response_observer) = AgentInfoFetcher::<NativeCapabilities>::new( | ||
| endpoint.clone(), | ||
| Duration::from_millis(100), | ||
| ); | ||
| let (fetcher, _response_observer) = | ||
| AgentInfoFetcher::<NativeCapabilities>::new(endpoint, Duration::from_millis(100)); | ||
| assert!(agent_info::get_agent_info().is_none()); | ||
| let shared_runtime = SharedRuntime::new().unwrap(); | ||
| let _ = shared_runtime.spawn_worker(fetcher, true).unwrap(); | ||
|
|
@@ -652,6 +675,8 @@ mod single_threaded_tests { | |
| #[cfg_attr(miri, ignore)] | ||
| #[test] | ||
| fn test_agent_info_trigger_different_state() { | ||
| const MAX_ATTEMPTS: u32 = 500; | ||
| const SLEEP_DURATION_MS: u64 = 10; | ||
| let server = MockServer::start(); | ||
| let mock = server.mock(|when, then| { | ||
| when.path("/info"); | ||
|
|
@@ -685,9 +710,6 @@ mod single_threaded_tests { | |
| response_observer.check_response(&response); | ||
|
|
||
| // Wait for the fetch to complete | ||
| const MAX_ATTEMPTS: u32 = 500; | ||
| const SLEEP_DURATION_MS: u64 = 10; | ||
|
|
||
| let mut attempts = 0; | ||
| while mock.calls() == 0 && attempts < MAX_ATTEMPTS { | ||
| attempts += 1; | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this is a clippy bug, I wonder if we shouldn't just disable it for the whole file and call it a day... that will be less place to update when we bump to 1.87, and this will happen soon ™️