Add realtime SBD shore.nc4 processing pipeline#182
Merged
Conversation
## Summary
- New `sbd2netcdf.py`: discovers `shore.nc4` files by YYYYMM month
directory, reads each variable via the shore.nc4 parallel-time pattern,
interpolates all variables to a common 1S grid, computes `profile_number`
using the same scipy peak-detection as delayed-mode data, and writes
`shore_1S.nc` alongside each source file
- New `process_lrauv_sbd.py`: entry point that calls `SbdExtract.process()`,
concatenates the month's `shore_1S.nc` files, and drives `CreateProducts`
for quick-look plots
- `create_products.py`: resolve `bin_median_`/`bin_mean_` SBD variable
names; skip gulpers for realtime missions; fix `distnav` NaN propagation
from sparse GPS by pre-filling lat/lon gaps before distance computation;
use `np.nanmedian` for UTC offset; align scatter arrays to `distnav` length
- `resample.py`: extract `add_profile()` logic into a standalone
`compute_profile_number()` shared by both pipelines
## Test plan
- [ ] Run `process_lrauv_sbd.py --auv_name ahi --start 20260406 --end 20260412 -v 1`
and confirm `shore_1S.nc` files are written alongside each `shore.nc4`
- [ ] Confirm `shore_1S.nc` is readable with `xr.open_dataset()` and variables
follow `<group>_<variable>` naming with `bin_median_` prefix preserved
- [ ] Confirm `profile_number` is present and monotonically non-decreasing
- [ ] Confirm quick-look PNG plots are produced in the YYYYMM output directory
- [ ] Confirm delayed-mode `resample.py` pipeline still produces correct
`profile_number` (regression check)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
…engineering plots.
…ecks, and pyxis CBIT variable Add --current_month, --previous_month, --last_n_days date modes (mutually exclusive with --start); --auv_name now accepts multiple names and defaults to all known LRAUVs via ALL_LRAUV_NAMES in common_args.py; main() loops over vehicles Skip shore_1S.nc, per-log plots, and monthly plots when outputs are newer than their source files; --clobber bypasses all freshness checks Add CBIT/ampHoursUsed to SBD_PARMS; substitute cbit_amphoursused for elevator angle in the engineering plot for pyxis realtime only; add "Ah Used" short label; fix _resolve_label to read units from dataset attrs when no long_name is present Write shore_1S.nc as NETCDF4_CLASSIC to match the delayed-mode pipeline Use vehicle-relative paths in log messages; remove plt.show() from plot methods to prevent macOS backend timer crash; demote several per-variable warnings to debug
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sbd2netcdf.py: discoversshore.nc4files by YYYYMM month directory, reads each variable via the shore.nc4 parallel-time pattern, interpolates all variables to a common 1S grid, computesprofile_numberusing the same scipy peak-detection as delayed-mode data, and writesshore_1S.ncalongside each source fileprocess_lrauv_sbd.py: entry point that callsSbdExtract.process(), concatenates the month'sshore_1S.ncfiles, and drivesCreateProductsfor quick-look plotscreate_products.py: resolvebin_median_/bin_mean_SBD variable names; skip gulpers for realtime missions; fixdistnavNaN propagation from sparse GPS by pre-filling lat/lon gaps before distance computation; usenp.nanmedianfor UTC offset; align scatter arrays todistnavlengthresample.py: extractadd_profile()logic into a standalonecompute_profile_number()shared by both pipelinesTest plan
process_lrauv_sbd.py --auv_name ahi --start 20260406 --end 20260412 -v 1and confirmshore_1S.ncfiles are written alongside eachshore.nc4shore_1S.ncis readable withxr.open_dataset()and variables follow<group>_<variable>naming withbin_median_prefix preservedprofile_numberis present and monotonically non-decreasingresample.pypipeline still produces correctprofile_number(regression check)🤖 Generated with Claude Code