[bug-fix] Fix reinstall-overwrites-kept-config: preserve config on plain reinstall after --keep-config#3449
Open
github-actions[bot] wants to merge 12 commits into
Conversation
…all after --keep-config Apply the remediation from the bug assessment on issue #3427. Before the unconditional shutil.rmtree(dest_dir), scan dest_dir for any *-config.yml and *-config.local.yml files and hold their contents in memory. After shutil.copytree succeeds, write them back so user-customized values always win over the packaged defaults. This mirrors the existing backup/restore logic for the --force reinstall path but handles the case where remove --keep-config left config files behind in an unregistered extension directory. Refs #3427 Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Fixes extension config loss when reinstalling after remove --keep-config.
Changes:
- Rescues and restores preserved extension configuration files.
- Adds regression tests for standard and local configs.
Show a summary per file
| File | Description |
|---|---|
src/specify_cli/extensions/__init__.py |
Adds preserved-config rescue and restoration. |
tests/test_extensions.py |
Adds reinstall config-preservation tests. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 3
- Review effort level: Medium
…eserve file mode - Restore missing `test_install_force_without_existing` method declaration in tests/test_extensions.py so pytest collects it as a separate test. - Move stranded-config restoration to immediately after `copytree`, before command/skill/hook registration, so a failed registration step can't leave preserved configs permanently lost. - Store `(bytes, mode)` tuples instead of bare bytes when rescuing stranded configs, and reapply the original file mode after writing so permission bits (e.g. 0600 for credential files) are faithfully restored. Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)
Only preserve user/group read-write bits (mode & 0o660) to avoid restoring setuid, setgid, or world-writable permissions from a user-modified config file. Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)
…ackaged default config - Wrap shutil.copytree in a try/except BaseException so stranded configs rescued before rmtree are written back even if copytree fails mid-way (addresses review comment: configs were permanently lost on copy failure) - Add a packaged default config to extension_dir in the regression test so a naive 'restore only when absent' implementation would fail; assert the user's customized values beat the packaged defaults after reinstall Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Assisted-by: GitHub Copilot (model: gpt-5, autonomous)
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
…rt' and 'import from'' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
- Stage stranded config files to a durable rescue_staging_dir (extensions_dir/.rescue-staging-<id>) before rmtree so original bytes survive partial rmtree, copytree failure, or partial restore on retry. On retry the staging dir is detected and its content reused instead of whatever mix of packaged defaults and partial restores remains on disk. The staging dir is cleaned up only after every restore succeeds. - Fix CodeQL: change `import specify_cli.extensions as _ext_module` to `from specify_cli import extensions as _ext_module` in test file. Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
…up errors - Thread 14: Change except BaseException to except Exception in the staging fallback block so KeyboardInterrupt/SystemExit propagate correctly - Thread 15: Add explanatory comment to the bare pass in the chmod except block to satisfy static analysis - Thread 16: Reject a symlinked staging directory and only reload non-symlinked files whose names match the two recognised config suffixes - Thread 17: Create each staging file via os.open with mode 0600 and O_CREAT|O_EXCL before writing so preserved bytes are never transiently exposed to other local users - Thread 18: Remove ignore_errors=True from the final staging-dir cleanup so a failed rmtree propagates rather than silently leaving a stale backup that could be misread on the next retry Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
- Add `import stat` to imports - Use `stat.S_IMODE(mode)` before chmod in staging write (thread 20, line 1464) - Use `stat.S_IMODE(preserved_mode)` and make chmod best-effort in `_restore_stranded_config_file` (thread 18, line 1492) - Add `not rescue_staging_dir.is_symlink()` guard to cleanup (thread 19, line 1522) Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Comment on lines
+1414
to
+1418
| if ( | ||
| rescue_staging_dir.is_dir() | ||
| and not rescue_staging_dir.is_symlink() | ||
| and not self.registry.is_installed(manifest.id) | ||
| ): |
Contributor
There was a problem hiding this comment.
Fixed in 6ff00ec. Staging is now trusted only when an explicit .rescue-complete marker is present (written after every staged file is complete). Stale/partial staging dirs without the marker are removed before re-staging, and the marker is removed before the non-atomic rmtree cleanup, so an interrupt at any point can no longer cause a retry to skip dest_dir and drop configs.
Comment on lines
+1468
to
+1472
| except Exception: | ||
| # Staging failed — clean up any partial marker dir and | ||
| # fall back to in-memory protection only (same as the | ||
| # pre-staging behaviour). | ||
| shutil.rmtree(rescue_staging_dir, ignore_errors=True) |
Contributor
There was a problem hiding this comment.
Fixed in 6ff00ec. On staging failure the code now cleans up the partial staging dir and re-raises (aborting the install before the destructive rmtree) rather than continuing with only the in-memory copy. The handler now catches BaseException so an interrupt also aborts cleanly.
Comment on lines
+1459
to
+1463
| fd = os.open(str(staged), os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600) | ||
| try: | ||
| os.write(fd, content) | ||
| finally: | ||
| os.close(fd) |
Contributor
There was a problem hiding this comment.
Fixed in 6ff00ec. The write now loops over a memoryview until every byte is written, so a short os.write() can no longer leave a truncated durable backup.
…re, full os.write Assisted-by: GitHub Copilot (model: GPT-5.3-Codex, autonomous)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug fix — reinstall-overwrites-kept-config
Proposed fix for issue #3427, applying the remediation from the bug assessment.
Verdict: Valid · Severity: medium
Summary
When
specify extension remove <ext> --keep-configis used, the extension is unregistered but its*-config.ymlfiles survive in the extension directory. A subsequent plainspecify extension add <ext>unconditionally deleted that directory before copying the fresh extension in, silently discarding the preserved config. The fix rescues those stranded config files into memory before thermtreeand writes them back aftercopytree, so user-customized values always win over the packaged defaults.Changes
src/specify_cli/extensions/__init__.pyrmtree(dest_dir), collect any*-config.yml/*-config.local.ymlfiles from an unregistereddest_dirinto memory; restore them aftercopytreetests/test_extensions.pytest_reinstall_after_keep_config_preserves_configandtest_reinstall_after_keep_config_preserves_local_configTests Added or Updated
tests/test_extensions.py::TestInstallFromDirectory::test_reinstall_after_keep_config_preserves_config— pins that a customized*-config.ymlsurvives aremove --keep-config→ plain reinstall cycletests/test_extensions.py::TestInstallFromDirectory::test_reinstall_after_keep_config_preserves_local_config— same for*-config.local.ymloverride filesLocal Verification
--forcereinstall path, applying it to the previously-unhandled "unregistered but config-bearing directory" case.Deviations from Assessment
None. The implementation follows the preferred remediation exactly as described.
Risks & Review Notes
not self.registry.is_installed(manifest.id)guard ensures we only rescue configs when the extension is genuinely unregistered, avoiding picking up stale files from a different install.shutil.copytree, so packaged defaults are always superseded by the user's values — no risk of defaults silently winning.install_from_zip()delegates toinstall_from_directory(), so it is covered without additional changes.install_from_directory().Refs #3427 · cc
@grafvonbremove --keep-config#3427